00010000:00010000:2.0:1787739048.585469:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aa8a5 lrc: 5/0,0 mode: PR/PR res: [0x200000400:0xae:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0947f9 expref: 17 pid: 6523 timeout: 735 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739048.585509:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1303 00000100:00100000:2.0:1787739048.585518:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652382208:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739048.585532:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739048.585538:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aa8a5 lrc: 5/0,0 mode: PR/PR res: [0x200000400:0xae:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0947f9 expref: 17 pid: 6523 timeout: 735 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.585550:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aa8a5 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xae:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0947f9 expref: 17 pid: 6523 timeout: 735 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.585573:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xae:0x0].0x0 (ffffa07304e53400) 00010000:00010000:2.0:1787739048.585578:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aa8cf lrc: 3/0,1 mode: --/EX res: [0x200000400:0xae:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.585586:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aa8cf lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xae:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.585593:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aa8cf lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xae:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.585610:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739048.585659:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652382208:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 142us (356us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739048.585666:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1303 00000100:00100000:2.0:1787739048.586398:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665459200 00000100:00100000:2.0:1787739048.586403:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665459200 00000100:00100000:2.0:1787739048.586463:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2212 00000100:00100000:2.0:1787739048.586471:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304000780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+5:6525:x1874579665459200:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787739048.586485:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739048.586499:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aa8dd lrc: 2/0,0 mode: --/EX res: [0x240000401:0xae:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aa8d6 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.586518:0:18535:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1aa8b3 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xae:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094800 expref: 7 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.586527:0:18535:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aa8dd lrc: 4/0,0 mode: --/EX res: [0x240000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aa8d6 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.586549:0:18535:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1aa8b3 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xae:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094800 expref: 7 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.586561:0:18535:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1aa8b3 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xae:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094800 expref: 7 pid: 6524 timeout: 735 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739048.586570:0:18535:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f78780 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:18535:1874579665459328:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739048.586616:0:18535:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b95c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739048.592311:0:18535:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f78780 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:18535:1874579665459328:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739048.592345:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aa8dd lrc: 3/0,0 mode: --/EX res: [0x240000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aa8d6 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.592357:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xae:0x0].0x0 (ffffa07305eaeb00) 00010000:00010000:2.0:1787739048.592361:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739048.592364:0:18535:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aa8dd lrc: 3/0,0 mode: --/EX res: [0x240000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aa8d6 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.592377:0:18535:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aa8dd lrc: 3/0,0 mode: --/EX res: [0x240000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aa8d6 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.592384:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633da00, rc 0) 00000100:00100000:2.0:1787739048.592473:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304000780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6525:x1874579665459200:12345-0@lo:101:mdt00_002.0 Request processed in 6002us (6164us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739048.592481:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2212 00000020:00080000:2.0:1787739048.595335:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 327 committed batchid 326 00000020:00080000:2.0:1787739048.595393:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 326 00000004:00200000:2.0:1787739048.620637:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14e2:0x0]: returning size 4096 00010000:00010000:2.0:1787739048.620669:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aa8cf lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.620680:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aa8cf lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.620687:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aa8cf lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.620694:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739048.620698:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aa8cf lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.620706:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aa8cf lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.620722:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xae:0x0].0x0 (ffffa07304e53400) 00010000:00010000:2.0:1787739048.620725:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739048.620730:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aa8cf lrc: 2/0,0 mode: --/EX res: [0x200000400:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.620736:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aa8cf lrc: 0/0,0 mode: --/EX res: [0x200000400:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.620751:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1aa8d6 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aa8dd expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.620759:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1aa8d6 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aa8dd expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.620776:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1aa8d6 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aa8dd expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.620797:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739048.620806:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xae:0x0] 00010000:00010000:2.0:1787739048.620814:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aa8c8 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14e2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.620820:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aa8c8 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x14e2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.620825:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aa8c8 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14e2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.620830:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739048.620832:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aa8c8 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14e2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.620838:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aa8c8 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x14e2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.620849:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14e2:0x0].0x0 (ffffa07304e53a00) 00010000:00010000:2.0:1787739048.620851:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739048.620854:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aa8c8 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14e2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.620859:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aa8c8 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14e2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.620876:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aa8ba lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.620882:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aa8ba lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.621055:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aa8ba lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.621062:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aa8ba lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.621067:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739048.621070:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aa8ba lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.621076:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aa8ba lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.621086:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730608bd00) 00010000:00010000:2.0:1787739048.621088:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739048.621091:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aa8ba lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.621100:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aa8ba lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739048.621125:0:6525:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009ea15be3 reply state 00000000f72fcb9e transno 4294983257 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1aa8c1 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x735d7c61 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.621137:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1aa8c1 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x735d7c61 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.621145:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x735d7c61 (ffffa07305eaee00) 00010000:00010000:2.0:1787739048.621147:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739048.621150:0:6525:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787739048.621238:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17340:x1874579652382080:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 40184us (40392us total) trans 4294983257 rc 0/0 00000100:00100000:2.0:1787739048.621247:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23020 00000100:00100000:2.0:1787739048.675664:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652382720 00000100:00100000:2.0:1787739048.675674:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652382720 00000100:00100000:2.0:1787739048.675734:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23023 00000100:00100000:2.0:1787739048.675744:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073040012c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17341:x1874579652382720:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739048.675773:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739048.675795:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1aa90e lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094815 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.675853:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1aa915 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.675866:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1aa915 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.675879:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1aa915 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739048.675935:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:2.0:1787739048.675941:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739048.675975:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1aa90e lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094815 expref: 19 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.675995:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1aa915 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094815 expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.676004:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5800) 00010000:00010000:2.0:1787739048.676007:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739048.676010:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633da00, rc 0) 00000100:00100000:2.0:1787739048.676060:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073040012c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17341:x1874579652382720:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 318us (548us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739048.676067:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23023 00000100:00100000:2.0:1787739048.704248:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665460608 00000100:00100000:2.0:1787739048.704252:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665460608 00000100:00100000:2.0:1787739048.704289:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 994 00000100:00100000:2.0:1787739048.704296:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665460608:12345-0@lo:105: 00010000:00010000:2.0:1787739048.704307:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1aa91c lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aa923 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.704316:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1aa91c lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aa923 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.704329:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1aa91c lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aa923 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.704336:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1aa91c lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1aa923 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.704343:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1aa91c lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1aa923 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.704350:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1aa91c lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1aa923 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.704367:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07441a1da00) 00000100:00100000:2.0:1787739048.704405:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665460608:12345-0@lo:105: Request processed in 108us (250us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739048.704436:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 994 00000100:00100000:2.0:1787739048.704777:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665460736 00000100:00100000:2.0:1787739048.704781:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665460736 00000100:00100000:2.0:1787739048.704816:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2218 00000100:00100000:2.0:1787739048.704822:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665460736:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739048.704836:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739048.704850:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1aa931 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x5d1254db bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aa92a expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.704868:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1aa931 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x5d1254db bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aa92a expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.704883:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1aa931 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x5d1254db bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aa92a expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.704889:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x5d1254db (ffffa074419d5700) 00010000:00010000:2.0:1787739048.704892:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739048.704894:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c000, rc 0) 00000100:00100000:2.0:1787739048.704923:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665460736:12345-0@lo:101:mdt00_004.0 Request processed in 102us (231us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739048.704928:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2218 00000100:00100000:2.0:1787739048.705234:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665460864 00000100:00100000:2.0:1787739048.705238:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665460864 00000100:00100000:2.0:1787739048.705271:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2219 00000100:00100000:2.0:1787739048.705277:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665460864:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739048.705349:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665460864:12345-0@lo:1000:mdt00_004.0 Request processed in 71us (182us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739048.705355:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2219 00000100:00100000:2.0:1787739048.715287:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665462016:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739048.715314:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1aa946 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aa94d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.715344:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1aa946 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aa94d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739048.715369:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002d0c0900 reply state 00000000c1b7320c transno 4294968248 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aa93f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xac:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.715388:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aa93f lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xac:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.715395:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xac:0x0].0x0 (ffffa0730608b100) 00010000:00010000:2.0:1787739048.715397:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739048.715401:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787739048.715407:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1aa946 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aa94d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.715439:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1aa946 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aa94d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739048.715451:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xac:0x0] 00000004:00010000:2.0:1787739048.715458:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002d0c0900 reply state 00000000c1b7320c transno 4294968248 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1aa938 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x14f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.715466:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1aa938 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x14f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.715472:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14f:0x0].0x0 (ffffa0730608b000) 00010000:00010000:2.0:1787739048.715475:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739048.715477:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:2.0:1787739048.715482:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x14f:0x0]: returning size 4096 00000004:00080000:2.0:1787739048.715491:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739048.715500:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1aa91c lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aa923 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.715507:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1aa91c lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aa923 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.715519:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1aa91c lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aa923 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.715540:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739048.715544:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739048.715548:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1aa92a lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x5d1254db bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aa931 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.715556:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1aa92a lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x5d1254db bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aa931 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739048.715642:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17341:x1874579652382976:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 22939us (23134us total) trans 4294968248 rc 0/0 00000100:00100000:2.0:1787739048.715649:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23024 00000100:00100000:2.0:1787739048.721817:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665462144 00000100:00100000:2.0:1787739048.721825:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665462144 00000100:00100000:2.0:1787739048.721882:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1100 00000100:00100000:2.0:1787739048.721901:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+11:3652:x1874579665462144:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739048.721915:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739048.721938:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1aa923 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aa91c expref: 11 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.721954:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5800) 00010000:00010000:2.0:1787739048.721963:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739048.721999:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+10:3652:x1874579665462144:12345-0@lo:103:ldlm_bl.0 Request processed in 100us (544us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739048.722010:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1100 00000020:00080000:2.0:1787739048.780282:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 327 00000004:00080000:2.0:1787739048.780353:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07410b950c0 with version 623 00000020:00080000:2.0:1787739048.780365:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 327 cancel update log [0x240000401:0x3:0x0].324: rc = 0 00000020:00080000:2.0:1787739048.780451:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 327 cancel update log [0x200000400:0x3:0x0].325: rc = 0 00000020:00080000:2.0:1787739048.780461:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 327 00000004:00080000:2.0:1787739048.780469:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 623 rpc_version 623 00000020:00100000:2.0:1787739048.780501:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739048.780512:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07305f78780 x1874579665462656/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739048.780534:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f78780 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665462656:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739048.780598:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739048.780629:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b95c00 going to sleep for 11 seconds 00000020:00080000:2.0:1787739048.781505:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 328 committed batchid 327 00000020:00080000:2.0:1787739048.781541:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 327 00000100:00100000:2.0:1787739048.781862:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f78780 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665462656:0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1787739048.782246:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aa962 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x5d1254db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.782254:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aa962 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x5d1254db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739048.782359:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665462784:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739048.782393:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739048.782443:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07406576a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739048.783185:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665462784:0@lo:1000:mdt00_003.0 00010000:00010000:2.0:1787739048.783229:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1aa969 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x14f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.783242:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1aa969 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x14f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.783252:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1aa969 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x14f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739048.783274:0:7944:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1aa969 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x14f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.783288:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1aa95b lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.783295:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1aa95b lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.783301:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1aa95b lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.783307:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1aa95b lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.783314:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739048.783317:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1aa95b lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.783324:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1aa95b lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.783333:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07304e53800) 00010000:00010000:2.0:1787739048.783336:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739048.783340:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1aa95b lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.783346:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1aa95b lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.783359:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aa962 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x5d1254db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.783365:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aa962 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x5d1254db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.783371:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aa962 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x5d1254db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.783378:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609bc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739048.783380:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aa962 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x5d1254db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.783385:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aa962 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x5d1254db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.783391:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x5d1254db (ffffa074419d5700) 00010000:00010000:2.0:1787739048.783393:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739048.783396:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aa962 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x5d1254db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.783401:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aa962 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x5d1254db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.783477:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1aa954 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09481c expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.783502:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1aa969 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x14f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09481c expref: 14 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.783509:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14f:0x0].0x0 (ffffa074419d5800) 00010000:00010000:2.0:1787739048.783511:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739048.783513:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d800, rc 0) 00000100:00100000:2.0:1787739048.783564:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17343:x1874579652383488:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 7596us (7771us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739048.783570:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23026 00000100:00100000:2.0:1787739048.792659:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652383616 00000100:00100000:2.0:1787739048.792668:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652383616 00000100:00100000:2.0:1787739048.792733:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23027 00000100:00100000:2.0:1787739048.792744:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17343:x1874579652383616:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739048.792778:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739048.792800:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1aa970 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x14f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094823 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.792861:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1aa977 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x14f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.792873:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1aa977 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x14f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.792884:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1aa977 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x14f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739048.792988:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x14f:0x0]: returning size 4096 00000004:00200000:2.0:1787739048.792994:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739048.793030:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1aa970 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x14f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094823 expref: 11 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.793048:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1aa977 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x14f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094823 expref: 10 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.793056:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14f:0x0].0x0 (ffffa0730608b000) 00010000:00010000:2.0:1787739048.793059:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739048.793062:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d200, rc 0) 00000100:00100000:2.0:1787739048.793136:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17343:x1874579652383616:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 394us (568us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739048.793143:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23027 00000100:00100000:2.0:1787739048.808271:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652383872 00000100:00100000:2.0:1787739048.808281:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652383872 00000100:00100000:2.0:1787739048.808343:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23029 00000100:00100000:2.0:1787739048.808353:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17343:x1874579652383872:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739048.808371:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739048.808393:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aa98c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xac:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094831 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.808493:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1aa993 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xac:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.808508:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1aa94d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aa946 expref: 5 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.808516:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1aa993 lrc: 4/1,0 mode: --/PR res: [0x200000401:0xac:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.808549:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1aa94d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xac:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1aa946 expref: 5 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739048.808564:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665462912:0@lo:104: 00000100:00100000:2.0:1787739048.808612:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739048.808643:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07406576c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739048.824507:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652384256 00000100:00100000:2.0:1787739048.824517:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652384256 00000100:00100000:2.0:1787739048.824578:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23030 00000100:00100000:2.0:1787739048.824588:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17343:x1874579652384256:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:2.0:1787739048.824660:0:6523:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739048.824686:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1aa977 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x14f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094823 expref: 11 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.824706:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x14f:0x0].0x0 (ffffa0730608b000) 00010000:00010000:2.0:1787739048.824710:0:6523:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739048.824769:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1aa99a lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.824777:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1aa99a lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.824788:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1aa99a lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739048.824808:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665463424:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1787739048.824854:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739048.824901:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07406576a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739048.829782:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665463936:0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739048.829796:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1aa9b6 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x14f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aa9bd expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.829804:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1aa9b6 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x14f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aa9bd expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.829825:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aa9c4 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x14f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.829834:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aa9c4 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x14f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.829842:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aa9c4 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x14f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.829857:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1aa9cb lrc: 3/0,1 mode: --/EX res: [0x240000400:0xac:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.829868:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1aa985 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xac:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09482a expref: 10 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.829876:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1aa9cb lrc: 4/0,1 mode: --/EX res: [0x240000400:0xac:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.829911:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1aa985 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xac:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09482a expref: 10 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.829924:0:6523:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1aa985 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xac:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09482a expref: 10 pid: 6525 timeout: 735 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739048.829941:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665464064:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739048.830003:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07406576480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739048.836230:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665464064:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739048.836259:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1aa9cb lrc: 3/0,1 mode: --/EX res: [0x240000400:0xac:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739048.869310:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f7112c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665465088:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739048.869382:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739048.909665:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652384640 00000100:00100000:2.0:1787739048.909675:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652384640 00000100:00100000:2.0:1787739048.909734:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23031 00000100:00100000:2.0:1787739048.909743:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073056183c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17344:x1874579652384640:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739048.909776:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739048.909796:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1aa9e0 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094838 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.909856:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1aa9e7 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.909869:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1aa9e7 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.909881:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1aa9e7 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.909908:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aa9ee lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.909916:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aa9ee lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.909923:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aa9ee lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.909964:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1aa9e7 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.909971:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1aa9e7 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.909977:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1aa9e7 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.909982:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1aa9e7 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.909988:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739048.909992:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1aa9e7 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.909999:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1aa9e7 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.910011:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b800) 00010000:00010000:2.0:1787739048.910014:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739048.910019:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1aa9e7 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.910025:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1aa9e7 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.910039:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aa9ee lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.910045:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aa9ee lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.910051:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aa9ee lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.910056:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739048.910059:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aa9ee lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.910065:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aa9ee lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.910072:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x17cadbd8 (ffffa074419d5300) 00010000:00010000:2.0:1787739048.910074:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739048.910077:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aa9ee lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.910082:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aa9ee lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.910113:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1aa9e0 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094838 expref: 11 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.910133:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b800) 00010000:00010000:2.0:1787739048.910136:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739048.910140:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1aa9e0 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094838 expref: 11 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.910148:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d200, rc 0) 00000100:00100000:2.0:1787739048.910211:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073056183c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17344:x1874579652384640:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 468us (627us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739048.910218:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23031 00000100:00100000:2.0:1787739048.920338:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739048.934747:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665465984 00000100:00100000:2.0:1787739048.934752:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665465984 00000100:00100000:2.0:1787739048.934796:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 998 00000100:00100000:2.0:1787739048.934804:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6509:x1874579665465984:12345-0@lo:105: 00010000:00010000:2.0:1787739048.934814:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1aaa03 lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aaa0a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.934823:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1aaa03 lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aaa0a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.934832:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1aaa03 lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aaa0a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.934839:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1aaa03 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1aaa0a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.934846:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1aaa03 lrc: 5/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1aaa0a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.934854:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1aaa03 lrc: 5/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1aaa0a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.934867:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069abc00) 00000100:00100000:2.0:1787739048.934920:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6509:x1874579665465984:12345-0@lo:105: Request processed in 116us (265us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739048.934926:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 998 00000100:00100000:2.0:1787739048.934961:0:6509:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f7112c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001_UUID:6509:1874579665465984:0@lo:105: 00010000:00010000:2.0:1787739048.934975:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739048.935010:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2409:x1874579652385152:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 535us (738us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739048.935017:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1310 00000100:00100000:2.0:1787739048.938860:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665466240 00000100:00100000:2.0:1787739048.938865:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665466240 00000100:00100000:2.0:1787739048.938905:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2244 00000100:00100000:2.0:1787739048.938912:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7112c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6523:x1874579665466240:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739048.938984:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7112c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6523:x1874579665466240:12345-0@lo:1000:mdt00_000.0 Request processed in 73us (238us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739048.938990:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2244 00000100:00100000:2.0:1787739048.939394:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665466368 00000100:00100000:2.0:1787739048.939399:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665466368 00000100:00100000:2.0:1787739048.939453:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2245 00000100:00100000:2.0:1787739048.939460:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6523:x1874579665466368:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739048.939527:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6523:x1874579665466368:12345-0@lo:1000:mdt00_000.0 Request processed in 68us (349us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739048.939534:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2245 00000100:00100000:2.0:1787739048.939559:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665466368:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739048.939629:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d0f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665466496:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739048.939658:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739048.939679:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665466496 00000100:00100000:2.0:1787739048.939683:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665466496 00000100:00100000:2.0:1787739048.939705:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2246 00000100:00100000:2.0:1787739048.939711:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d1680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6523:x1874579665466496:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739048.939758:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d1680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6523:x1874579665466496:12345-0@lo:1000:mdt00_000.0 Request processed in 47us (99us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739048.939880:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2246 00000100:00100000:2.0:1787739048.940009:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07406576a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739048.940025:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d0f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665466496:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739048.940070:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d0f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665466624:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739048.940094:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739048.940156:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b95840 going to sleep for 11 seconds 00000100:00100000:2.0:1787739048.940481:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d0f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665466624:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739048.940520:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d0f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665466752:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739048.940543:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739048.940896:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b95840 going to sleep for 11 seconds 00000100:00100000:2.0:1787739048.941451:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d0f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665466752:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739048.941496:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d0f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665466880:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739048.941521:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739048.941533:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b95840 going to sleep for 11 seconds 00000100:00100000:2.0:1787739048.941783:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d0f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665466880:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739048.941814:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d0f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665467008:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739048.941839:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739048.941859:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b95840 going to sleep for 11 seconds 00000100:00100000:2.0:1787739048.942182:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d0f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665467008:0@lo:1000:mdt00_000.0 00000040:00080000:2.0:1787739048.943072:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].328, 2168 off505304 00000040:00080000:2.0:1787739048.943163:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].327, 2368 off545904 00000004:00080000:2.0:1787739048.943169:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07410b95600 with version 625 00010000:00010000:2.0:1787739048.945019:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1aaa1f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x14e5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.945035:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1aaa1f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14e5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.945046:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1aaa1f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14e5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.945064:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1aaa26 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.945071:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1aaa26 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.945077:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1aaa26 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.945105:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1aaa2d lrc: 3/0,1 mode: --/PW res: [0x240000401:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.945113:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1aaa2d lrc: 3/0,1 mode: --/PW res: [0x240000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.945133:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1aaa2d lrc: 3/0,1 mode: --/PW res: [0x240000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739048.945153:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d0f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665467392:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1787739048.945190:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739048.945249:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07406576c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739048.945611:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d0f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665467392:0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739048.945624:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1aaa2d lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aaa34 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.945633:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1aaa2d lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aaa34 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739048.945652:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000073c04204 reply state 00000000bf154936 transno 4294983260 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1aaa26 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.945664:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1aaa26 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.945671:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xaf:0x0].0x0 (ffffa074419d5900) 00010000:00010000:2.0:1787739048.945673:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739048.945676:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787739048.945682:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1aaa2d lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aaa34 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.945690:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1aaa2d lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aaa34 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739048.945700:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xaf:0x0] 00000004:00010000:2.0:1787739048.945706:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000073c04204 reply state 00000000bf154936 transno 4294983260 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1aaa1f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14e5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.945719:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1aaa1f lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x14e5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.945725:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14e5:0x0].0x0 (ffffa074419d5300) 00010000:00010000:2.0:1787739048.945727:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739048.945729:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:2.0:1787739048.945734:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14e5:0x0]: returning size 4096 00000004:00080000:2.0:1787739048.945740:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:2.0:1787739048.945748:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1aaa03 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aaa0a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.945755:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1aaa03 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aaa0a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.945771:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1aaa03 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aaa0a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.945788:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739048.945793:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:2.0:1787739048.945798:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1aaa11 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aaa18 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739048.945805:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1aaa11 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aaa18 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739048.945874:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305618f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17344:x1874579652385024:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 17523us (17724us total) trans 4294983260 rc 0/0 00000100:00100000:2.0:1787739048.945881:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23033 00000100:00100000:2.0:1787739049.010544:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739049.011482:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665467648 00000100:00100000:2.0:1787739049.011488:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665467648 00000100:00100000:2.0:1787739049.011522:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 999 00000100:00100000:2.0:1787739049.011529:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073040003c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6525:x1874579665467648:12345-0@lo:104: 00010000:00010000:2.0:1787739049.011540:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1aaa11 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aaa18 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.011607:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739049.011613:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073040003c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6525:x1874579665467648:12345-0@lo:104: Request processed in 86us (632us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739049.011620:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 999 00010000:00010000:2.0:1787739049.011642:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739049.011647:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1aaa11 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aaa18 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.011655:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ac00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739049.011660:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1aaa11 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1aaa18 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739049.011669:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1aaa11 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1aaa18 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.011694:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1aaa11 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aaa18 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.011701:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739049.011721:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1aaa11 lrc: 1/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aaa18 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.011733:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1aaa11 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aaa18 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.011750:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739049.011751:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739049.011753:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739049.011776:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157ebc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665467776:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739049.011806:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.011841:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665467776 00000100:00100000:2.0:1787739049.011845:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665467776 00010000:00010000:2.0:1787739049.011862:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aaa18 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1aaa11 expref: 11 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739049.011883:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1311 00000100:00100000:2.0:1787739049.011889:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157e92c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+11:3653:x1874579665467776:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739049.011896:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1787739049.022612:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415701e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665467904:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:2.0:1787739049.022682:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.041576:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415701e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665467904:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:2.0:1787739049.043306:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652385664 00000100:00100000:2.0:1787739049.043318:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652385664 00000100:00100000:2.0:1787739049.043407:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23036 00000100:00100000:2.0:1787739049.043446:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17346:x1874579652385664:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739049.043468:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739049.043490:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1aaa57 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x14e5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09484d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.043548:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aaa5e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14e5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.043561:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aaa5e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14e5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.043573:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aaa5e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14e5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739049.043657:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14e5:0x0]: returning size 4096 00000004:00200000:2.0:1787739049.043662:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739049.043699:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1aaa57 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x14e5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09484d expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.043717:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aaa5e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14e5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09484d expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.043726:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14e5:0x0].0x0 (ffffa074419d5300) 00010000:00010000:2.0:1787739049.043729:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739049.043732:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633dc00, rc 0) 00000100:00100000:2.0:1787739049.044970:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17346:x1874579652385664:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 1523us (3149us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739049.044980:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23036 00000100:00100000:2.0:1787739049.064587:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000020:00080000:2.0:1787739049.082739:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 320 00000004:00080000:2.0:1787739049.082792:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07406576c00 with version 599 00000020:00080000:2.0:1787739049.082808:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 320 cancel update log [0x200000401:0x3:0x0].312: rc = 0 00000020:00080000:2.0:1787739049.082888:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 320 cancel update log [0x240000400:0x3:0x0].312: rc = 0 00000020:00080000:2.0:1787739049.082896:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 320 00000100:00100000:2.0:1787739049.093194:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652386048 00000100:00100000:2.0:1787739049.093207:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652386048 00000100:00100000:2.0:1787739049.093275:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1107 00000100:00100000:2.0:1787739049.093285:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c2580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2408:x1874579652386048:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:2.0:1787739049.093301:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739049.093325:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1aaa50 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14e5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094846 expref: 8 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.093342:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14e5:0x0].0x0 (ffffa07433235400) 00010000:00010000:2.0:1787739049.093352:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739049.093427:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c2580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652386048:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 128us (3591us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739049.093435:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1107 00000100:00100000:2.0:1787739049.094487:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665469184 00000100:00100000:2.0:1787739049.094492:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665469184 00000100:00100000:2.0:1787739049.094521:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2257 00000100:00100000:2.0:1787739049.094528:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562cb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+5:6523:x1874579665469184:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739049.094538:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739049.094552:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1aaa88 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aaa81 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.094572:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1aaa88 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaa81 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.094584:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1aaa88 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaa81 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.094591:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074419d5a00) 00010000:00010000:2.0:1787739049.094594:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739049.094597:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c200, rc 0) 00000100:00100000:2.0:1787739049.094638:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562cb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6523:x1874579665469184:12345-0@lo:101:mdt00_000.0 Request processed in 110us (462us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739049.094644:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2257 00000100:00100000:2.0:1787739049.094678:0:6523:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968252 gen 2 00000004:00080000:2.0:1787739049.094682:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968247 committed transno 4294968252 00000020:00080000:2.0:1787739049.094687:0:6523:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa074355ed600 refcount 2 committed 0 result 0 batchid 329 00000020:00080000:2.0:1787739049.094690:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa074356c7100 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa074355edf00 00000020:00080000:2.0:1787739049.094693:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].326 00000020:00080000:2.0:1787739049.094695:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743563f300 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa074357c0000 00000020:00080000:2.0:1787739049.094697:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].327 00000004:00080000:2.0:1787739049.094720:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968249 committed transno 4294968252 00000004:00080000:2.0:1787739049.094729:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968250 committed transno 4294968252 00000004:00080000:2.0:1787739049.094737:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968252 committed transno 4294968252 00000100:00100000:2.0:1787739049.094754:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562de00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665469184:0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739049.094769:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1aaa81 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aaa88 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.094780:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1aaa81 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aaa88 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739049.094795:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:2.0:1787739049.094816:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d600/0x7ee4453e4a1aaa8f lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.094833:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d600/0x7ee4453e4a1aaa8f lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.094842:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d600/0x7ee4453e4a1aaa8f lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739049.094854:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f73840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665469312:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1787739049.094880:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.094898:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4dcc0 going to sleep for 11 seconds 00000020:00080000:2.0:1787739049.094909:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 329 committed batchid 328 00000020:00080000:2.0:1787739049.094966:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 328 00000100:00100000:2.0:1787739049.094977:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665469312 00000100:00100000:2.0:1787739049.094980:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665469312 00000100:00100000:2.0:1787739049.095005:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2258 00000100:00100000:2.0:1787739049.095011:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f721c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6523:x1874579665469312:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739049.095020:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739049.095031:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1aaa96 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aaa8f expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.095046:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1aaa96 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaa8f expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.095056:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1aaa96 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaa8f expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.095061:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x17cadbd8 (ffffa074419d5000) 00010000:00010000:2.0:1787739049.095064:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739049.095066:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d800, rc 0) 00000100:00100000:2.0:1787739049.095095:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f721c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665469312:12345-0@lo:101:mdt00_000.0 Request processed in 84us (214us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739049.095100:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2258 00000100:00100000:2.0:1787739049.095141:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f73840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665469312:0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739049.095151:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d600/0x7ee4453e4a1aaa8f lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aaa96 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.095158:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d600/0x7ee4453e4a1aaa8f lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aaa96 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739049.095168:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:2.0:1787739049.095197:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f73840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665469440:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739049.095237:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.095253:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074157c76c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739049.095265:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665469440 00000100:00100000:2.0:1787739049.095268:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665469440 00000100:00100000:2.0:1787739049.095290:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2259 00000100:00100000:2.0:1787739049.095295:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f72d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665469440:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739049.095352:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f72d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665469440:12345-0@lo:1000:mdt00_000.0 Request processed in 57us (116us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739049.095369:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2259 00000100:00100000:2.0:1787739049.095396:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f73840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665469440:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739049.095454:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f73840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665469568:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739049.095478:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.095493:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739049.095504:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665469568 00000100:00100000:2.0:1787739049.095507:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665469568 00000100:00100000:2.0:1787739049.095529:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2260 00000100:00100000:2.0:1787739049.095536:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f703c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665469568:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739049.095607:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f703c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665469568:12345-0@lo:1000:mdt00_000.0 Request processed in 72us (128us total) trans 0 rc 1/1 00000100:00100000:2.0:1787739049.095613:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2260 00000100:00100000:2.0:1787739049.095638:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f73840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665469568:0@lo:1000:mdt00_000.0 00010000:00010000:2.0:1787739049.095673:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1aaa9d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14e5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.095681:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1aaa9d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14e5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.095688:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1aaa9d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14e5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739049.095699:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f73840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665469696:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1787739049.095722:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.095736:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739049.095746:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665469696 00000100:00100000:2.0:1787739049.095749:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665469696 00000100:00100000:2.0:1787739049.095768:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2261 00000100:00100000:2.0:1787739049.095773:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f72940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665469696:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739049.095781:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739049.095791:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1aaaa4 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14e5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aaa9d expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.095806:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1aaaa4 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14e5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaa9d expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.095821:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1aaaa4 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14e5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaa9d expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.095828:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14e5:0x0].0x0 (ffffa074419d5f00) 00010000:00010000:2.0:1787739049.095830:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739049.095833:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c000, rc 0) 00000100:00100000:2.0:1787739049.095860:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f72940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6523:x1874579665469696:12345-0@lo:101:mdt00_000.0 Request processed in 86us (137us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739049.095865:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2261 00000100:00100000:2.0:1787739049.095890:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f73840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665469696:0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739049.095899:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1aaa9d lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14e5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aaaa4 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.095907:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1aaa9d lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14e5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aaaa4 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.095926:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaaab lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14e5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.095935:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaaab lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14e5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.095942:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaaab lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14e5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.095957:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1aaab2 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.095971:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aaa6c lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094854 expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.095979:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1aaab2 lrc: 4/0,1 mode: --/EX res: [0x200000400:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.096000:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aaa6c lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094854 expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.096013:0:6523:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aaa6c lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094854 expref: 17 pid: 13793 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739049.096024:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f73840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665469824:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739049.096075:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739049.125618:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665470336 00000100:00100000:2.0:1787739049.125633:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665470336 00000100:00100000:2.0:1787739049.125690:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2263 00000100:00100000:2.0:1787739049.125700:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6523:x1874579665470336:12345-0@lo:403:mdt00_000.0 00000100:00100000:2.0:1787739049.130990:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6523:x1874579665470336:12345-0@lo:403:mdt00_000.0 Request processed in 5282us (13328us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739049.131005:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2263 00000004:00080000:2.0:1787739049.135982:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 626 rpc_version 626 00000020:00100000:2.0:1787739049.136020:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739049.136030:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303f70000 x1874579665470592/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2064/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739049.136063:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f70000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665470592:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739049.136139:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.136176:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739049.138102:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f70000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665470592:0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1787739049.147634:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739049.180185:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739049.186130:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23041 00000100:00100000:2.0:1787739049.186143:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb7c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17347:x1874579652386816:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739049.186164:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739049.186186:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1aaadc lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094869 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.186259:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaae3 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.186273:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaae3 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.186284:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaae3 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.186295:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aaaea lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x490825be bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.186302:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aaaea lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.186307:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aaaea lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.186344:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaae3 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.186352:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaae3 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.186357:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaae3 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.186364:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaae3 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.186370:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739049.186375:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaae3 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.186383:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaae3 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.186395:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074419d5a00) 00010000:00010000:2.0:1787739049.186397:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739049.186401:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaae3 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.186407:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaae3 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.186569:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aaaea lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.186575:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aaaea lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.186581:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aaaea lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.186587:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739049.186590:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aaaea lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.186596:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aaaea lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.186604:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x490825be (ffffa074419d5f00) 00010000:00010000:2.0:1787739049.186606:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739049.186609:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aaaea lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.186613:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aaaea lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.186646:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1aaadc lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094869 expref: 7 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.186656:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074419d5a00) 00010000:00010000:2.0:1787739049.186659:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739049.186662:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1aaadc lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094869 expref: 7 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.186673:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c200, rc 0) 00000100:00100000:2.0:1787739049.186739:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb7c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17347:x1874579652386816:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 598us (6563us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739049.186745:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23041 00000004:00080000:2.0:1787739049.223407:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 600 rpc_version 600 00000020:00100000:2.0:1787739049.223564:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739049.223576:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304260f00 x1874579665471232/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739049.223600:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304260f00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665471232:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739049.223673:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.224817:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739049.228362:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304260f00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665471232:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739049.245521:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652387456 00000100:00100000:2.0:1787739049.245532:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652387456 00000100:00100000:2.0:1787739049.245596:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23044 00000100:00100000:2.0:1787739049.245606:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17348:x1874579652387456:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:2.0:1787739049.246049:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17348:x1874579652387456:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 443us (658us total) trans 0 rc -17/-17 00000100:00100000:2.0:1787739049.246057:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23044 00010000:00010000:2.0:1787739049.288935:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739049.296907:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739049.297749:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665471488 00000100:00100000:2.0:1787739049.297755:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665471488 00000100:00100000:2.0:1787739049.297798:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1002 00000100:00100000:2.0:1787739049.297806:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6523:x1874579665471488:12345-0@lo:104: 00010000:00010000:2.0:1787739049.297819:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1aab1b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aab22 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.297882:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739049.297889:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6523:x1874579665471488:12345-0@lo:104: Request processed in 85us (393us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739049.297897:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1002 00010000:00010000:2.0:1787739049.297919:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739049.297925:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1aab1b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aab22 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.297934:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739049.297940:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1aab1b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1aab22 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739049.297948:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1aab1b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1aab22 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.297994:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1aab1b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aab22 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.298002:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739049.298021:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1aab1b lrc: 1/0,0 mode: --/PW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aab22 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.298029:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1aab1b lrc: 0/0,0 mode: --/PW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aab22 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.298046:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739049.298048:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739049.298049:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739049.298061:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665471616:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739049.298089:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.298133:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665471616 00000100:00100000:2.0:1787739049.298137:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665471616 00010000:00010000:2.0:1787739049.298153:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aab22 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1aab1b expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739049.298181:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1315 00000100:00100000:2.0:1787739049.298188:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f76d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3651:x1874579665471616:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739049.298694:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1787739049.310176:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302528780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665471744:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787739049.310264:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000020:00080000:2.0:1787739049.310311:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 330 committed batchid 329 00000004:00080000:2.0:1787739049.310447:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07435539a80 with version 627 00000020:00080000:2.0:1787739049.310458:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 329 cancel update log [0x240000401:0x3:0x0].326: rc = 0 00000020:00080000:2.0:1787739049.310497:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 329 cancel update log [0x200000400:0x3:0x0].327: rc = 0 00000020:00080000:2.0:1787739049.310507:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 329 00000004:00080000:2.0:1787739049.310515:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 627 rpc_version 627 00000020:00100000:2.0:1787739049.310534:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739049.310542:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303f74780 x1874579665472000/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739049.310558:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665472000:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739049.310589:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.310606:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739049.310622:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665472000 00000100:00100000:2.0:1787739049.310628:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665472000 00000100:00100000:2.0:1787739049.310669:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2267 00000100:00100000:2.0:1787739049.310678:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f752c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665472000:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739049.310805:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f752c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665472000:12345-0@lo:1000:osp_up1-0.0 Request processed in 127us (214us total) trans 4294968255 rc 0/0 00000100:00100000:2.0:1787739049.310812:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2267 00000100:00100000:2.0:1787739049.310857:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665472000:0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1787739049.310910:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aab22 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1aab1b expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.310922:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffffa0730608b700) 00010000:00010000:2.0:1787739049.310927:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1aab5a lrc: 3/1,0 mode: --/PR res: [0x200000401:0xad:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.310936:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1aab5a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.310943:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1aab5a lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.310963:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739049.310996:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f76d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665471616:12345-0@lo:103:ldlm_bl.0 Request processed in 12809us (12906us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739049.311003:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1315 00000100:00100000:2.0:1787739049.311018:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665471744 00000100:00100000:2.0:1787739049.311021:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665471744 00000100:00100000:2.0:1787739049.311048:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1111 00000100:00100000:2.0:1787739049.311054:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f77c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+8:3652:x1874579665471744:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:2.0:1787739049.311063:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739049.311076:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1aaa96 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x17cadbd8 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aaa8f expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.311086:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x17cadbd8 (ffffa074419d5000) 00010000:00010000:2.0:1787739049.311093:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739049.311135:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f77c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+7:3652:x1874579665471744:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 81us (871us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739049.311141:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1111 00000100:00100000:2.0:1787739049.311176:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665471616:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739049.311216:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302528780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665471744:0@lo:103:jbd2/dm-0-8.0 00010000:00010000:2.0:1787739049.311234:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1aab5a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.311243:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1aab5a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739049.311293:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xad:0x0]: returning size 4096 00000004:00200000:2.0:1787739049.311298:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739049.311337:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aab53 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xad:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094885 expref: 16 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.311352:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1aab5a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094885 expref: 15 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.311359:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffffa0730608b700) 00010000:00010000:2.0:1787739049.311362:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739049.311364:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aae00, rc 0) 00000100:00100000:2.0:1787739049.311404:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304261a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17349:x1874579652387840:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 14175us (14507us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739049.311431:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23047 00000100:00100000:2.0:1787739049.321973:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652388096 00000100:00100000:2.0:1787739049.321986:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652388096 00000100:00100000:2.0:1787739049.322055:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23048 00000100:00100000:2.0:1787739049.322065:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17349:x1874579652388096:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:2.0:1787739049.322128:0:6525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739049.322155:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aab3e lrc: 0/0,0 mode: --/PR res: [0x240000402:0x151:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094877 expref: 10 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.322175:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x151:0x0].0x0 (ffffa07433235100) 00010000:00010000:2.0:1787739049.322180:0:6525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739049.322222:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aab61 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.322234:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aab61 lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.322244:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aab61 lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.322255:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aab68 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x490825be bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.322263:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aab68 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.322270:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aab68 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.322345:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1aab6f lrc: 3/0,1 mode: --/EX res: [0x240000402:0x151:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.322354:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1aab6f lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x151:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.322361:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1aab6f lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x151:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.322378:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1aab76 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xad:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.322388:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aab4c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xad:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09487e expref: 9 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.322396:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1aab76 lrc: 4/0,1 mode: --/EX res: [0x240000400:0xad:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.322458:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aab4c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xad:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09487e expref: 9 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.322472:0:6525:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aab4c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xad:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09487e expref: 9 pid: 13793 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739049.322485:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74f00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665472128:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739049.322572:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739049.331974:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652388224 00000100:00100000:2.0:1787739049.331984:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652388224 00010000:00010000:2.0:1787739049.332024:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aab4c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xad:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09487e expref: 10 pid: 13793 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739049.332069:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1316 00000100:00100000:2.0:1787739049.332080:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba7850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652388224:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739049.332095:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739049.332102:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aab4c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xad:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09487e expref: 10 pid: 13793 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.332132:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aab4c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xad:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09487e expref: 10 pid: 13793 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.332154:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aab4c lrc: 0/0,0 mode: --/PR res: [0x240000400:0xad:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09487e expref: 10 pid: 13793 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.332172:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xad:0x0].0x0 (ffffa07433235c00) 00010000:00010000:2.0:1787739049.332177:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1aab76 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xad:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.332186:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1aab76 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xad:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.332211:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1aab76 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xad:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.332240:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739049.332293:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba7850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:2408:x1874579652388224:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 216us (462us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739049.332301:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1316 00000100:00100000:2.0:1787739049.332784:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665472256 00000100:00100000:2.0:1787739049.332789:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665472256 00000100:00100000:2.0:1787739049.332824:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2268 00000100:00100000:2.0:1787739049.332830:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c03c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+5:6525:x1874579665472256:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787739049.332845:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739049.332858:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aab84 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xad:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aab7d expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.333649:0:8447:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1aab5a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094885 expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.333662:0:8447:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aab84 lrc: 4/0,0 mode: --/EX res: [0x200000401:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aab7d expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.333692:0:8447:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1aab5a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094885 expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.333707:0:8447:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1aab5a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xad:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094885 expref: 14 pid: 6523 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739049.333719:0:8447:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f76d00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8447:1874579665472384:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739049.333792:0:8447:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739049.342933:0:8447:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f76d00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8447:1874579665472384:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739049.342976:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aab84 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aab7d expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.342990:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xad:0x0].0x0 (ffffa0730608b700) 00010000:00010000:2.0:1787739049.342994:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739049.342998:0:8447:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aab84 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aab7d expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.343009:0:8447:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aab84 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xad:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aab7d expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.343023:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633dc00, rc 0) 00000100:00100000:2.0:1787739049.343090:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c03c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:6525:x1874579665472256:12345-0@lo:101:mdt00_002.0 Request processed in 10259us (10408us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739049.343099:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2268 00000020:00080000:2.0:1787739049.343548:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 321 committed batchid 320 00000020:00080000:2.0:1787739049.343616:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 320 00000100:00100000:2.0:1787739049.344741:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739049.347465:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1317 00000100:00100000:3.0:1787739049.350894:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665472640 00000100:00100000:3.0:1787739049.350907:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665472640 00000100:00100000:3.0:1787739049.350974:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2269 00000100:00100000:3.0:1787739049.350984:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c30c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:6525:x1874579665472640:12345-0@lo:403:mdt00_002.0 00000100:00100000:3.0:1787739049.352722:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c30c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:6525:x1874579665472640:12345-0@lo:403:mdt00_002.0 Request processed in 1733us (4470us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739049.352734:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2269 00000004:00080000:3.0:1787739049.357593:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 601 rpc_version 601 00000020:00100000:3.0:1787739049.357637:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739049.357648:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743562d2c0 x1874579665472896/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1448/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739049.357672:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562d2c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665472896:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739049.357737:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739049.357792:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355edcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739049.425176:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652388480 00000100:00100000:3.0:1787739049.425186:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652388480 00000100:00100000:3.0:1787739049.425247:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23049 00000100:00100000:3.0:1787739049.425257:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17350:x1874579652388480:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739049.425277:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739049.425299:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aab8b lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09488c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.425361:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aab92 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.425374:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aab92 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.425388:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aab92 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.425454:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aab99 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.425462:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aab99 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.425469:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aab99 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.425512:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aab92 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.425518:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aab92 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.425524:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aab92 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.425530:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aab92 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.425536:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aba00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739049.425540:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aab92 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.425548:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aab92 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.425559:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2b00) 00010000:00010000:3.0:1787739049.425562:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739049.425566:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aab92 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.425572:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aab92 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.425583:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aab99 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.425588:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aab99 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.425593:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aab99 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.425597:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739049.425600:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aab99 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.425604:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aab99 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.425610:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x3fb0b00e (ffffa0730608bf00) 00010000:00010000:3.0:1787739049.425613:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739049.425615:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aab99 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.425620:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aab99 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.425646:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aab8b lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09488c expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.425660:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2b00) 00010000:00010000:3.0:1787739049.425663:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739049.425666:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aab8b lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09488c expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.425674:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab800, rc 0) 00000100:00100000:3.0:1787739049.425728:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17350:x1874579652388480:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 472us (648us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739049.425735:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23049 00000100:00100000:3.0:1787739049.430384:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739049.430436:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652388608 00000100:00100000:3.0:1787739049.430447:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652388608 00000100:00100000:3.0:1787739049.430497:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23050 00000100:00100000:3.0:1787739049.430506:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17350:x1874579652388608:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739049.430600:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739049.430626:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aaba0 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094893 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.430687:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1aaba7 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.430700:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1aaba7 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.430710:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1aaba7 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739049.430769:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x5:0x0]: returning size 4096 00000004:00200000:3.0:1787739049.430774:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739049.430811:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aaba0 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094893 expref: 12 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.430829:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1aaba7 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094893 expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.430836:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b200) 00010000:00010000:3.0:1787739049.430839:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739049.430841:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aae00, rc 0) 00000100:00100000:3.0:1787739049.431015:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17350:x1874579652388608:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 508us (639us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739049.431022:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23050 00000100:00100000:2.0:1787739049.444290:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652388864 00000100:00100000:2.0:1787739049.444300:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652388864 00000100:00100000:2.0:1787739049.444371:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23051 00000100:00100000:2.0:1787739049.444383:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17350:x1874579652388864:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:2.0:1787739049.444474:0:6525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739049.444501:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aa9fc lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09483f expref: 14 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.444520:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2b00) 00010000:00010000:2.0:1787739049.444530:0:6525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739049.444599:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aabae lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.444612:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aabae lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.444621:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aabae lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.444632:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aabb5 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.444640:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aabb5 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.444647:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aabb5 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787739049.448665:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].330, 2168 off508136 00000040:00080000:3.0:1787739049.448746:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].329, 2368 off549016 00000004:00080000:3.0:1787739049.448751:0:6525:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07435539240 with version 628 00000004:00080000:2.0:1787739049.449004:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 628 rpc_version 628 00000020:00100000:2.0:1787739049.449042:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739049.449052:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303f74000 x1874579665473408/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 3976/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739049.449078:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665473408:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739049.449156:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.449236:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739049.454731:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665473408:0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1787739049.454963:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aabbc lrc: 3/0,1 mode: --/PW res: [0x200000402:0x14e6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.454979:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aabbc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14e6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.454990:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aabbc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14e6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.455008:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1aabc3 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.455015:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1aabc3 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.455021:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1aabc3 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.455057:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1aabca lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.455063:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1aabca lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.455071:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1aabca lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739049.455089:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561e940 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665473536:0@lo:101:mdt00_002.0 00000100:00100000:2.0:1787739049.455148:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.455195:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739049.455356:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665473536 00000100:00100000:3.0:1787739049.455363:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665473536 00000100:00100000:3.0:1787739049.455437:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2273 00000100:00100000:3.0:1787739049.455447:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6525:x1874579665473536:12345-0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739049.455465:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739049.455483:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aabd1 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aabca expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.455507:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aabd1 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aabca expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.455518:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aabd1 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aabca expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.455525:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb0:0x0].0x0 (ffffa0730608bf00) 00010000:00010000:3.0:1787739049.455528:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739049.455531:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abe00, rc 0) 00000100:00100000:3.0:1787739049.455581:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6525:x1874579665473536:12345-0@lo:101:mdt00_002.0 Request processed in 136us (433us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739049.455587:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2273 00000100:00100000:3.0:1787739049.494576:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739049.494686:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652389120 00000100:00100000:2.0:1787739049.494696:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652389120 00000100:00100000:2.0:1787739049.494753:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23052 00000100:00100000:2.0:1787739049.494773:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17351:x1874579652389120:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:2.0:1787739049.494954:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17351:x1874579652389120:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 192us (386us total) trans 0 rc -17/-17 00000100:00100000:2.0:1787739049.494972:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23052 00000100:00100000:3.0:1787739049.525379:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739049.530271:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739049.530899:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665473664 00000100:00100000:3.0:1787739049.530904:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665473664 00000100:00100000:3.0:1787739049.530942:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1004 00000100:00100000:3.0:1787739049.530949:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:13793:x1874579665473664:12345-0@lo:104: 00010000:00010000:3.0:1787739049.530961:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1aabca lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aabd1 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.531026:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739049.531032:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:13793:x1874579665473664:12345-0@lo:104: Request processed in 84us (343us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739049.531039:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1004 00010000:00010000:3.0:1787739049.531060:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739049.531067:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1aabca lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aabd1 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.531075:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633de00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739049.531081:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1aabca lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xb0:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1aabd1 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739049.531089:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1aabca lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xb0:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1aabd1 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.531124:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1aabca lrc: 2/0,0 mode: --/PW res: [0x240000401:0xb0:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aabd1 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.531130:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739049.531148:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1aabca lrc: 1/0,0 mode: --/PW res: [0x240000401:0xb0:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aabd1 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.531155:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1aabca lrc: 0/0,0 mode: --/PW res: [0x240000401:0xb0:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aabd1 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.531173:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739049.531175:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739049.531176:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739049.531187:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665473792:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739049.531215:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739049.531250:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665473792 00000100:00100000:3.0:1787739049.531253:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665473792 00010000:00010000:3.0:1787739049.531269:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aabd1 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1aabca expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739049.531295:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1318 00000100:00100000:3.0:1787739049.531301:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562f840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+9:3651:x1874579665473792:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739049.531310:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787739049.532068:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665473664:0@lo:104: 00010000:00010000:3.0:1787739049.532086:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aac09 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.535270:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aab06 lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.535281:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aab06 lrc: 2/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.535288:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aab06 lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.535294:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739049.535299:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aab06 lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.535306:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aab06 lrc: 4/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.535322:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x490825be (ffffa07433235800) 00010000:00010000:2.0:1787739049.535385:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739049.535391:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aab06 lrc: 2/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.535398:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aab06 lrc: 0/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.535438:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aab0d lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x151:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.535444:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aab0d lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x151:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.535449:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aab0d lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x151:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.535454:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739049.535456:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aab0d lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x151:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.535461:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aab0d lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x151:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.535469:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x151:0x0].0x0 (ffffa07433235100) 00010000:00010000:2.0:1787739049.535472:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739049.535474:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aab0d lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x151:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.535480:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aab0d lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x151:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1787739049.535717:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 322 committed batchid 321 00010000:00010000:2.0:1787739049.535829:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1aab14 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xad:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.535837:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1aab14 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xad:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.535844:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1aab14 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xad:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.535850:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739049.535853:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1aab14 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xad:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.535858:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1aab14 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xad:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.535868:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xad:0x0].0x0 (ffffa07433235c00) 00010000:00010000:2.0:1787739049.535870:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739049.535873:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1aab14 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xad:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.535878:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1aab14 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xad:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739049.536029:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa074355ed540 with version 602 00000020:00080000:3.0:1787739049.536042:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 321 cancel update log [0x200000401:0x3:0x0].313: rc = 0 00000020:00080000:3.0:1787739049.536083:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 321 cancel update log [0x240000400:0x3:0x0].313: rc = 0 00000020:00080000:3.0:1787739049.536092:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 321 00010000:00010000:3.0:1787739049.536160:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aabd1 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1aabca expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.536174:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb0:0x0].0x0 (ffffa0730608bf00) 00010000:00010000:3.0:1787739049.536178:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aac09 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.536187:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aac09 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.536194:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aac09 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.536220:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739049.536305:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562f840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665473792:12345-0@lo:103:ldlm_bl.0 Request processed in 5003us (5087us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739049.536315:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1318 00010000:00010000:2.0:1787739049.536627:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aab68 lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.536637:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aab68 lrc: 2/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.536643:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aab68 lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.536648:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739049.536651:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aab68 lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.536657:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aab68 lrc: 4/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.536666:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x490825be (ffffa07433235800) 00010000:00010000:2.0:1787739049.536668:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739049.536671:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aab68 lrc: 2/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.536676:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aab68 lrc: 0/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0x490825be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739049.537069:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665474048 00000100:00100000:2.0:1787739049.537077:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665474048 00000100:00100000:2.0:1787739049.537134:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2274 00000100:00100000:2.0:1787739049.537143:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:7467:x1874579665474048:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739049.537272:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665474048:12345-0@lo:1000:osp_up0-1.0 Request processed in 129us (408us total) trans 4294983268 rc 0/0 00000100:00100000:2.0:1787739049.537279:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2274 00000100:00100000:3.0:1787739049.538089:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665473792:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739049.538126:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aac09 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.538136:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aac09 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739049.538187:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xb0:0x0]: returning size 4096 00000004:00200000:3.0:1787739049.538193:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739049.538237:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1aac02 lrc: 0/0,0 mode: --/CR res: [0x240000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948a8 expref: 9 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.538264:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aac09 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948a8 expref: 8 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.538273:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb0:0x0].0x0 (ffffa0730608bf00) 00010000:00010000:3.0:1787739049.538276:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739049.538279:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609aa00, rc 0) 00000100:00100000:3.0:1787739049.540021:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17352:x1874579652389504:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 9548us (9753us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739049.540033:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23055 00000100:00100000:3.0:1787739049.546584:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652389760 00000100:00100000:3.0:1787739049.546594:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652389760 00000100:00100000:3.0:1787739049.546650:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23056 00000100:00100000:3.0:1787739049.546660:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17352:x1874579652389760:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:3.0:1787739049.546712:0:6524:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739049.546739:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aabed lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14e6:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09489a expref: 17 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.546768:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14e6:0x0].0x0 (ffffa074419d5600) 00010000:00010000:3.0:1787739049.546773:0:6524:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739049.546796:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aac10 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.546808:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aac10 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.546818:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aac10 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.546829:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aac17 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.546835:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aac17 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.546842:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aac17 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.547073:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aac1e lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14e6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.547082:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aac1e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14e6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.547090:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aac1e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14e6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.547107:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aac25 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.547128:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1aabfb lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948a1 expref: 16 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.547136:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aac25 lrc: 4/0,1 mode: --/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.547171:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1aabfb lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948a1 expref: 16 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.547184:0:6524:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1aabfb lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948a1 expref: 16 pid: 6525 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739049.547197:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562f480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665474176:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739049.547279:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355ed780 going to sleep for 11 seconds 00000100:00100000:3.0:1787739049.572196:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00010000:00010000:2.0:1787739049.573312:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1aabfb lrc: 0/0,0 mode: --/PR res: [0x200000400:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948a1 expref: 16 pid: 6525 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739049.573331:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562f480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665474176:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739049.573350:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aac25 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.573359:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aac25 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.573364:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aac25 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.573403:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1aac2c lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.573435:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1aac2c lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.573446:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1aac2c lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739049.573466:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561e940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665474304:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1787739049.573509:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.573534:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0efd80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739049.574080:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665474304 00000100:00100000:3.0:1787739049.574086:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665474304 00000100:00100000:3.0:1787739049.574145:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2275 00000100:00100000:3.0:1787739049.574154:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c382580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+5:6524:x1874579665474304:12345-0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787739049.574169:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739049.574184:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aac33 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aac2c expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.574205:0:18535:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aac09 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948a8 expref: 7 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.574215:0:18535:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aac33 lrc: 4/0,0 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aac2c expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.574236:0:18535:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aac09 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948a8 expref: 7 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.574247:0:18535:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aac09 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948a8 expref: 7 pid: 13793 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739049.574256:0:18535:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562da40 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:18535:1874579665474432:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739049.574312:0:18535:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355ed600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739049.578670:0:6524:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968257 gen 2 00000004:00080000:2.0:1787739049.578677:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968253 committed transno 4294968257 00000020:00080000:2.0:1787739049.578683:0:6524:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07307054480 refcount 2 committed 0 result 0 batchid 331 00000020:00080000:2.0:1787739049.578687:0:6524:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c695700 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07307054780 00000020:00080000:2.0:1787739049.578691:0:6524:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].328 00000020:00080000:2.0:1787739049.578693:0:6524:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa074035e8100 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306962800 00000020:00080000:2.0:1787739049.578696:0:6524:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].329 00000004:00080000:2.0:1787739049.578717:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968255 committed transno 4294968257 00000004:00080000:2.0:1787739049.578726:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968257 committed transno 4294968257 00000100:00100000:2.0:1787739049.578737:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561e940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665474304:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739049.578754:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1aac2c lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aac33 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.578776:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1aac2c lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aac33 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739049.586305:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739049.586983:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652390016 00000100:00100000:3.0:1787739049.586992:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652390016 00010000:00010000:3.0:1787739049.587030:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aac09 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948a8 expref: 8 pid: 13793 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739049.587079:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1320 00000100:00100000:3.0:1787739049.587088:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2408:x1874579652390016:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739049.587103:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739049.587119:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aac09 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948a8 expref: 8 pid: 13793 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.587131:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aac09 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948a8 expref: 8 pid: 13793 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.587151:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aac09 lrc: 0/0,0 mode: --/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948a8 expref: 8 pid: 13793 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.587164:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb0:0x0].0x0 (ffffa0730608bf00) 00010000:00010000:3.0:1787739049.587171:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aac33 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aac2c expref: 5 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.587181:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aac33 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1aac2c expref: 5 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.587190:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aac33 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1aac2c expref: 5 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.587224:0:10340:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aac33 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aac2c expref: 5 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739049.587239:0:10340:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562cf00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665474560:0@lo:105: 00000100:00100000:3.0:1787739049.587287:0:10340:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739049.587390:0:10340:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355ed600 going to sleep for 11 seconds 00010000:00010000:3.0:1787739049.592222:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1aac2c lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aac33 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.592235:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1aac2c lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aac33 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739049.593169:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562f480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665474688:0@lo:403:mdt00_001.0 00000100:00100000:3.0:1787739049.593237:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739049.593285:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355ed780 going to sleep for 11 seconds 00000100:00100000:2.0:1787739049.597073:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665474688 00000100:00100000:2.0:1787739049.597086:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665474688 00000100:00100000:2.0:1787739049.597155:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2276 00000100:00100000:2.0:1787739049.597166:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6524:x1874579665474688:12345-0@lo:403:mdt00_001.0 00000100:00100000:2.0:1787739049.599699:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6524:x1874579665474688:12345-0@lo:403:mdt00_001.0 Request processed in 2531us (6457us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739049.599712:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2276 00000100:00100000:3.0:1787739049.603906:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562f480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665474688:0@lo:403:mdt00_001.0 00080000:02000000:3.0:1787739049.604701:0:6524:0:(osd_handler.c:2218:osd_trans_start()) *** cfs_fail_loc=19a, val=0*** 00000004:00080000:3.0:1787739049.604716:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa074355ed780 with version 0 00000020:00080000:3.0:1787739049.604721:0:6524:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa074355edf00 refcount 3 committed 0 result -5 batchid 333 00000020:00080000:3.0:1787739049.604725:0:6524:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c695900 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result -5 sub_th ffffa074355ed780 00000020:00080000:3.0:1787739049.604729:0:6524:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c695000 obd lustre-MDT0000-osd committed 1 started 0 stopped 1 result 0 sub_th ffffa07302fab600 00010000:00010000:3.0:1787739049.604753:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aac25 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604762:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aac25 lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604770:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aac25 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604776:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739049.604781:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aac25 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604788:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aac25 lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604804:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb0:0x0].0x0 (ffffa074419d5500) 00010000:00010000:3.0:1787739049.604807:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739049.604811:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aac25 lrc: 2/0,0 mode: --/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604817:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aac25 lrc: 0/0,0 mode: --/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604831:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1aac2c lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aac33 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604840:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1aac2c lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aac33 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604847:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1aac2c lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aac33 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604861:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739049.604867:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xb0:0x0] 00010000:00010000:3.0:1787739049.604873:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aac1e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14e6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604879:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aac1e lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x14e6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604884:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aac1e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14e6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604890:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aba00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739049.604892:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aac1e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14e6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604899:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aac1e lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x14e6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604905:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14e6:0x0].0x0 (ffffa074419d5600) 00010000:00010000:3.0:1787739049.604907:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739049.604910:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aac1e lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14e6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604916:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aac1e lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14e6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604928:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aac10 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604934:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aac10 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604940:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aac10 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604946:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aac10 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604950:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739049.604953:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aac10 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604958:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aac10 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604964:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2100) 00010000:00010000:3.0:1787739049.604966:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739049.604968:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aac10 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604974:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aac10 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604986:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aac17 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604991:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aac17 lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.604998:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aac17 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.605003:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739049.605006:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aac17 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.605011:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aac17 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.605017:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x3fb0b00e (ffffa074419d5f00) 00010000:00010000:3.0:1787739049.605019:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739049.605021:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aac17 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.605027:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aac17 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739049.605100:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17352:x1874579652389760:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 58439us (58607us total) trans 0 rc -5/-5 00000100:00100000:3.0:1787739049.605108:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23056 00010000:00010000:2.0:1787739049.605303:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739049.605310:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1aac2c lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aac33 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.605317:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633da00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739049.605322:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1aac2c lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aac33 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739049.605344:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1aac2c lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aac33 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.605373:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1aac2c lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aac33 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.605379:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739049.605398:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1aac2c lrc: 1/0,0 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aac33 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.605405:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1aac2c lrc: 0/0,0 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aac33 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.605435:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739049.605437:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739049.605439:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739049.605452:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665474816:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739049.605500:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.605546:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665474816 00000100:00100000:2.0:1787739049.605553:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665474816 00000100:00100000:2.0:1787739049.605591:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1113 00000100:00100000:2.0:1787739049.605600:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356acf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+6:3652:x1874579665474816:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739049.605612:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739049.605630:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aac33 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aac2c expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.605641:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb0:0x0].0x0 (ffffa0730608bf00) 00010000:00010000:2.0:1787739049.605647:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739049.605678:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356acf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+5:3652:x1874579665474816:12345-0@lo:103:ldlm_bl.0 Request processed in 80us (178us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739049.605688:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1113 00000100:00100000:2.0:1787739049.605719:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665474816:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739049.640075:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739049.656313:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652390272 00000100:00100000:3.0:1787739049.656323:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652390272 00000100:00100000:3.0:1787739049.656389:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23058 00000100:00100000:3.0:1787739049.656400:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17353:x1874579652390272:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739049.656434:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739049.656458:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aac4f lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0948b6 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.656528:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aac56 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.656540:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aac56 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.656551:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aac56 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.656564:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aac5d lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.656570:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aac5d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.656576:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aac5d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.656613:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aac56 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.656620:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aac56 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.656626:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aac56 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.656633:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aac56 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.656639:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739049.656644:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aac56 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.656651:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aac56 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.656662:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2100) 00010000:00010000:3.0:1787739049.656665:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739049.656668:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aac56 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.656674:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aac56 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.656684:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aac5d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.656690:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aac5d lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.656696:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aac5d lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.656700:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aba00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739049.656703:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aac5d lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.656709:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aac5d lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.656716:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xe2e7e698 (ffffa074331c2500) 00010000:00010000:3.0:1787739049.656719:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739049.656722:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aac5d lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.656727:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aac5d lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.657004:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aac4f lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948b6 expref: 16 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.657017:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2100) 00010000:00010000:3.0:1787739049.657019:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739049.657024:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aac4f lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948b6 expref: 16 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.657037:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab200, rc 0) 00000100:00100000:3.0:1787739049.657097:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17353:x1874579652390272:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 699us (861us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739049.657103:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23058 00000100:00100000:3.0:1787739049.665130:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739049.681801:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652390656 00000100:00100000:3.0:1787739049.681810:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652390656 00000100:00100000:3.0:1787739049.681872:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23060 00000100:00100000:3.0:1787739049.681882:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17353:x1874579652390656:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:3.0:1787739049.681988:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562cf00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665474944:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739049.682038:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739049.682096:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355ed780 going to sleep for 11 seconds 00000100:00100000:2.0:1787739049.682121:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665474944 00000100:00100000:2.0:1787739049.682126:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665474944 00000100:00100000:2.0:1787739049.682163:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2277 00000100:00100000:2.0:1787739049.682171:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561e940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665474944:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739049.682241:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561e940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665474944:12345-0@lo:1000:mdt00_000.0 Request processed in 70us (201us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739049.682247:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2277 00000100:00100000:3.0:1787739049.682318:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562cf00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665474944:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739049.682361:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562cf00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665475072:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739049.682386:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739049.682441:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355ed600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739049.682454:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665475072 00000100:00100000:2.0:1787739049.682458:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665475072 00000100:00100000:2.0:1787739049.682490:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2278 00000100:00100000:2.0:1787739049.682497:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665475072:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739049.682573:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665475072:12345-0@lo:1000:mdt00_000.0 Request processed in 75us (185us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739049.682580:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2278 00000100:00100000:3.0:1787739049.682638:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562cf00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665475072:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739049.683178:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665475200 00000100:00100000:3.0:1787739049.683182:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665475200 00000100:00100000:3.0:1787739049.683212:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2279 00000100:00100000:3.0:1787739049.683219:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665475200:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739049.683227:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739049.683240:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aac79 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aac72 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.683262:0:17834:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aac6b lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948bd expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.683272:0:17834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aac79 lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aac72 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.683293:0:17834:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aac6b lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948bd expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.683305:0:17834:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aac6b lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948bd expref: 15 pid: 6524 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739049.683316:0:17834:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d3480 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:17834:1874579665475328:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739049.683365:0:17834:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355ed600 going to sleep for 11 seconds 00000100:00100000:3.0:1787739049.692364:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00010000:00010000:2.0:1787739049.694985:0:17834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aac6b lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948bd expref: 15 pid: 6524 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739049.695002:0:17834:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d3480 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:17834:1874579665475328:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739049.695018:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aac79 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aac72 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.695026:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:2.0:1787739049.695029:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739049.695032:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa200, rc 0) 00000100:00100000:2.0:1787739049.695080:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665475200:12345-0@lo:101:mdt00_000.0 Request processed in 11861us (12030us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739049.695088:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2279 00000100:00100000:3.0:1787739049.696010:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665475584 00000100:00100000:3.0:1787739049.696016:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665475584 00000100:00100000:3.0:1787739049.696062:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2280 00000100:00100000:3.0:1787739049.696073:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665475584:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739049.696088:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739049.696104:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aac87 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aac80 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.696143:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aac87 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aac80 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.696156:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aac87 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aac80 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.696162:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xe2e7e698 (ffffa074331c2100) 00010000:00010000:3.0:1787739049.696164:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739049.696166:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab200, rc 0) 00000100:00100000:3.0:1787739049.696216:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665475584:12345-0@lo:101:mdt00_000.0 Request processed in 148us (337us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739049.696222:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2280 00000100:00100000:2.0:1787739049.696862:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383ed00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665475584:0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739049.696878:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1aac80 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aac87 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.696899:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1aac80 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aac87 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739049.696917:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:2.0:1787739049.696952:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356af840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665475712:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739049.696986:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.697013:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef000 going to sleep for 11 seconds 00000100:00100000:3.0:1787739049.697029:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665475712 00000100:00100000:3.0:1787739049.697034:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665475712 00000100:00100000:3.0:1787739049.697075:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2281 00000100:00100000:3.0:1787739049.697083:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665475712:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739049.697176:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665475712:12345-0@lo:1000:mdt00_000.0 Request processed in 93us (189us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739049.697184:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2281 00000100:00100000:2.0:1787739049.697223:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356af840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665475712:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739049.697264:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356af840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665475840:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739049.697287:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.697311:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef000 going to sleep for 11 seconds 00000100:00100000:3.0:1787739049.697406:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665475840 00000100:00100000:3.0:1787739049.697410:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665475840 00000100:00100000:3.0:1787739049.697463:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2282 00000100:00100000:3.0:1787739049.697471:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ad680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665475840:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739049.697543:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ad680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665475840:12345-0@lo:1000:mdt00_000.0 Request processed in 74us (254us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739049.697550:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2282 00000100:00100000:3.0:1787739049.697828:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665475968 00000100:00100000:3.0:1787739049.697833:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665475968 00000100:00100000:3.0:1787739049.697868:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2283 00000100:00100000:3.0:1787739049.697875:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665475968:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739049.697929:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665475968:12345-0@lo:1000:mdt00_000.0 Request processed in 54us (247us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739049.697936:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2283 00000100:00100000:3.0:1787739049.698401:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665476096 00000100:00100000:3.0:1787739049.698406:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665476096 00000100:00100000:3.0:1787739049.700189:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2284 00000100:00100000:3.0:1787739049.700202:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665476096:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739049.700312:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665476096:12345-0@lo:1000:mdt00_000.0 Request processed in 113us (2012us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739049.700320:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2284 00000100:00100000:3.0:1787739049.700537:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665476224 00000100:00100000:3.0:1787739049.700542:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665476224 00000100:00100000:3.0:1787739049.700569:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2285 00000100:00100000:3.0:1787739049.700575:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665476224:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739049.700645:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665476224:12345-0@lo:1000:mdt00_000.0 Request processed in 69us (170us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739049.700651:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2285 00000100:00100000:3.0:1787739049.700796:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665476352 00000100:00100000:3.0:1787739049.700800:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665476352 00000100:00100000:3.0:1787739049.700828:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2286 00000100:00100000:3.0:1787739049.700834:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665476352:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739049.700880:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665476352:12345-0@lo:1000:mdt00_000.0 Request processed in 46us (117us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739049.700886:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2286 00000100:00100000:3.0:1787739049.700981:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665476480 00000100:00100000:3.0:1787739049.700984:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665476480 00000100:00100000:3.0:1787739049.701007:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2287 00000100:00100000:3.0:1787739049.701013:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304260f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665476480:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739049.701067:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304260f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665476480:12345-0@lo:1000:mdt00_000.0 Request processed in 53us (115us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739049.701084:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2287 00000004:00080000:3.0:1787739049.703564:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 603 rpc_version 603 00000020:00100000:3.0:1787739049.703611:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739049.703621:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074080d1680 x1874579665476736/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739049.703645:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d1680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665476736:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739049.703718:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739049.703768:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355ed780 going to sleep for 11 seconds 00000100:00100000:3.0:1787739049.704762:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d1680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665476736:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739049.705230:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665476864 00000100:00100000:3.0:1787739049.705236:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665476864 00000100:00100000:3.0:1787739049.705273:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2289 00000100:00100000:3.0:1787739049.705279:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be43c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665476864:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739049.705294:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739049.705311:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aaca3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xae:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aac9c expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.705330:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aaca3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xae:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aac9c expref: 11 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.705341:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aaca3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xae:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aac9c expref: 11 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.705350:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xae:0x0].0x0 (ffffa074331c2500) 00010000:00010000:3.0:1787739049.705353:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739049.705356:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739049.705407:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be43c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665476864:12345-0@lo:101:mdt00_000.0 Request processed in 128us (220us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739049.706089:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2289 00010000:00010000:3.0:1787739049.706908:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739049.706915:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1aac72 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aac79 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.706923:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1da00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739049.706929:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1aac72 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aac79 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739049.706938:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1aac72 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aac79 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.706965:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1aac72 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aac79 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.706977:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739049.707075:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1aac72 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aac79 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.707084:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1aac72 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aac79 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.707100:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739049.707102:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739049.707103:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739049.746386:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739049.766515:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652391168 00000100:00100000:3.0:1787739049.766526:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652391168 00000100:00100000:3.0:1787739049.766588:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23062 00000100:00100000:3.0:1787739049.766598:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17364:x1874579652391168:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739049.766619:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739049.766642:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aacaa lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0948c4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.766706:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aacb1 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.766721:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aacb1 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.766731:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aacb1 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.766741:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aacb8 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.766752:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aac87 lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aac80 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.767495:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aacb8 lrc: 4/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.767532:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1aac87 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1aac80 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739049.767549:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d1a40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665477120:0@lo:104: 00000100:00100000:3.0:1787739049.767610:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739049.767671:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4dcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739049.768128:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665477248:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739049.768159:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739049.768225:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d1a40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665477120:0@lo:104: 00010000:00010000:3.0:1787739049.768240:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aacb8 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739049.776925:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 629 rpc_version 629 00000020:00100000:3.0:1787739049.776956:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739049.776965:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074080d1a40 x1874579665477504/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739049.776985:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d1a40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665477504:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739049.777027:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739049.777066:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4dcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739049.777103:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665477248:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739049.777144:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aacb8 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.777152:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aacb8 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739049.777231:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562fc00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665477632:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739049.777257:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739049.777275:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d840 going to sleep for 11 seconds 00000100:00100000:3.0:1787739049.777292:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665477504 00000100:00100000:3.0:1787739049.777298:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665477504 00000100:00100000:3.0:1787739049.777331:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665477632 00000100:00100000:3.0:1787739049.777335:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665477632 00000100:00100000:3.0:1787739049.777352:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2290 00000100:00100000:3.0:1787739049.777359:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665477504:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739049.777496:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665477504:12345-0@lo:1000:osp_up1-0.0 Request processed in 136us (468us total) trans 4294968259 rc 0/0 00000100:00100000:3.0:1787739049.777503:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2290 00000100:00100000:3.0:1787739049.777517:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2291 00000100:00100000:3.0:1787739049.777525:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6524:x1874579665477632:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739049.777572:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6524:x1874579665477632:12345-0@lo:1000:mdt00_001.0 Request processed in 48us (314us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739049.777578:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2291 00000100:00100000:3.0:1787739049.777615:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d1a40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665477504:0@lo:1000:osp_up1-0.0 00000004:00080000:3.0:1787739049.777624:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 630 rpc_version 630 00000020:00100000:3.0:1787739049.777634:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739049.777639:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743562f840 x1874579665477888/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739049.777652:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562f840 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665477888:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739049.777676:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739049.777692:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739049.777708:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562fc00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665477632:0@lo:1000:mdt00_001.0 00010000:00010000:3.0:1787739049.777747:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1aacbf lrc: 3/1,0 mode: --/PR res: [0x240000402:0x153:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.777774:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1aacbf lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x153:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.777783:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1aacbf lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x153:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739049.777805:0:6524:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1aacbf lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x153:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.777819:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aacb1 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.777826:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aacb1 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.777833:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aacb1 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.777840:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aacb1 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.777846:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739049.777851:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aacb1 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.777859:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aacb1 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.777868:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2000) 00010000:00010000:3.0:1787739049.777871:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739049.777875:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aacb1 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.777881:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aacb1 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.777893:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aacb8 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.777900:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aacb8 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.777906:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aacb8 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.777911:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739049.777914:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aacb8 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.777919:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aacb8 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.777927:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xe2e7e698 (ffffa074331c2100) 00010000:00010000:3.0:1787739049.777930:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739049.777932:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aacb8 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.777942:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aacb8 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.777975:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aacaa lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948c4 expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.777990:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1aacbf lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x153:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948c4 expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.777998:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x153:0x0].0x0 (ffffa074331c2e00) 00010000:00010000:3.0:1787739049.778001:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739049.778003:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa800, rc 0) 00000100:00100000:3.0:1787739049.778045:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17364:x1874579652391168:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 11448us (13038us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739049.778053:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23062 00000100:00100000:3.0:1787739049.778181:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665477888 00000100:00100000:3.0:1787739049.778185:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665477888 00000100:00100000:3.0:1787739049.778209:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2292 00000100:00100000:3.0:1787739049.778215:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665477888:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739049.778294:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665477888:12345-0@lo:1000:osp_up1-0.0 Request processed in 78us (615us total) trans 4294968260 rc 0/0 00000100:00100000:3.0:1787739049.778300:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2292 00000100:00100000:3.0:1787739049.778333:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562f840 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665477888:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739049.788176:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652391296 00000100:00100000:3.0:1787739049.788189:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652391296 00000100:00100000:3.0:1787739049.788254:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23063 00000100:00100000:3.0:1787739049.788264:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17364:x1874579652391296:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739049.788288:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739049.788307:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1aacc6 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x153:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0948cb expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.788368:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aaccd lrc: 3/1,0 mode: --/PR res: [0x240000402:0x153:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.788383:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aaccd lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x153:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.788393:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aaccd lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x153:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739049.788505:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x153:0x0]: returning size 4096 00000004:00200000:3.0:1787739049.788510:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739049.788547:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1aacc6 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x153:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948cb expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.788564:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aaccd lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x153:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948cb expref: 10 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.788573:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x153:0x0].0x0 (ffffa07433235700) 00010000:00010000:3.0:1787739049.788575:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739049.788579:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa400, rc 0) 00000100:00100000:3.0:1787739049.788638:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17364:x1874579652391296:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 374us (2063us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739049.788645:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23063 00000100:00100000:3.0:1787739049.806527:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739049.806605:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652391424 00000100:00100000:2.0:1787739049.806612:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652391424 00000100:00100000:2.0:1787739049.806671:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23064 00000100:00100000:2.0:1787739049.806681:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17364:x1874579652391424:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739049.806706:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739049.806727:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1aacd4 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xae:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0948d2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.806798:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1aacdb lrc: 3/1,0 mode: --/PR res: [0x240000400:0xae:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.806811:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1aacdb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xae:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.806823:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1aacdb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xae:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739049.806874:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xae:0x0]: returning size 4096 00000004:00200000:2.0:1787739049.806880:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739049.806913:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1aacd4 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xae:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948d2 expref: 12 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.806929:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1aacdb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xae:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948d2 expref: 11 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.806938:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xae:0x0].0x0 (ffffa07433235200) 00010000:00010000:2.0:1787739049.806941:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739049.806944:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d200, rc 0) 00000100:00100000:2.0:1787739049.806999:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17364:x1874579652391424:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 319us (490us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739049.807006:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23064 00000020:00080000:3.0:1787739049.828310:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 333 batchid updated 00000100:00100000:3.0:1787739049.829248:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665478400 00000100:00100000:3.0:1787739049.829264:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665478400 00000100:00100000:3.0:1787739049.829327:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2293 00000100:00100000:3.0:1787739049.829341:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665478400:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739049.829561:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665478400:12345-0@lo:1000:osp_up1-0.0 Request processed in 222us (697us total) trans 4294968261 rc 0/0 00000100:00100000:3.0:1787739049.829586:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2293 00000100:00100000:3.0:1787739049.844868:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739049.850308:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739049.851638:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652391936 00000100:00100000:3.0:1787739049.851656:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652391936 00000100:00100000:3.0:1787739049.851732:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23066 00000100:00100000:3.0:1787739049.851743:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17364:x1874579652391936:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:3.0:1787739049.851831:0:13793:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739049.851857:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aaccd lrc: 0/0,0 mode: --/PR res: [0x240000402:0x153:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948cb expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.851877:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x153:0x0].0x0 (ffffa07433235700) 00010000:00010000:3.0:1787739049.851882:0:13793:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739049.851934:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1aacf0 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.851943:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1aacf0 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.851953:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1aacf0 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739049.851977:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb7840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665478528:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787739049.852037:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739049.852077:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4dcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739049.852254:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665478528 00000100:00100000:3.0:1787739049.852259:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665478528 00000100:00100000:3.0:1787739049.852306:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2294 00000100:00100000:3.0:1787739049.852313:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+5:13793:x1874579665478528:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739049.852323:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739049.852471:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aacf7 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aacf0 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.852713:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aacf7 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aacf0 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.852737:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aacf7 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aacf0 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.852744:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2100) 00010000:00010000:3.0:1787739049.852747:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739049.852822:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aac00, rc 0) 00000100:00100000:3.0:1787739049.852878:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:13793:x1874579665478528:12345-0@lo:101:mdt00_004.0 Request processed in 565us (840us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739049.852885:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2294 00000100:00100000:3.0:1787739049.852940:0:13793:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983269 gen 2 00000004:00080000:3.0:1787739049.852945:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983266 committed transno 4294983269 00000020:00080000:3.0:1787739049.852951:0:13793:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07302e89a80 refcount 2 committed 0 result 0 batchid 323 00000020:00080000:3.0:1787739049.852954:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00500 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07302e89300 00000020:00080000:3.0:1787739049.852958:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].316 00000020:00080000:3.0:1787739049.852966:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00400 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073027db800 00000020:00080000:3.0:1787739049.852969:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].315 00000004:00080000:3.0:1787739049.852998:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983268 committed transno 4294983269 00000004:00080000:3.0:1787739049.853007:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983269 committed transno 4294983269 00000100:00100000:3.0:1787739049.853027:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb7840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665478528:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739049.853041:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1aacf0 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aacf7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.853052:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1aacf0 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aacf7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739049.853074:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739049.853183:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1aacfe lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.853191:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1aacfe lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.853201:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1aacfe lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739049.853216:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb7840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665478656:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787739049.853256:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739049.853343:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4dcc0 going to sleep for 11 seconds 00000020:00080000:3.0:1787739049.853356:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 323 committed batchid 321 00000020:00080000:3.0:1787739049.853401:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 321 00000100:00100000:3.0:1787739049.853875:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665478784 00000100:00100000:3.0:1787739049.853879:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665478784 00000100:00100000:3.0:1787739049.853921:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2296 00000100:00100000:3.0:1787739049.853935:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665478784:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739049.854005:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665478784:12345-0@lo:1000:mdt00_004.0 Request processed in 70us (170us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739049.854012:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2296 00000100:00100000:3.0:1787739049.854316:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665478912 00000100:00100000:3.0:1787739049.854321:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665478912 00000100:00100000:3.0:1787739049.854390:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2297 00000100:00100000:3.0:1787739049.854398:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665478912:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739049.854489:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665478912:12345-0@lo:1000:mdt00_004.0 Request processed in 92us (212us total) trans 0 rc 1/1 00000100:00100000:3.0:1787739049.854496:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2297 00010000:00010000:0.0:1787739049.855279:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1aad13 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x153:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aad0c expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.855287:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x153:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:0.0:1787739049.855289:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739049.855292:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a400, rc 0) 00000100:00100000:0.0:1787739049.855328:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665479040:12345-0@lo:101:mdt00_004.0 Request processed in 115us (666us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739049.855335:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2298 00000100:00100000:3.0:1787739049.860622:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739049.860682:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652392064 00000100:00100000:0.0:1787739049.860689:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652392064 00010000:00010000:0.0:1787739049.860723:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1aacdb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xae:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948d2 expref: 11 pid: 7944 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739049.860760:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1324 00000100:00100000:0.0:1787739049.860769:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb6940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2408:x1874579652392064:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739049.860785:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739049.860791:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1aacdb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xae:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948d2 expref: 11 pid: 7944 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.860804:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1aacdb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xae:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948d2 expref: 11 pid: 7944 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.860828:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1aacdb lrc: 0/0,0 mode: --/PR res: [0x240000400:0xae:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948d2 expref: 11 pid: 7944 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.860843:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xae:0x0].0x0 (ffffa07433235200) 00010000:00010000:0.0:1787739049.860847:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1aad21 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xae:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.860857:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1aad21 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xae:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.860864:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1aad21 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xae:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.860896:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739049.860946:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb6940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652392064:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 177us (325us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739049.860955:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1324 00010000:00010000:3.0:1787739049.861036:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1aad21 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.861044:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1aad21 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.861099:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aad28 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xae:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.861107:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aad28 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.861125:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aad28 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739049.861150:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb7840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665479296:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787739049.861195:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739049.861246:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665479296 00000100:00100000:3.0:1787739049.861249:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d840 going to sleep for 11 seconds 00000100:00100000:0.0:1787739049.861252:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665479296 00000100:00100000:0.0:1787739049.861285:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2299 00000100:00100000:0.0:1787739049.861293:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d1680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665479296:12345-0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739049.861308:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739049.861321:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aad2f lrc: 2/0,0 mode: --/EX res: [0x200000401:0xae:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aad28 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.861342:0:8448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aace9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948d9 expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.861351:0:8448:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aad2f lrc: 4/0,0 mode: --/EX res: [0x200000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aad28 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.861374:0:8448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aace9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948d9 expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.861388:0:8448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aace9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xae:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948d9 expref: 13 pid: 6524 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739049.861399:0:8448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f2d00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8448:1874579665479424:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739049.861466:0:8448:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739049.865571:0:8448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f2d00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8448:1874579665479424:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739049.865609:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aad2f lrc: 3/0,0 mode: --/EX res: [0x200000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aad28 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.865622:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xae:0x0].0x0 (ffffa074331c2500) 00010000:00010000:0.0:1787739049.865625:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739049.865628:0:8448:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aad2f lrc: 3/0,0 mode: --/EX res: [0x200000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aad28 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.865638:0:8448:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aad2f lrc: 3/0,0 mode: --/EX res: [0x200000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aad28 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.865644:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609ba00, rc 0) 00000100:00100000:0.0:1787739049.865702:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d1680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665479296:12345-0@lo:101:mdt00_004.0 Request processed in 4410us (4505us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739049.865711:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2299 00000100:00100000:3.0:1787739049.866684:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb7840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665479296:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739049.866713:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aad28 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aad2f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.866726:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aad28 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aad2f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739049.867608:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739049.868016:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665479552 00000100:00100000:3.0:1787739049.868022:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665479552 00000100:00100000:3.0:1787739049.868064:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1009 00000100:00100000:3.0:1787739049.868071:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665479552:12345-0@lo:105: 00010000:00010000:3.0:1787739049.868082:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aad28 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aad2f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.868092:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aad28 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aad2f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.868107:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aad28 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aad2f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.868124:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aad28 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1aad2f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.868131:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aad28 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1aad2f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.868145:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aad28 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1aad2f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.868177:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069aa000) 00000100:00100000:3.0:1787739049.868253:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665479552:12345-0@lo:105: Request processed in 182us (316us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739049.868261:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1009 00000100:00100000:2.0:1787739049.868305:0:6508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383da40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665479552:0@lo:105: 00010000:00010000:2.0:1787739049.868321:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739049.868364:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:2408:x1874579652392192:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 604us (759us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739049.868376:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1325 00000100:00100000:3.0:1787739049.871702:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665479680 00000100:00100000:3.0:1787739049.871709:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665479680 00000100:00100000:0.0:1787739049.872010:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2300 00000100:00100000:0.0:1787739049.872030:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665479680:12345-0@lo:403:mdt00_004.0 00000100:00100000:0.0:1787739049.874440:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665479680:12345-0@lo:403:mdt00_004.0 Request processed in 2391us (2777us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739049.874453:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2300 00000100:00100000:3.0:1787739049.874772:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665479680:0@lo:403:mdt00_004.0 00000040:00080000:3.0:1787739049.875795:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].317, 664 off484528 00000040:00080000:3.0:1787739049.878463:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].318, 744 off530512 00000004:00080000:3.0:1787739049.878475:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07410b4d840 with version 604 00000100:00100000:3.0:1787739049.878868:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665479936 00000100:00100000:3.0:1787739049.878874:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665479936 00000100:00100000:3.0:1787739049.878929:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2301 00000100:00100000:3.0:1787739049.878937:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb7840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665479936:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739049.879612:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb7840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665479936:12345-0@lo:1000:osp_up0-1.0 Request processed in 675us (971us total) trans 4294983270 rc 0/0 00000100:00100000:3.0:1787739049.879621:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2301 00000100:00100000:0.0:1787739049.879831:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665479936:0@lo:1000:osp_up0-1.0 00000004:00200000:3.0:1787739049.879880:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x153:0x0]: returning size 4096 00010000:00010000:3.0:1787739049.879899:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1aad21 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.879908:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1aad21 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.879920:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1aad21 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.879927:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739049.879931:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1aad21 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.880059:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1aad21 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.880075:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xae:0x0].0x0 (ffffa07433235200) 00010000:00010000:3.0:1787739049.880077:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739049.880082:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1aad21 lrc: 2/0,0 mode: --/EX res: [0x240000400:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.880088:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1aad21 lrc: 0/0,0 mode: --/EX res: [0x240000400:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.880101:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aad28 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aad2f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.880119:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aad28 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aad2f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.880125:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aad28 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aad2f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.880178:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739049.880188:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xae:0x0] 00010000:00010000:3.0:1787739049.880195:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1aad1a lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x153:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.880201:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1aad1a lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x153:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.880206:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1aad1a lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x153:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.880211:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739049.880214:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1aad1a lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x153:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.880219:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1aad1a lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x153:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.880227:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x153:0x0].0x0 (ffffa07433235700) 00010000:00010000:3.0:1787739049.880229:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739049.880232:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1aad1a lrc: 2/0,0 mode: --/EX res: [0x240000402:0x153:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.880238:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1aad1a lrc: 0/0,0 mode: --/EX res: [0x240000402:0x153:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.880248:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1aad0c lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x153:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aad13 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.880255:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1aad0c lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x153:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aad13 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.880274:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1aad0c lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x153:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aad13 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.880523:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1787739049.880532:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aad28 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aad2f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.880540:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739049.880544:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aad28 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aad2f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739049.880553:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aad28 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aad2f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.880582:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aad28 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aad2f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.880588:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1787739049.880646:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356aed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665480064:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739049.880691:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787739049.880738:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aad28 lrc: 1/0,0 mode: --/EX res: [0x200000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aad2f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.880747:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aad28 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aad2f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.880763:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739049.880766:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1aad0c lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x153:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aad13 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.880772:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1de00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739049.880775:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1aad0c lrc: 5/0,0 mode: EX/EX res: [0x240000402:0x153:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aad13 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739049.880782:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1aad0c lrc: 5/0,0 mode: EX/EX res: [0x240000402:0x153:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aad13 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.880799:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1aad0c lrc: 4/0,0 mode: --/EX res: [0x240000402:0x153:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aad13 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.880806:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739049.880858:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00000004:00080000:3.0:1787739049.880870:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739049.880875:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1aacf0 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aacf7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.880882:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1aacf0 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aacf7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.880889:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1aacf0 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aacf7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.880908:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739049.880917:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739049.880923:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1aacfe lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aad05 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.880936:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1aacfe lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aad05 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739049.881014:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17364:x1874579652391936:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 29271us (30719us total) trans 4294968262 rc 0/0 00000100:00100000:3.0:1787739049.881027:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23066 00010000:00010000:3.0:1787739049.881056:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739049.881070:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665480064 00000100:00100000:3.0:1787739049.881075:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665480064 00000100:00100000:3.0:1787739049.881123:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1116 00000100:00100000:3.0:1787739049.881132:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba6050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+10:3652:x1874579665480064:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739049.881144:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739049.881161:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aad2f lrc: 0/0,0 mode: --/EX res: [0x200000401:0xae:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aad28 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.881172:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xae:0x0].0x0 (ffffa074331c2500) 00010000:00010000:3.0:1787739049.881179:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739049.881238:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1aad0c lrc: 1/0,0 mode: --/EX res: [0x240000402:0x153:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aad13 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.881246:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1aad0c lrc: 0/0,0 mode: --/EX res: [0x240000402:0x153:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aad13 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.881268:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739049.881270:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739049.881272:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739049.881354:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba6050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+9:3652:x1874579665480064:12345-0@lo:103:ldlm_bl.0 Request processed in 223us (662us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739049.881362:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1116 00000100:00100000:0.0:1787739049.881664:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356aed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665480064:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739049.881680:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787739049.881687:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ade00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665480192:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739049.881713:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739049.881854:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665480320:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739049.881881:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.882029:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665480192 00000100:00100000:2.0:1787739049.882034:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665480192 00000100:00100000:3.0:1787739049.882079:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665480320 00000100:00100000:3.0:1787739049.882083:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665480320 00000100:00100000:3.0:1787739049.882125:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1117 00000100:00100000:3.0:1787739049.882133:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f2d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+10:3652:x1874579665480192:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739049.882142:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739049.882155:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1aad13 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x153:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aad0c expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.882166:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x153:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:3.0:1787739049.882172:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739049.882206:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f2d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+9:3652:x1874579665480192:12345-0@lo:103:ldlm_bl.0 Request processed in 74us (492us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739049.882211:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1117 00000100:00100000:3.0:1787739049.882226:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1118 00000100:00100000:3.0:1787739049.882232:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f2580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+8:3650:x1874579665480320:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739049.882239:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739049.882250:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aacf7 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aacf0 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.882368:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2100) 00010000:00010000:3.0:1787739049.882375:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739049.882481:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665480320:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739049.882551:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f2580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+7:3650:x1874579665480320:12345-0@lo:103:ldlm_bl.0 Request processed in 319us (668us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739049.882559:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1118 00000100:00100000:3.0:1787739049.882589:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ade00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665480192:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739049.935482:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652392320 00000100:00100000:2.0:1787739049.935498:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652392320 00000100:00100000:2.0:1787739049.935561:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23067 00000100:00100000:2.0:1787739049.935572:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17365:x1874579652392320:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739049.935596:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739049.935616:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1aad36 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0948e0 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.935682:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aad3d lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.935701:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aad3d lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.935713:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aad3d lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.935727:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aad44 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.935735:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aad44 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.935759:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aad44 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.935800:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aad3d lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.935807:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aad3d lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.935815:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aad3d lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.935822:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aad3d lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.935829:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739049.935833:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aad3d lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.935840:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aad3d lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.935853:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b200) 00010000:00010000:2.0:1787739049.935856:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739049.935861:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aad3d lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.935867:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aad3d lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.935883:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aad44 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.935889:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aad44 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.935896:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aad44 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.935902:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739049.935905:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aad44 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.935911:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aad44 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.935919:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x7423dc1f (ffffa074419d5800) 00010000:00010000:2.0:1787739049.935921:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739049.935924:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aad44 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.935930:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aad44 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.935966:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1aad36 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948e0 expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.935976:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b200) 00010000:00010000:2.0:1787739049.935979:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739049.935982:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1aad36 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948e0 expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.935991:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d400, rc 0) 00000100:00100000:2.0:1787739049.936058:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17365:x1874579652392320:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 487us (9566us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739049.936066:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23067 00000100:00100000:3.0:1787739049.942776:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739049.955598:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739049.956511:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665480448 00000100:00100000:0.0:1787739049.956516:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665480448 00000100:00100000:3.0:1787739049.956843:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665480448:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739049.956904:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be5e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665480576:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739049.956934:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739049.956976:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739049.957064:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665480576 00000100:00100000:0.0:1787739049.957069:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665480576 00000100:00100000:0.0:1787739049.957119:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2303 00000100:00100000:0.0:1787739049.957129:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb70c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665480576:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739049.957219:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb70c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665480576:12345-0@lo:1000:mdt00_004.0 Request processed in 93us (284us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739049.957227:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2303 00000100:00100000:2.0:1787739049.957519:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665480576:0@lo:1000:mdt00_004.0 00010000:00010000:2.0:1787739049.957570:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1aad59 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.957579:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1aad59 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.957588:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1aad59 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739049.957602:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ac780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665480704:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1787739049.957633:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.957690:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074157c2780 going to sleep for 11 seconds 00000100:00100000:0.0:1787739049.957722:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665480704 00000100:00100000:0.0:1787739049.957727:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665480704 00000100:00100000:0.0:1787739049.957805:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2304 00000100:00100000:0.0:1787739049.957814:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665480704:12345-0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739049.957824:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739049.957840:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aad60 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aad59 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.957870:0:18535:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1aaba7 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094893 expref: 9 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.957974:0:18535:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aad60 lrc: 4/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aad59 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.958002:0:18535:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1aaba7 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094893 expref: 9 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.958016:0:18535:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1aaba7 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094893 expref: 9 pid: 6523 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739049.958027:0:18535:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f0000 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:18535:1874579665480832:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739049.958090:0:18535:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739049.968314:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ac780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665480704:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739049.968331:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1aad59 lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aad60 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.968341:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1aad59 lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aad60 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739049.968579:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739049.968637:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652392832 00000100:00100000:0.0:1787739049.968645:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652392832 00010000:00010000:0.0:1787739049.968678:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1aaba7 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094893 expref: 10 pid: 6523 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739049.968711:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1326 00000100:00100000:0.0:1787739049.968721:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ade00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652392832:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739049.968747:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739049.968759:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1aaba7 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094893 expref: 10 pid: 6523 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.968776:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1aaba7 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094893 expref: 10 pid: 6523 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.968796:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1aaba7 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094893 expref: 10 pid: 6523 timeout: 736 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.968809:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b200) 00010000:00010000:0.0:1787739049.968814:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aad60 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aad59 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.968823:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aad60 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1aad59 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.968832:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aad60 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1aad59 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.968865:0:6509:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aad60 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aad59 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739049.968877:0:6509:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f0000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001_UUID:6509:1874579665480960:0@lo:105: 00000100:00100000:0.0:1787739049.968924:0:6509:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739049.968962:0:6509:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4e40 going to sleep for 11 seconds 00010000:00010000:2.0:1787739049.969958:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1aad59 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aad60 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.969967:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1aad59 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aad60 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739049.969990:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:2.0:1787739049.970022:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aad67 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.970029:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aad67 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.970039:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aad67 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739049.970055:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ac780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665481088:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1787739049.970090:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.970125:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074157c2780 going to sleep for 11 seconds 00000100:00100000:3.0:1787739049.970200:0:6509:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f0000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001_UUID:6509:1874579665480960:0@lo:105: 00010000:00010000:3.0:1787739049.970214:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739049.970261:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ade00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:2408:x1874579652392832:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 1541us (1681us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739049.970269:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1326 00000100:00100000:0.0:1787739049.971452:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665481088 00000100:00100000:0.0:1787739049.971462:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665481088 00000100:00100000:0.0:1787739049.971512:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2305 00000100:00100000:0.0:1787739049.971519:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356aed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:13793:x1874579665481088:12345-0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739049.971538:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739049.971556:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1aad6e lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aad67 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.971580:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1aad6e lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aad67 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.971593:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1aad6e lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aad67 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.971601:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x7423dc1f (ffffa07403a6f100) 00010000:00010000:0.0:1787739049.971604:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739049.971607:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a600, rc 0) 00000100:00100000:0.0:1787739049.971991:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356aed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665481088:12345-0@lo:101:mdt00_004.0 Request processed in 472us (1899us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739049.971999:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2305 00000100:00100000:2.0:1787739049.972713:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ac780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665481088:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739049.972741:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aad67 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aad6e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.972759:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aad67 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aad6e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739049.972776:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:2.0:1787739049.972821:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ac780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665481216:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739049.972858:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.972919:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739049.973375:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665481216 00000100:00100000:3.0:1787739049.973389:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665481216 00000100:00100000:3.0:1787739049.973468:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2306 00000100:00100000:3.0:1787739049.973486:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356af840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665481216:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739049.973639:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356af840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665481216:12345-0@lo:1000:mdt00_004.0 Request processed in 152us (775us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739049.973647:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2306 00000100:00100000:2.0:1787739049.973881:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ac780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665481216:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739049.973926:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ac780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665481344:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739049.973945:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.973993:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d600 going to sleep for 11 seconds 00000100:00100000:3.0:1787739049.974016:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665481344 00000100:00100000:3.0:1787739049.974020:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665481344 00000100:00100000:3.0:1787739049.974047:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2307 00000100:00100000:3.0:1787739049.974052:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356acf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665481344:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739049.974179:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356acf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665481344:12345-0@lo:1000:mdt00_004.0 Request processed in 126us (231us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739049.974186:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2307 00000100:00100000:2.0:1787739049.974186:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ac780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665481344:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739049.974252:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ac780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665481472:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739049.974275:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.974296:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d600 going to sleep for 11 seconds 00000100:00100000:3.0:1787739049.974336:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665481472 00000100:00100000:3.0:1787739049.974340:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665481472 00000100:00100000:3.0:1787739049.974371:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2308 00000100:00100000:3.0:1787739049.974377:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665481472:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739049.974449:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665481472:12345-0@lo:1000:mdt00_004.0 Request processed in 72us (172us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739049.974455:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2308 00000100:00100000:2.0:1787739049.974555:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ac780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665481472:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739049.974598:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ac780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665481600:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739049.974621:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.974643:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739049.975019:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ac780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665481600:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739049.975061:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665481728:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739049.975082:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.975149:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739049.975788:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665481728:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739049.975825:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665481856:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739049.975848:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.975900:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739049.976126:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665481856:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739049.976155:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665481984:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739049.976189:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739049.976399:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d600 going to sleep for 11 seconds 00000100:00100000:0.0:1787739049.976580:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665481984 00000100:00100000:0.0:1787739049.976585:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665481984 00000100:00100000:2.0:1787739049.976707:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2312 00000100:00100000:2.0:1787739049.976715:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ac780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665481984:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739049.976770:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ac780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665481984:12345-0@lo:1000:mdt00_004.0 Request processed in 57us (580us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739049.976777:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2312 00000100:00100000:2.0:1787739049.976959:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665481984:0@lo:1000:mdt00_004.0 00000040:00080000:2.0:1787739049.977914:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].331, 2168 off510304 00000040:00080000:2.0:1787739049.978002:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].330, 2368 off551384 00000004:00080000:2.0:1787739049.978008:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730688d540 with version 632 00000004:00080000:0.0:1787739049.978090:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 632 rpc_version 632 00000020:00100000:0.0:1787739049.978138:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739049.978147:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0730432d680 x1874579665482240/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739049.978172:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432d680 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665482240:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739049.978223:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739049.978252:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739049.980185:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432d680 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665482240:0@lo:1000:osp_up1-0.0 00010000:00010000:3.0:1787739049.982272:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aad75 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x14e9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.982290:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aad75 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14e9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.982302:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aad75 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14e9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.982322:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aad7c lrc: 3/0,1 mode: --/PW res: [0x200000400:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.982329:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aad7c lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.982337:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aad7c lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.982370:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1aad83 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.982377:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1aad83 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739049.982387:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1aad83 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739049.982409:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb6940 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665482368:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787739049.982512:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739049.982550:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739049.984344:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665482368 00000100:00100000:2.0:1787739049.984356:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665482368 00000100:00100000:2.0:1787739049.984464:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2314 00000100:00100000:2.0:1787739049.984475:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:13793:x1874579665482368:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739049.984502:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739049.984526:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aad8a lrc: 2/0,0 mode: --/PW res: [0x240000401:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aad83 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.984555:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aad8a lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aad83 expref: 12 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.984569:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aad8a lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aad83 expref: 12 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739049.984576:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb1:0x0].0x0 (ffffa074419d5b00) 00010000:00010000:2.0:1787739049.984579:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739049.984583:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d000, rc 0) 00000100:00100000:2.0:1787739049.984656:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:13793:x1874579665482368:12345-0@lo:101:mdt00_004.0 Request processed in 184us (2143us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739049.984663:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2314 00000100:00100000:0.0:1787739049.985187:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb6940 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665482368:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739049.985206:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1aad83 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aad8a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.985220:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1aad83 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aad8a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739049.985244:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000353cd71a reply state 00000000210df725 transno 4294983271 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aad7c lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.985257:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aad7c lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.985263:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb1:0x0].0x0 (ffffa074331c2500) 00010000:00010000:0.0:1787739049.985266:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739049.985268:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739049.985273:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1aad83 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aad8a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.985281:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1aad83 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aad8a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739049.985292:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xb1:0x0] 00000004:00010000:0.0:1787739049.985299:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000353cd71a reply state 00000000210df725 transno 4294983271 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aad75 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14e9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.985307:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aad75 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x14e9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.985313:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14e9:0x0].0x0 (ffffa074331c2100) 00010000:00010000:0.0:1787739049.985315:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739049.985317:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787739049.985322:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14e9:0x0]: returning size 4096 00000004:00080000:0.0:1787739049.985331:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739049.985340:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1aad59 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aad60 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.985348:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1aad59 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aad60 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.985354:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1aad59 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aad60 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.985380:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739049.985384:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739049.985388:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aad67 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aad6e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739049.985395:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aad67 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aad6e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739049.985514:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17365:x1874579652392704:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 29516us (29922us total) trans 4294983271 rc 0/0 00000100:00100000:0.0:1787739049.985521:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23069 00000100:00100000:2.0:1787739050.002677:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739050.013586:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739050.013660:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652393216 00000100:00100000:3.0:1787739050.013670:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652393216 00000100:00100000:3.0:1787739050.013748:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23071 00000100:00100000:3.0:1787739050.013758:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17367:x1874579652393216:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739050.013779:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739050.013801:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1aad91 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0948ee expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.013863:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1aad98 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.013877:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1aad98 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.013888:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1aad98 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.013904:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1aad9f lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.013914:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1aad6e lrc: 2/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aad67 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.013921:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1aad9f lrc: 4/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.013954:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1aad6e lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1aad67 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.013970:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f70000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665482624:0@lo:104: 00000100:00100000:3.0:1787739050.014017:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.014072:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e890c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.014473:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665482624 00000100:00100000:0.0:1787739050.014478:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665482624 00000100:00100000:0.0:1787739050.014519:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1011 00000100:00100000:0.0:1787739050.014527:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f721c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665482624:12345-0@lo:104: 00010000:00010000:0.0:1787739050.014540:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aad67 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aad6e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.014618:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787739050.014624:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f721c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665482624:12345-0@lo:104: Request processed in 99us (608us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.014630:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1011 00010000:00010000:0.0:1787739050.014652:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739050.014658:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aad67 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aad6e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.014667:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739050.014671:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aad67 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1aad6e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739050.014679:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aad67 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1aad6e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.014703:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aad67 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aad6e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.014710:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739050.014728:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aad67 lrc: 1/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aad6e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.014839:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aad67 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aad6e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.014859:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739050.014861:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739050.014863:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739050.014871:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787739050.014882:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665482752:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739050.014912:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.014943:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665482752 00000100:00100000:0.0:1787739050.014947:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665482752 00010000:00010000:0.0:1787739050.014964:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1aad6e lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1aad67 expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739050.014987:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1327 00000100:00100000:0.0:1787739050.014994:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+11:3653:x1874579665482752:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739050.015005:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787739050.015512:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f70000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665482624:0@lo:104: 00010000:00010000:3.0:1787739050.015531:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1aad9f lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.022485:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1aacfe lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aad05 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739050.022504:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1aacfe lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aad05 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.022546:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1aacfe lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aad05 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.022558:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1787739050.022726:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f70000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665482880:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:0.0:1787739050.022808:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787739050.024041:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1aacfe lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xe2e7e698 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aad05 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787739050.024057:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968263 is committed 00000001:00080000:3.0:1787739050.024073:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968262 is committed 00000020:00080000:3.0:1787739050.024080:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 323 batchid updated 00000020:00080000:3.0:1787739050.024090:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07302e89780 refcount 2 committed 0 result 0 batchid 324 00000020:00080000:3.0:1787739050.024093:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00200 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07302e890c0 00000020:00080000:3.0:1787739050.024096:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].317 00000020:00080000:3.0:1787739050.024098:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00100 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa074080f4a00 00000020:00080000:3.0:1787739050.024100:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].316 00000020:00080000:3.0:1787739050.024118:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 322 batchid updated 00010000:00010000:3.0:1787739050.024143:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aac8e lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x153:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.024151:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aac8e lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x153:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.024158:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aac8e lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x153:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.024164:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739050.024168:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aac8e lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x153:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.024175:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aac8e lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x153:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.024188:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x153:0x0].0x0 (ffffa07433235700) 00010000:00010000:3.0:1787739050.024191:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739050.024194:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aac8e lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x153:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.024200:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aac8e lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x153:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.024213:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aac95 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xae:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.024220:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aac95 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xae:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.024225:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aac95 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xae:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.024230:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739050.024233:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aac95 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xae:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.024238:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aac95 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xae:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.024245:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xae:0x0].0x0 (ffffa07433235200) 00010000:00010000:3.0:1787739050.024247:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739050.024250:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aac95 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xae:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.024256:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aac95 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xae:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1787739050.024293:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 324 committed batchid 323 00000004:00080000:3.0:1787739050.024394:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07302e890c0 with version 605 00000020:00080000:3.0:1787739050.024434:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 322 cancel update log [0x200000401:0x3:0x0].315: rc = 0 00000004:00080000:2.0:1787739050.024447:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 605 rpc_version 605 00000020:00100000:2.0:1787739050.024472:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000020:00080000:3.0:1787739050.024475:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 322 cancel update log [0x240000400:0x3:0x0].314: rc = 0 00000100:00100000:2.0:1787739050.024480:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074156ef840 x1874579665483136/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000004:00080000:3.0:1787739050.024496:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07302e89180 with version 606 00000100:00100000:2.0:1787739050.024498:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665483136:0@lo:1000:osp_up0-1.0 00000020:00080000:3.0:1787739050.024499:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 323 cancel update log [0x200000401:0x3:0x0].316: rc = 0 00000020:00080000:3.0:1787739050.024516:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 323 cancel update log [0x240000400:0x3:0x0].315: rc = 0 00000020:00080000:3.0:1787739050.024522:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 323 00000100:00100000:2.0:1787739050.024536:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.024570:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688df00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.024999:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665483136 00000100:00100000:3.0:1787739050.025005:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665483136 00000100:00100000:3.0:1787739050.025088:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2315 00000100:00100000:3.0:1787739050.025096:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665483136:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739050.025233:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665483136:12345-0@lo:1000:osp_up0-1.0 Request processed in 137us (696us total) trans 4294983272 rc 0/0 00000100:00100000:3.0:1787739050.025241:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2315 00000100:00100000:3.0:1787739050.027887:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665483136:0@lo:1000:osp_up0-1.0 00000004:00080000:3.0:1787739050.027904:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 606 rpc_version 606 00000020:00100000:3.0:1787739050.027928:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739050.027936:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303f72d00 x1874579665483392/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739050.027955:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f72d00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665483392:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739050.028015:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.028048:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.028177:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665483392 00000100:00100000:2.0:1787739050.028183:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665483392 00000100:00100000:2.0:1787739050.028232:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2316 00000100:00100000:2.0:1787739050.028241:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07436d8cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665483392:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739050.028637:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07436d8cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665483392:12345-0@lo:1000:osp_up0-1.0 Request processed in 396us (620us total) trans 4294983273 rc 0/0 00000100:00100000:2.0:1787739050.028645:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2316 00000100:00100000:3.0:1787739050.029882:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f72d00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665483392:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739050.030987:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f70000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665482880:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:0.0:1787739050.031166:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665482752:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739050.033631:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1aad9f lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.033650:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1aad9f lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.033771:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f703c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665483520:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739050.033870:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.033918:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.034633:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665483520 00000100:00100000:2.0:1787739050.034642:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665483520 00000100:00100000:2.0:1787739050.036045:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2317 00000100:00100000:2.0:1787739050.036057:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f72940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665483520:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739050.036394:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f72940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665483520:12345-0@lo:1000:mdt00_003.0 Request processed in 338us (2523us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.036401:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2317 00000100:00100000:3.0:1787739050.048127:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739050.048233:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652393344 00000100:00100000:0.0:1787739050.048242:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652393344 00000100:00100000:0.0:1787739050.048340:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23072 00000100:00100000:0.0:1787739050.048350:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f73840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17367:x1874579652393344:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739050.048372:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739050.048392:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aadad lrc: 2/0,0 mode: --/CR res: [0x200000402:0x14e9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0948f5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.048473:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1aadb4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14e9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.048491:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1aadb4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14e9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.048504:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1aadb4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14e9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739050.048594:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14e9:0x0]: returning size 4096 00000004:00200000:0.0:1787739050.048599:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739050.048649:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aadad lrc: 0/0,0 mode: --/CR res: [0x200000402:0x14e9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948f5 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.048666:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1aadb4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14e9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948f5 expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.048675:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14e9:0x0].0x0 (ffffa074331c2100) 00010000:00010000:0.0:1787739050.048678:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739050.048681:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a800, rc 0) 00000100:00100000:0.0:1787739050.048745:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f73840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17367:x1874579652393344:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 397us (636us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.048752:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23072 00000100:00100000:2.0:1787739050.055204:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739050.060746:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739050.060873:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652393600 00000100:00100000:2.0:1787739050.060893:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652393600 00000100:00100000:2.0:1787739050.060945:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23074 00000100:00100000:2.0:1787739050.060955:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17367:x1874579652393600:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739050.060977:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739050.060997:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1aadc9 lrc: 2/0,0 mode: --/CR res: [0x240000401:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094903 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.061056:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aadd0 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.061069:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aad8a lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aad83 expref: 5 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.061079:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aadd0 lrc: 4/1,0 mode: --/PR res: [0x240000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.061121:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aad8a lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1aad83 expref: 5 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.061137:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ed2c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665483648:0@lo:104: 00000100:00100000:2.0:1787739050.061184:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.061226:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074157c2780 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.061485:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665483648 00000100:00100000:3.0:1787739050.061490:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665483648 00000100:00100000:3.0:1787739050.061522:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1012 00000100:00100000:3.0:1787739050.061529:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ec780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665483648:12345-0@lo:104: 00010000:00010000:3.0:1787739050.061553:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1aad83 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aad8a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.061612:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739050.061618:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ec780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665483648:12345-0@lo:104: Request processed in 90us (434us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.061625:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1012 00010000:00010000:3.0:1787739050.061646:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739050.061652:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1aad83 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aad8a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.061661:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739050.061667:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1aad83 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xb1:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1aad8a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739050.061681:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1aad83 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xb1:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1aad8a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.061708:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1aad83 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xb1:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aad8a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.061715:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739050.061734:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1aad83 lrc: 1/0,0 mode: --/PW res: [0x240000401:0xb1:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aad8a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.063160:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1aad83 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xb1:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aad8a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.063185:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739050.063187:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739050.063190:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739050.066808:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ed2c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665483648:0@lo:104: 00010000:00010000:3.0:1787739050.066839:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aadd0 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.067370:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665483776 00000100:00100000:3.0:1787739050.067377:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665483776 00010000:00010000:3.0:1787739050.067440:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aad8a lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1aad83 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.067470:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1328 00000100:00100000:3.0:1787739050.067479:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba7050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+6:3651:x1874579665483776:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739050.067494:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00080000:3.0:1787739050.073202:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 324 00000004:00080000:3.0:1787739050.073277:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07302e89a80 with version 607 00000020:00080000:3.0:1787739050.073290:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 324 cancel update log [0x200000401:0x3:0x0].317: rc = 0 00000020:00080000:3.0:1787739050.076568:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 324 cancel update log [0x240000400:0x3:0x0].316: rc = 0 00000020:00080000:3.0:1787739050.076587:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 324 00000004:00080000:3.0:1787739050.076596:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 607 rpc_version 607 00000020:00100000:3.0:1787739050.076640:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739050.076651:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303f70000 x1874579665484032/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739050.076674:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f70000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665484032:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739050.076755:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.076796:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89780 going to sleep for 11 seconds 00010000:00010000:2.0:1787739050.077090:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1aad8a lrc: 0/0,0 mode: --/PW res: [0x240000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1aad83 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.077118:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb1:0x0].0x0 (ffffa074419d5b00) 00010000:00010000:2.0:1787739050.077123:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aadd0 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.077133:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aadd0 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.077141:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aadd0 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.077208:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739050.077263:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba7050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665483776:12345-0@lo:103:ldlm_bl.0 Request processed in 9784us (10105us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.077273:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1328 00000100:00100000:2.0:1787739050.077332:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665484032 00000100:00100000:2.0:1787739050.077339:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665484032 00000100:00100000:2.0:1787739050.077386:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2318 00000100:00100000:2.0:1787739050.077395:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304000b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665484032:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739050.077556:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304000b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665484032:12345-0@lo:1000:osp_up0-1.0 Request processed in 161us (812us total) trans 4294983274 rc 0/0 00000100:00100000:2.0:1787739050.077563:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2318 00010000:00010000:3.0:1787739050.077948:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aadd0 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.077958:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aadd0 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739050.078013:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xb1:0x0]: returning size 4096 00000004:00200000:3.0:1787739050.078018:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739050.078061:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1aadc9 lrc: 0/0,0 mode: --/CR res: [0x240000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094903 expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.078079:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aadd0 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094903 expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.078086:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb1:0x0].0x0 (ffffa074419d5b00) 00010000:00010000:3.0:1787739050.078089:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739050.078093:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c600, rc 0) 00000100:00100000:3.0:1787739050.078189:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17367:x1874579652393600:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 17235us (17451us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.078196:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23074 00000100:00100000:3.0:1787739050.078240:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f70000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665484032:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739050.083844:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739050.084018:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739050.084321:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652393984 00000100:00100000:3.0:1787739050.084330:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652393984 00000100:00100000:3.0:1787739050.084402:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23075 00000100:00100000:3.0:1787739050.084441:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17367:x1874579652393984:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:3.0:1787739050.084497:0:13793:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739050.084525:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1aadb4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14e9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948f5 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.084543:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14e9:0x0].0x0 (ffffa074331c2100) 00010000:00010000:3.0:1787739050.084549:0:13793:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739050.084597:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1aadd7 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.084605:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1aadd7 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.084628:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1aadd7 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.084649:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304000780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665484160:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787739050.084698:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.084837:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89780 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.085767:0:13793:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968263 gen 2 00000004:00080000:3.0:1787739050.085774:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968259 committed transno 4294968263 00000004:00080000:3.0:1787739050.085793:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968260 committed transno 4294968263 00000004:00080000:3.0:1787739050.085802:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968261 committed transno 4294968263 00000004:00080000:3.0:1787739050.085810:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968263 committed transno 4294968263 00000100:00100000:3.0:1787739050.085829:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304000780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665484160:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739050.085844:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1aadd7 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aadde expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.085854:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1aadd7 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aadde expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739050.085867:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739050.085905:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1aade5 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.085912:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1aade5 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.085919:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1aade5 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.085932:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304000780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665484288:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787739050.085962:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.086225:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89780 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.086381:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652393728 00000100:00100000:2.0:1787739050.086387:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652393728 00000100:00100000:2.0:1787739050.086445:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1121 00000100:00100000:2.0:1787739050.086453:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432ed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2409:x1874579652393728:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:2.0:1787739050.086466:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739050.086481:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1aada6 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14e9:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948ee expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.086494:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14e9:0x0].0x0 (ffffa07305eae900) 00010000:00010000:2.0:1787739050.086501:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739050.086540:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432ed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2409:x1874579652393728:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 88us (2706us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.086547:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1121 00000100:00100000:3.0:1787739050.089646:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304000780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665484288:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739050.089665:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1aade5 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aadec expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.089676:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1aade5 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aadec expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739050.089698:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:3.0:1787739050.089758:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305655e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665484416:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739050.089799:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.089832:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.090057:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665484416 00000100:00100000:0.0:1787739050.090063:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665484416 00000100:00100000:0.0:1787739050.090104:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2321 00000100:00100000:0.0:1787739050.090121:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305656d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665484416:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739050.090192:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305656d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665484416:12345-0@lo:1000:mdt00_004.0 Request processed in 72us (392us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.090199:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2321 00000100:00100000:3.0:1787739050.091290:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305655e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665484416:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739050.091357:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305655e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665484544:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739050.091399:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.091453:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89300 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.091472:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665484544 00000100:00100000:3.0:1787739050.091477:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665484544 00000100:00100000:3.0:1787739050.091511:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2322 00000100:00100000:3.0:1787739050.091517:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305654000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665484544:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739050.091598:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305654000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665484544:12345-0@lo:1000:mdt00_004.0 Request processed in 80us (198us total) trans 0 rc 1/1 00000100:00100000:3.0:1787739050.091605:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2322 00000100:00100000:3.0:1787739050.091631:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305655e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665484544:0@lo:1000:mdt00_004.0 00010000:00010000:3.0:1787739050.091680:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1aadf3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14e9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.091689:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1aadf3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14e9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.091699:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1aadf3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14e9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.091710:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305655e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665484672:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787739050.091733:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.091751:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89780 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.091763:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665484672 00000100:00100000:3.0:1787739050.091767:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665484672 00000100:00100000:3.0:1787739050.091784:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2323 00000100:00100000:3.0:1787739050.091789:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305654f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665484672:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739050.091798:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739050.091812:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1aadfa lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14e9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aadf3 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.091832:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1aadfa lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14e9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aadf3 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.091845:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1aadfa lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14e9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aadf3 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.091853:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14e9:0x0].0x0 (ffffa0730608b600) 00010000:00010000:3.0:1787739050.091856:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739050.091859:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abc00, rc 0) 00000100:00100000:3.0:1787739050.091889:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305654f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665484672:12345-0@lo:101:mdt00_004.0 Request processed in 98us (154us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.091895:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2323 00000100:00100000:3.0:1787739050.091919:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305655e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665484672:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739050.091931:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1aadf3 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14e9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aadfa expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.091941:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1aadf3 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14e9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aadfa expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.091963:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1aae01 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14e9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.091972:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1aae01 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14e9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.091981:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1aae01 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14e9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.092311:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1aae08 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.092325:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aadc2 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948fc expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.092334:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1aae08 lrc: 4/0,1 mode: --/EX res: [0x200000400:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.092356:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aadc2 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948fc expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.092369:0:13793:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aadc2 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948fc expref: 17 pid: 6523 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.092380:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305655e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665484800:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739050.092517:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89780 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.106116:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00010000:00010000:3.0:1787739050.113098:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aadc2 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948fc expref: 17 pid: 6523 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.113137:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305655e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665484800:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739050.113161:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1aae08 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.113171:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1aae08 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.113176:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1aae08 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.113221:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1aae0f lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.113229:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1aae0f lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.113241:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1aae0f lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.113261:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305655e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665484928:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787739050.113322:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.113364:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89780 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.117685:0:8447:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f78000 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8447:1874579665485056:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739050.117721:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aae16 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aae0f expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.117745:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb1:0x0].0x0 (ffffa074419d5b00) 00010000:00010000:3.0:1787739050.117749:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739050.117762:0:8447:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aae16 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aae0f expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.117773:0:8447:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aae16 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aae0f expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.117780:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1de00, rc 0) 00000100:00100000:3.0:1787739050.117854:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305656580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:13793:x1874579665484928:12345-0@lo:101:mdt00_004.0 Request processed in 4391us (4531us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.117863:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2324 00000100:00100000:0.0:1787739050.118159:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305655e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665484928:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739050.118179:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1aae0f lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aae16 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.118190:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1aae0f lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aae16 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.121574:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739050.122266:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652394240 00000100:00100000:0.0:1787739050.122277:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652394240 00010000:00010000:0.0:1787739050.122320:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aadd0 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094903 expref: 7 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739050.122365:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1330 00000100:00100000:0.0:1787739050.122375:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652394240:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739050.122390:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739050.122396:0:6510:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aadd0 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094903 expref: 7 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.122408:0:6510:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aadd0 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094903 expref: 7 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.122520:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aadd0 lrc: 0/0,0 mode: --/PR res: [0x240000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094903 expref: 7 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.122538:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb1:0x0].0x0 (ffffa074419d5b00) 00010000:00010000:0.0:1787739050.122544:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aae16 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aae0f expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.122554:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aae16 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1aae0f expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.122562:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aae16 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1aae0f expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.122592:0:6510:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aae16 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aae0f expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739050.122603:0:6510:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432d680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6510:1874579665485184:0@lo:105: 00000100:00100000:0.0:1787739050.122651:0:6510:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.122691:0:6510:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4900 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.122825:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665485184 00000100:00100000:2.0:1787739050.122830:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665485184 00000100:00100000:2.0:1787739050.122871:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1013 00000100:00100000:2.0:1787739050.123042:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6510:x1874579665485184:12345-0@lo:105: 00010000:00010000:2.0:1787739050.123053:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1aae0f lrc: 5/0,1 mode: --/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aae16 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.123063:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1aae0f lrc: 5/0,1 mode: --/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aae16 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.123072:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1aae0f lrc: 5/0,1 mode: --/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aae16 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.123080:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1aae0f lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1aae16 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.123087:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1aae0f lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1aae16 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.123095:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1aae0f lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1aae16 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.123131:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069aa000) 00010000:00010000:0.0:1787739050.123205:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1aae0f lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aae16 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.123215:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1aae0f lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aae16 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.123318:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6510:x1874579665485184:12345-0@lo:105: Request processed in 440us (665us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.123326:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1013 00000100:00100000:3.0:1787739050.123380:0:6510:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432d680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6510:1874579665485184:0@lo:105: 00010000:00010000:3.0:1787739050.123392:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739050.124401:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432c000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665485312:0@lo:403:mdt00_004.0 00000100:00100000:0.0:1787739050.127694:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.128058:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b43c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.135202:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2408:x1874579652394240:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 12814us (13623us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.135217:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1330 00000004:00080000:0.0:1787739050.137710:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 633 rpc_version 633 00000020:00100000:0.0:1787739050.137750:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739050.137994:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0730432c000 x1874579665485568/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2064/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739050.138023:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432c000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665485568:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739050.138097:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.138177:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.139189:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432c000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665485568:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787739050.139713:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787739050.139720:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1aae0f lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aae16 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.139728:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739050.139732:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1aae0f lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aae16 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739050.139740:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1aae0f lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aae16 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.139840:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1aae0f lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aae16 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.139848:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:3.0:1787739050.139896:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665485696:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739050.139940:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.139991:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665485696 00010000:00010000:0.0:1787739050.139994:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1aae0f lrc: 1/0,0 mode: --/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aae16 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.139996:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665485696 00010000:00010000:0.0:1787739050.140006:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1aae0f lrc: 0/0,0 mode: --/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aae16 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.140029:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739050.140033:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1aadf3 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x14e9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aadfa expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.140037:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1122 00010000:00010000:0.0:1787739050.140046:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa600 already unused, calling callback (ffffffffc18e6ee0) 00000100:00100000:3.0:1787739050.140048:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305654780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+10:3651:x1874579665485696:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739050.140052:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1aadf3 lrc: 5/0,0 mode: EX/EX res: [0x200000402:0x14e9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aadfa expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.140060:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000004:00010000:0.0:1787739050.140062:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1aadf3 lrc: 5/0,0 mode: EX/EX res: [0x200000402:0x14e9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aadfa expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.140080:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aae16 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aae0f expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.140086:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1aadf3 lrc: 4/0,0 mode: --/EX res: [0x200000402:0x14e9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aadfa expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.140096:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739050.140097:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb1:0x0].0x0 (ffffa074419d5b00) 00010000:00010000:3.0:1787739050.140118:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739050.140126:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1aadf3 lrc: 3/0,0 mode: --/EX res: [0x200000402:0x14e9:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aadfa expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.140138:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739050.140141:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739050.140144:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739050.140153:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305654780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+9:3651:x1874579665485696:12345-0@lo:103:ldlm_bl.0 Request processed in 106us (212us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.140153:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787739050.140162:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1122 00000100:00100000:0.0:1787739050.140172:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665485824:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739050.140193:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665485696:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739050.140208:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.140232:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665485824 00000100:00100000:3.0:1787739050.140236:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665485824 00000100:00100000:3.0:1787739050.140264:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1123 00000100:00100000:3.0:1787739050.140270:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb7840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+9:3653:x1874579665485824:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739050.140287:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739050.140299:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1aadfa lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14e9:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aadf3 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.140309:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14e9:0x0].0x0 (ffffa0730608b600) 00010000:00010000:3.0:1787739050.140316:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739050.140441:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739050.140447:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1aadd7 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aadde expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.140458:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739050.140463:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1aadd7 lrc: 5/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aadde expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739050.140471:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1aadd7 lrc: 5/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aadde expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.140490:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1aadd7 lrc: 4/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aadde expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.140497:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739050.140508:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1aadd7 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aadde expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.140516:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739050.140518:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739050.140520:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739050.140527:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787739050.140536:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665485952:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739050.140565:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.140655:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb7840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+8:3653:x1874579665485824:12345-0@lo:103:ldlm_bl.0 Request processed in 384us (446us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.140662:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1123 00000100:00100000:3.0:1787739050.140673:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665485952 00000100:00100000:3.0:1787739050.140677:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665485952 00000100:00100000:3.0:1787739050.140699:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1124 00000100:00100000:3.0:1787739050.140707:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb6940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+8:3652:x1874579665485952:12345-0@lo:103:ldlm_bl.0 00000004:00080000:1.0:1787739050.140707:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739050.140716:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1aade5 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aadec expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.140719:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739050.140727:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1aade5 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aadec expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.140735:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aadde lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aadd7 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.140748:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305eaef00) 00010000:00010000:3.0:1787739050.140756:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739050.140782:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb6940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+7:3652:x1874579665485952:12345-0@lo:103:ldlm_bl.0 Request processed in 77us (217us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.140788:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1124 00000100:00100000:1.0:1787739050.140857:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17367:x1874579652393984:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 56444us (56837us total) trans 4294983275 rc 0/0 00000100:00100000:1.0:1787739050.140865:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23075 00000100:00100000:0.0:1787739050.141037:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665485824:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739050.141057:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739050.141073:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665485952:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739050.181446:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739050.181479:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652394368 00000100:00100000:1.0:1787739050.181489:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652394368 00000100:00100000:1.0:1787739050.181548:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23076 00000100:00100000:1.0:1787739050.181564:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f78780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17368:x1874579652394368:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739050.181586:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739050.181605:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1aae1d lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09490a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.181661:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1aae24 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.181674:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1aae24 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.181684:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1aae24 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.181698:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1aae2b lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.181709:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1aae2b lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.181715:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1aae2b lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.181757:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1aae24 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.181765:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1aae24 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.181771:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1aae24 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.181778:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1aae24 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.181784:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739050.181788:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1aae24 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.181795:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1aae24 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.181807:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae500) 00010000:00010000:1.0:1787739050.181810:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739050.181814:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1aae24 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.181820:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1aae24 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.181832:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1aae2b lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.181838:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1aae2b lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.181843:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1aae2b lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.181847:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739050.181850:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1aae2b lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.181867:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1aae2b lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.181874:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x79cdf7e6 (ffffa07305eaeb00) 00010000:00010000:1.0:1787739050.181876:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739050.181879:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1aae2b lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.181884:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1aae2b lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.181919:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1aae1d lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09490a expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.181929:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae500) 00010000:00010000:1.0:1787739050.181931:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739050.181935:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1aae1d lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09490a expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.181944:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c600, rc 0) 00000100:00100000:1.0:1787739050.182008:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f78780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17368:x1874579652394368:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 446us (571us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739050.182015:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23076 00000100:00100000:3.0:1787739050.194478:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739050.194555:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652394496 00000100:00100000:2.0:1787739050.194566:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652394496 00000100:00100000:2.0:1787739050.194623:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23077 00000100:00100000:2.0:1787739050.194632:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305654b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17368:x1874579652394496:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739050.194651:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739050.194680:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aae32 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094911 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.194734:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1aae39 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.194748:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1aae39 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.194758:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1aae39 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739050.194809:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x5:0x0]: returning size 4096 00000004:00200000:2.0:1787739050.194815:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739050.194848:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aae32 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094911 expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.194865:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1aae39 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094911 expref: 7 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.194874:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074419d5400) 00010000:00010000:2.0:1787739050.194877:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739050.194880:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c000, rc 0) 00000100:00100000:2.0:1787739050.194941:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305654b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17368:x1874579652394496:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 310us (472us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.194948:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23077 00000100:00100000:1.0:1787739050.214167:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739050.214293:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652394752 00000100:00100000:2.0:1787739050.214303:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652394752 00000100:00100000:2.0:1787739050.214375:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23078 00000100:00100000:2.0:1787739050.214389:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17368:x1874579652394752:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:2.0:1787739050.214525:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156eda40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665486080:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739050.214578:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.214629:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.214648:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665486080 00000100:00100000:2.0:1787739050.214651:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665486080 00000100:00100000:2.0:1787739050.214676:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2327 00000100:00100000:2.0:1787739050.214682:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665486080:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739050.214765:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665486080:12345-0@lo:1000:mdt00_004.0 Request processed in 81us (187us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.214774:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2327 00000100:00100000:2.0:1787739050.214822:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156eda40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665486080:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739050.214878:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156eda40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665486208:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739050.214905:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.214927:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.214947:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665486208 00000100:00100000:2.0:1787739050.214951:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665486208 00000100:00100000:2.0:1787739050.214991:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2328 00000100:00100000:2.0:1787739050.214999:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665486208:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739050.215092:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665486208:12345-0@lo:1000:mdt00_004.0 Request processed in 92us (185us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739050.215099:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2328 00000100:00100000:2.0:1787739050.215154:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156eda40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665486208:0@lo:1000:mdt00_004.0 00010000:00010000:2.0:1787739050.215209:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1aae40 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.215220:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1aae40 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.215230:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1aae40 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.215242:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156eda40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665486336:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1787739050.215271:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.215290:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.215306:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665486336 00000100:00100000:2.0:1787739050.215308:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665486336 00000100:00100000:2.0:1787739050.215333:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2329 00000100:00100000:2.0:1787739050.215339:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ee940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665486336:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739050.215346:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739050.215358:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1aae47 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aae40 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.215376:0:8448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1aad52 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948e7 expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.215383:0:8448:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1aae47 lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aae40 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.215399:0:8448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1aad52 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948e7 expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.216029:0:8448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1aad52 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948e7 expref: 16 pid: 6524 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.216042:0:8448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8448:1874579665486464:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739050.216199:0:8448:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.221370:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739050.221486:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652394880 00000100:00100000:1.0:1787739050.221497:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652394880 00010000:00010000:1.0:1787739050.221538:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1aad52 lrc: 5/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948e7 expref: 17 pid: 6524 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739050.221576:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1331 00000100:00100000:1.0:1787739050.221588:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba4450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2408:x1874579652394880:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739050.221605:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739050.221612:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1aad52 lrc: 5/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948e7 expref: 17 pid: 6524 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.221627:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1aad52 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948e7 expref: 17 pid: 6524 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.221652:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae500) 00010000:00010000:1.0:1787739050.221659:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1aae47 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aae40 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.221670:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1aae47 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1aae40 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.221686:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1aae47 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1aae40 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.221720:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1aae47 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aae40 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739050.221749:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f78000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665486592:0@lo:105: 00000100:00100000:1.0:1787739050.221811:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739050.221848:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623db40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739050.221872:0:15907:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787739050.221895:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665486592 00000100:00100000:1.0:1787739050.221899:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665486592 00000100:00100000:1.0:1787739050.221933:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1014 00000100:00100000:1.0:1787739050.221940:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306b99050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665486592:12345-0@lo:105: 00010000:00010000:1.0:1787739050.221951:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1aae40 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aae47 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.221960:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1aae40 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aae47 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.221969:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1aae40 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aae47 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.221977:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1aae40 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1aae47 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.221984:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1aae40 lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1aae47 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.221993:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1aae40 lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1aae47 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.222007:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633da00) 00000100:00100000:1.0:1787739050.222042:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306b99050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665486592:12345-0@lo:105: Request processed in 102us (231us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.222048:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1014 00000100:00100000:1.0:1787739050.222132:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f78000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665486592:0@lo:105: 00010000:00010000:1.0:1787739050.222146:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739050.222181:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba4450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2408:x1874579652394880:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 595us (829us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.222188:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1331 00010000:00010000:2.0:1787739050.222576:0:8448:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1aad52 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0948e7 expref: 16 pid: 6524 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.222593:0:8448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8448:1874579665486464:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739050.222613:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1aae47 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aae40 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.222626:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae500) 00010000:00010000:2.0:1787739050.222629:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739050.222632:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633de00, rc 0) 00000100:00100000:2.0:1787739050.222678:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ee940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665486336:12345-0@lo:101:mdt00_004.0 Request processed in 7340us (7408us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.222685:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2329 00000100:00100000:3.0:1787739050.222748:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156eda40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665486336:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739050.222763:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1aae40 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aae47 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739050.222781:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739050.222811:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1aae4e lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.222818:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1aae4e lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.222827:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1aae4e lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.222840:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073056543c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665486720:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787739050.222865:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.222896:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.222978:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665486720 00000100:00100000:2.0:1787739050.222982:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665486720 00000100:00100000:2.0:1787739050.223015:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2330 00000100:00100000:2.0:1787739050.223021:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305657480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665486720:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739050.223033:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739050.223045:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aae55 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aae4e expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.223063:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aae55 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aae4e expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.223073:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aae55 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aae4e expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.223080:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x79cdf7e6 (ffffa074419d5d00) 00010000:00010000:2.0:1787739050.223082:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739050.223085:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633ce00, rc 0) 00000100:00100000:2.0:1787739050.223125:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305657480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665486720:12345-0@lo:101:mdt00_004.0 Request processed in 103us (258us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.223131:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2330 00000100:00100000:3.0:1787739050.223162:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073056543c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665486720:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739050.223173:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1aae4e lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aae55 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.223182:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1aae4e lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aae55 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739050.223192:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:3.0:1787739050.223235:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073056543c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665486848:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739050.223259:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.223285:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.223368:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665486848 00000100:00100000:2.0:1787739050.223372:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665486848 00000100:00100000:2.0:1787739050.223399:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2331 00000100:00100000:2.0:1787739050.223405:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305655a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665486848:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739050.223479:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305655a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665486848:12345-0@lo:1000:mdt00_004.0 Request processed in 74us (218us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.223486:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2331 00000100:00100000:1.0:1787739050.223534:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073056543c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665486848:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739050.223573:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f78000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665486976:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739050.223600:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739050.223623:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d780 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.223641:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665486976 00000100:00100000:2.0:1787739050.223646:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665486976 00000100:00100000:2.0:1787739050.223715:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2332 00000100:00100000:2.0:1787739050.223722:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7bc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665486976:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739050.223790:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7bc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665486976:12345-0@lo:1000:mdt00_004.0 Request processed in 68us (190us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739050.223798:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2332 00000100:00100000:1.0:1787739050.223827:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f78000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665486976:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739050.223909:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f78000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665487104:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739050.223937:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739050.223965:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d780 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.223996:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665487104 00000100:00100000:3.0:1787739050.224000:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665487104 00000100:00100000:3.0:1787739050.224030:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2333 00000100:00100000:3.0:1787739050.224037:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665487104:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739050.224101:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665487104:12345-0@lo:1000:mdt00_004.0 Request processed in 65us (165us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739050.224122:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2333 00000100:00100000:1.0:1787739050.224292:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f78000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665487104:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739050.224333:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f78000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665487232:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739050.224354:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739050.224386:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d780 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.224410:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665487232 00000100:00100000:2.0:1787739050.224438:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665487232 00000100:00100000:2.0:1787739050.224481:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2334 00000100:00100000:2.0:1787739050.224487:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665487232:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739050.224545:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665487232:12345-0@lo:1000:mdt00_004.0 Request processed in 48us (179us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.224553:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2334 00000100:00100000:2.0:1787739050.224593:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f78000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665487232:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739050.224636:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665487360:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739050.224664:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.224686:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665487360 00000100:00100000:2.0:1787739050.224688:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.224691:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665487360 00000100:00100000:3.0:1787739050.224716:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2335 00000100:00100000:3.0:1787739050.224722:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665487360:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739050.224777:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665487360:12345-0@lo:1000:mdt00_004.0 Request processed in 55us (112us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.224783:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2335 00000100:00100000:2.0:1787739050.225355:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665487360:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739050.225546:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665487488:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739050.225573:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.225589:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739050.225668:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665487488 00000100:00100000:1.0:1787739050.225673:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665487488 00000100:00100000:1.0:1787739050.225700:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2336 00000100:00100000:1.0:1787739050.225705:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156efc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665487488:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739050.225860:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156efc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665487488:12345-0@lo:1000:mdt00_004.0 Request processed in 154us (285us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.225868:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2336 00000100:00100000:3.0:1787739050.226005:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665487488:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739050.226043:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073056552c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665487616:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739050.226071:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.226163:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.226166:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665487616 00000100:00100000:2.0:1787739050.226170:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665487616 00000100:00100000:2.0:1787739050.226197:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2337 00000100:00100000:2.0:1787739050.226204:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305656940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665487616:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739050.226264:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305656940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665487616:12345-0@lo:1000:mdt00_004.0 Request processed in 61us (192us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.226270:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2337 00000100:00100000:3.0:1787739050.226289:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073056552c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665487616:0@lo:1000:mdt00_004.0 00000040:00080000:2.0:1787739050.227149:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].318, 2168 off486696 00000040:00080000:2.0:1787739050.227227:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].319, 2368 off532880 00000004:00080000:2.0:1787739050.227234:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07302e89300 with version 608 00000004:00080000:3.0:1787739050.227275:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 608 rpc_version 608 00000020:00100000:3.0:1787739050.227305:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739050.227313:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073056552c0 x1874579665487872/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739050.227328:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073056552c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665487872:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739050.227370:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.227397:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89780 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.227408:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665487872 00000100:00100000:2.0:1787739050.227476:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665487872 00000100:00100000:2.0:1787739050.227549:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2338 00000100:00100000:2.0:1787739050.227557:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305657840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:7467:x1874579665487872:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739050.228181:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305657840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:7467:x1874579665487872:12345-0@lo:1000:osp_up0-1.0 Request processed in 623us (808us total) trans 4294983276 rc 0/0 00000100:00100000:2.0:1787739050.228190:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2338 00000100:00100000:3.0:1787739050.228246:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073056552c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665487872:0@lo:1000:osp_up0-1.0 00010000:00010000:2.0:1787739050.228346:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aae5c lrc: 3/0,1 mode: --/PW res: [0x240000402:0x155:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.228363:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aae5c lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x155:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.228373:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aae5c lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x155:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.228396:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aae63 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.228403:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aae63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.228409:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aae63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.228533:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1aae6a lrc: 3/0,1 mode: --/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.228540:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1aae6a lrc: 3/0,1 mode: --/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.228549:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1aae6a lrc: 3/0,1 mode: --/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.228563:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665488000:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1787739050.228591:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.228615:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ecc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.228844:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665488000 00000100:00100000:3.0:1787739050.228850:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665488000 00000100:00100000:3.0:1787739050.228892:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2339 00000100:00100000:3.0:1787739050.228899:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156eda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:13793:x1874579665488000:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739050.228916:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739050.228931:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aae71 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aae6a expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.228952:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aae71 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aae6a expref: 13 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.228971:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aae71 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aae6a expref: 13 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.228980:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xaf:0x0].0x0 (ffffa0730608bb00) 00010000:00010000:3.0:1787739050.228982:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739050.228985:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab800, rc 0) 00000100:00100000:3.0:1787739050.229024:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156eda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+13:13793:x1874579665488000:12345-0@lo:101:mdt00_004.0 Request processed in 126us (433us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.229031:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2339 00000100:00100000:2.0:1787739050.229045:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665488000:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739050.229061:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1aae6a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aae71 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.229082:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1aae6a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aae71 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739050.229119:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000443f75c0 reply state 0000000083813109 transno 4294968265 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aae63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.229133:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aae63 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.229139:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xaf:0x0].0x0 (ffffa074419d5100) 00010000:00010000:2.0:1787739050.229142:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739050.229145:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787739050.229151:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1aae6a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aae71 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.229159:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1aae6a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aae71 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739050.229168:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xaf:0x0] 00000004:00010000:2.0:1787739050.229174:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000443f75c0 reply state 0000000083813109 transno 4294968265 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aae5c lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x155:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.229180:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aae5c lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x155:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.229184:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x155:0x0].0x0 (ffffa074419d5b00) 00010000:00010000:2.0:1787739050.229185:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739050.229187:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:2.0:1787739050.229190:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x155:0x0]: returning size 4096 00000004:00080000:2.0:1787739050.229195:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739050.229201:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1aae40 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aae47 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.229206:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1aae40 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aae47 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.229214:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1aae40 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aae47 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.229233:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739050.229238:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739050.229242:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1aae4e lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aae55 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.229249:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1aae4e lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aae55 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.229269:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739050.229277:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1aae40 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aae47 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.229286:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633da00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739050.229291:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1aae40 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aae47 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739050.229304:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1aae40 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aae47 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.229330:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1aae40 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aae47 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.229337:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1787739050.229359:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17368:x1874579652394752:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 14973us (15200us total) trans 4294968265 rc 0/0 00010000:00010000:3.0:1787739050.229362:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1aae40 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aae47 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.229373:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23078 00010000:00010000:3.0:1787739050.229375:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1aae40 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aae47 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.229395:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739050.229397:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739050.229400:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739050.229551:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305655680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665488128:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739050.229589:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.229699:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665488128 00000100:00100000:2.0:1787739050.229704:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665488128 00000100:00100000:2.0:1787739050.229741:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1125 00000100:00100000:2.0:1787739050.229748:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073056543c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+13:3650:x1874579665488128:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739050.229758:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739050.229772:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1aae47 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aae40 expref: 13 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.229782:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae500) 00010000:00010000:2.0:1787739050.229789:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739050.229829:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073056543c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+12:3650:x1874579665488128:12345-0@lo:103:ldlm_bl.0 Request processed in 81us (239us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.229838:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1125 00000100:00100000:1.0:1787739050.229902:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305655680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665488128:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739050.242515:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739050.242615:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652395136 00000100:00100000:1.0:1787739050.242626:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652395136 00000100:00100000:1.0:1787739050.242693:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23079 00000100:00100000:1.0:1787739050.242715:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305654780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17369:x1874579652395136:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:1.0:1787739050.242875:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305654780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17369:x1874579652395136:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 162us (368us total) trans 0 rc -17/-17 00000100:00100000:1.0:1787739050.242882:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23079 00000100:00100000:1.0:1787739050.268327:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739050.268578:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652395264 00000100:00100000:2.0:1787739050.268586:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652395264 00000100:00100000:2.0:1787739050.268697:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23080 00000100:00100000:2.0:1787739050.268705:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17370:x1874579652395264:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739050.268722:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739050.268741:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1aae78 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094918 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.268797:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aae7f lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.268807:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aae7f lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.268815:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aae7f lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.268821:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aae86 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.268828:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aae55 lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aae4e expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.268834:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aae86 lrc: 4/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.268857:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aae55 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1aae4e expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.268871:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665488256:0@lo:104: 00000100:00100000:2.0:1787739050.268906:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.268979:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.269050:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665488256 00000100:00100000:3.0:1787739050.269054:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665488256 00000100:00100000:3.0:1787739050.269119:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1015 00000100:00100000:3.0:1787739050.269128:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665488256:12345-0@lo:104: 00010000:00010000:3.0:1787739050.269139:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1aae4e lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aae55 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.269236:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739050.269242:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665488256:12345-0@lo:104: Request processed in 117us (334us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.269249:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1015 00010000:00010000:0.0:1787739050.269306:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739050.269312:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1aae4e lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aae55 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.269325:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739050.269330:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1aae4e lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1aae55 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739050.269339:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1aae4e lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1aae55 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.269365:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1aae4e lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aae55 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.269371:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739050.269401:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1aae4e lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aae55 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.269410:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1aae4e lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aae55 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.269443:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739050.269445:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739050.269446:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739050.269488:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665488384:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739050.269518:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.269586:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665488384 00000100:00100000:0.0:1787739050.269591:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665488384 00010000:00010000:0.0:1787739050.269620:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aae55 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1aae4e expref: 12 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739050.269648:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1332 00000100:00100000:0.0:1787739050.269656:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305655e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+12:3651:x1874579665488384:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739050.269668:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1787739050.269906:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665488256:0@lo:104: 00010000:00010000:2.0:1787739050.269932:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aae86 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.271899:0:6502:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1aade5 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aadec expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739050.271918:0:6502:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1aade5 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aadec expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.271960:0:6502:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1aade5 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aadec expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.271968:0:6502:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739050.272005:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1aade5 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aadec expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787739050.272022:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983276 is committed 00000001:00080000:3.0:1787739050.272079:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983275 is committed 00000020:00080000:3.0:1787739050.272092:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa0730688d600 refcount 2 committed 0 result 0 batchid 334 00000020:00080000:3.0:1787739050.272096:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2f00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730688d540 00000020:00080000:3.0:1787739050.272100:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].330 00000020:00080000:3.0:1787739050.272102:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2700 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306aada00 00000100:00100000:0.0:1787739050.272160:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305655680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665488512:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:0.0:1787739050.272225:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000020:00080000:3.0:1787739050.272238:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].331 00000020:00080000:3.0:1787739050.272270:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 334 committed batchid 333 00000100:00100000:0.0:1787739050.272288:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665488512 00000100:00100000:0.0:1787739050.272295:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665488512 00000020:00080000:3.0:1787739050.272325:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 333 00010000:00010000:1.0:1787739050.272327:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aae55 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1aae4e expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739050.272339:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1126 00000100:00100000:0.0:1787739050.272350:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb6580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+7:3652:x1874579665488512:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:1.0:1787739050.272351:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x79cdf7e6 (ffffa074419d5d00) 00010000:00010000:1.0:1787739050.272360:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aae86 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.272365:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739050.272371:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aae86 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.272380:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aae86 lrc: 4/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.272385:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1aadec lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x7423dc1f bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aade5 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.272400:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x7423dc1f (ffffa07305eaea00) 00010000:00010000:0.0:1787739050.272408:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739050.272442:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739050.272494:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305655e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+6:3651:x1874579665488384:12345-0@lo:103:ldlm_bl.0 Request processed in 2837us (2974us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.272498:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb6580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+6:3652:x1874579665488512:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 148us (271us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.272508:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1332 00010000:00010000:2.0:1787739050.272689:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aad75 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x14e9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.272699:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aad75 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x14e9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.272706:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aad75 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x14e9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.272715:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739050.272719:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aad75 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x14e9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.272726:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aad75 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x14e9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.272735:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14e9:0x0].0x0 (ffffa074331c2100) 00010000:00010000:2.0:1787739050.272738:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739050.272742:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aad75 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x14e9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.272749:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aad75 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x14e9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.272763:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aad7c lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.272769:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aad7c lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.272775:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aad7c lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.272780:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aaa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739050.272783:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aad7c lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.272789:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aad7c lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.272797:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb1:0x0].0x0 (ffffa074331c2500) 00010000:00010000:2.0:1787739050.272800:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739050.272803:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aad7c lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.272808:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aad7c lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.272871:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665488384:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739050.272897:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aae86 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.272904:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aae86 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.272990:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304000780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665488640:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739050.273024:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.273048:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688db40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739050.273064:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665488640 00000100:00100000:1.0:1787739050.273070:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665488640 00000100:00100000:1.0:1787739050.273117:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2340 00000100:00100000:1.0:1787739050.273133:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41d680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665488640:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739050.273207:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41d680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665488640:12345-0@lo:1000:mdt00_003.0 Request processed in 82us (181us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.273214:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2340 00000100:00100000:3.0:1787739050.273234:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304000780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665488640:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739050.273247:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1126 00010000:00010000:3.0:1787739050.273279:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1aae8d lrc: 3/1,0 mode: --/PR res: [0x240000402:0x155:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739050.273286:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305655680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665488512:0@lo:103:jbd2/dm-0-8.0 00010000:00010000:3.0:1787739050.273294:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1aae8d lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x155:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.273305:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1aae8d lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x155:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739050.273326:0:7944:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1aae8d lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x155:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.273338:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aae7f lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.273345:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aae7f lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.273561:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aae7f lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.273568:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aae7f lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.273574:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739050.273578:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aae7f lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.273584:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aae7f lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.273594:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5500) 00010000:00010000:3.0:1787739050.273597:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739050.273600:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aae7f lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.273607:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aae7f lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.273897:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aae86 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.273903:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aae86 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.273909:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aae86 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.273914:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739050.273917:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aae86 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.273924:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aae86 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.273936:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x79cdf7e6 (ffffa074419d5d00) 00010000:00010000:3.0:1787739050.273938:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739050.273941:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aae86 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.273947:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aae86 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.273981:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1aae78 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094918 expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.273997:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1aae8d lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x155:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094918 expref: 14 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.274004:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x155:0x0].0x0 (ffffa0730608b500) 00010000:00010000:3.0:1787739050.274006:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739050.274008:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa800, rc 0) 00000100:00100000:3.0:1787739050.274059:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17370:x1874579652395264:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 5354us (5739us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.274146:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23080 00000100:00100000:3.0:1787739050.280060:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739050.280153:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652395392 00000100:00100000:2.0:1787739050.280162:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652395392 00000100:00100000:2.0:1787739050.280225:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23081 00000100:00100000:2.0:1787739050.280236:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17370:x1874579652395392:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739050.280261:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739050.280280:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1aae94 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x155:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09491f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.280342:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1aae9b lrc: 3/1,0 mode: --/PR res: [0x240000402:0x155:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.280357:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1aae9b lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x155:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.280368:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1aae9b lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x155:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739050.280495:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x155:0x0]: returning size 4096 00000004:00200000:2.0:1787739050.280500:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739050.280536:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1aae94 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x155:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09491f expref: 11 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.280552:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1aae9b lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x155:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09491f expref: 10 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.280562:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x155:0x0].0x0 (ffffa074419d5b00) 00010000:00010000:2.0:1787739050.280565:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739050.280567:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d800, rc 0) 00000100:00100000:2.0:1787739050.280622:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17370:x1874579652395392:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 388us (573us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.280629:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23081 00000100:00100000:1.0:1787739050.287000:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739050.287173:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652395520 00000100:00100000:2.0:1787739050.287182:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652395520 00000100:00100000:2.0:1787739050.287322:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23082 00000100:00100000:2.0:1787739050.287333:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17370:x1874579652395520:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739050.287358:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739050.287380:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1aaea2 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094926 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.287470:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aaea9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.287483:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aaea9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.287492:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aaea9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739050.287536:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xaf:0x0]: returning size 4096 00000004:00200000:2.0:1787739050.287550:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739050.287582:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1aaea2 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094926 expref: 12 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.287598:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aaea9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094926 expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.287606:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xaf:0x0].0x0 (ffffa074419d5100) 00010000:00010000:2.0:1787739050.287609:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739050.287612:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d600, rc 0) 00000100:00100000:2.0:1787739050.287669:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17370:x1874579652395520:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 338us (680us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.287676:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23082 00000100:00100000:3.0:1787739050.291185:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739050.291279:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652395648 00000100:00100000:2.0:1787739050.291290:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652395648 00000100:00100000:2.0:1787739050.291388:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23083 00000100:00100000:2.0:1787739050.291398:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17370:x1874579652395648:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739050.291445:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739050.291475:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1aaeb0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09492d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.291529:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1aaeb7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.291554:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aae71 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aae6a expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.291563:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1aaeb7 lrc: 4/1,0 mode: --/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.291594:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aae71 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1aae6a expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.291609:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665488768:0@lo:104: 00000100:00100000:2.0:1787739050.291658:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.291688:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739050.291937:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665488768 00000100:00100000:1.0:1787739050.291942:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665488768 00000100:00100000:1.0:1787739050.292020:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1016 00000100:00100000:1.0:1787739050.292028:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ec780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665488768:12345-0@lo:104: 00010000:00010000:1.0:1787739050.292041:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1aae6a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aae71 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.292104:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787739050.292119:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ec780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665488768:12345-0@lo:104: Request processed in 93us (461us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.292126:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1016 00010000:00010000:0.0:1787739050.292188:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739050.292196:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1aae6a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aae71 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.292204:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739050.292209:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1aae6a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1aae71 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739050.292219:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1aae6a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1aae71 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.292247:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1aae6a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aae71 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.292254:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739050.292274:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1aae6a lrc: 1/0,0 mode: --/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aae71 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.292282:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1aae6a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aae71 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.292301:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739050.292303:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739050.292305:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739050.292315:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665488896:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739050.292346:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.292377:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665488896 00000100:00100000:0.0:1787739050.292381:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665488896 00010000:00010000:0.0:1787739050.292399:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aae71 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1aae6a expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739050.292447:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1333 00000100:00100000:0.0:1787739050.292466:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb7840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+6:3653:x1874579665488896:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739050.292476:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1787739050.293099:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665488768:0@lo:104: 00010000:00010000:2.0:1787739050.293132:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1aaeb7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1787739050.294799:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 334 batchid updated 00000020:00080000:0.0:1787739050.294873:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 334 00010000:00010000:2.0:1787739050.294941:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1aae71 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1aae6a expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.294957:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xaf:0x0].0x0 (ffffa0730608bb00) 00010000:00010000:2.0:1787739050.294963:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1aaeb7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.294971:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1aaeb7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.294978:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1aaeb7 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739050.294989:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa073068b4900 with version 634 00000020:00080000:0.0:1787739050.295005:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 334 cancel update log [0x240000401:0x3:0x0].330: rc = 0 00010000:00010000:2.0:1787739050.295019:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000020:00080000:0.0:1787739050.295071:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 334 cancel update log [0x200000400:0x3:0x0].331: rc = 0 00000020:00080000:0.0:1787739050.295078:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 334 00000100:00100000:2.0:1787739050.295081:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb7840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+5:3653:x1874579665488896:12345-0@lo:103:ldlm_bl.0 Request processed in 2626us (2734us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.295089:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1333 00010000:00010000:3.0:1787739050.295133:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1aaeb7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.295140:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1aaeb7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.295149:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665488896:0@lo:103:ldlm_bl.0 00000004:00200000:3.0:1787739050.295194:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xaf:0x0]: returning size 4096 00000004:00200000:3.0:1787739050.295199:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739050.295240:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1aaeb0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09492d expref: 16 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.295257:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1aaeb7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09492d expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.295264:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xaf:0x0].0x0 (ffffa0730608bb00) 00010000:00010000:3.0:1787739050.295267:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739050.295269:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d400, rc 0) 00000100:00100000:3.0:1787739050.295311:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17370:x1874579652395648:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 3916us (4133us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.295315:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23083 00000004:00080000:1.0:1787739050.295721:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 634 rpc_version 634 00000020:00100000:1.0:1787739050.295745:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739050.295751:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303f721c0 x1874579665489152/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739050.295776:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f721c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665489152:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739050.295824:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739050.295855:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.295980:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665489152 00000100:00100000:2.0:1787739050.295986:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665489152 00000100:00100000:2.0:1787739050.296021:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2341 00000100:00100000:2.0:1787739050.296029:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f703c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665489152:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739050.296164:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f703c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665489152:12345-0@lo:1000:osp_up1-0.0 Request processed in 135us (339us total) trans 4294968266 rc 0/0 00000100:00100000:2.0:1787739050.296171:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2341 00000100:00100000:1.0:1787739050.296249:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f721c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665489152:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739050.300839:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739050.300975:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739050.300991:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652395776 00000100:00100000:0.0:1787739050.301003:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652395776 00000100:00100000:0.0:1787739050.301100:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1127 00000100:00100000:0.0:1787739050.301120:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41de00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2409:x1874579652395776:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:0.0:1787739050.301139:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787739050.301147:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652396032 00000100:00100000:1.0:1787739050.301150:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652396032 00010000:00010000:0.0:1787739050.301162:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1aae8d lrc: 0/0,0 mode: --/PR res: [0x240000402:0x155:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094918 expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.301180:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x155:0x0].0x0 (ffffa0730608b500) 00010000:00010000:0.0:1787739050.301191:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739050.301213:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23084 00000100:00100000:1.0:1787739050.301220:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17370:x1874579652396032:12345-192.168.203.15@tcp:36:rmdir.0 00000100:00100000:0.0:1787739050.301237:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41de00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652395776:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 130us (406us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.301243:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1127 00010000:00010000:1.0:1787739050.301263:0:6524:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739050.301276:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1aae9b lrc: 0/0,0 mode: --/PR res: [0x240000402:0x155:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09491f expref: 11 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.301294:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x155:0x0].0x0 (ffffa074419d5b00) 00010000:00010000:1.0:1787739050.301297:0:6524:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739050.301355:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1aaebe lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.301363:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1aaebe lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.301373:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1aaebe lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739050.301393:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561a1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665489280:0@lo:101:mdt00_001.0 00000100:00100000:1.0:1787739050.301455:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739050.301483:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.302374:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665489280 00000100:00100000:2.0:1787739050.302381:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665489280 00000100:00100000:2.0:1787739050.302435:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2342 00000100:00100000:2.0:1787739050.302442:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+5:6524:x1874579665489280:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739050.302454:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739050.302469:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1aaec5 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aaebe expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.302498:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1aaec5 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaebe expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.302510:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1aaec5 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaebe expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.302518:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403eebb00) 00010000:00010000:2.0:1787739050.302521:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739050.302524:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c200, rc 0) 00000100:00100000:2.0:1787739050.302619:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:6524:x1874579665489280:12345-0@lo:101:mdt00_001.0 Request processed in 177us (1163us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.302625:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2342 00000100:00100000:2.0:1787739050.302657:0:6524:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983276 gen 2 00000004:00080000:2.0:1787739050.302662:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983270 committed transno 4294983276 00000020:00080000:2.0:1787739050.302667:0:6524:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07410b4d180 refcount 2 committed 0 result 0 batchid 325 00000020:00080000:2.0:1787739050.302670:0:6524:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c695000 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07410b4d840 00000020:00080000:2.0:1787739050.302673:0:6524:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].318 00000020:00080000:2.0:1787739050.302675:0:6524:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c695f00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa074080f5800 00000020:00080000:2.0:1787739050.302678:0:6524:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].317 00000004:00080000:2.0:1787739050.302698:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983272 committed transno 4294983276 00000004:00080000:2.0:1787739050.302706:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983273 committed transno 4294983276 00000004:00080000:2.0:1787739050.302714:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983274 committed transno 4294983276 00000004:00080000:2.0:1787739050.302723:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983276 committed transno 4294983276 00000100:00100000:2.0:1787739050.302739:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561a1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665489280:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739050.302753:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1aaebe lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aaec5 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.302763:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1aaebe lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aaec5 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739050.302778:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739050.302798:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1aaecc lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.302805:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1aaecc lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.302812:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1aaecc lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.302825:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665489408:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1787739050.302848:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.303034:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000020:00080000:2.0:1787739050.303046:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 325 committed batchid 324 00000020:00080000:2.0:1787739050.303097:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 324 00000100:00100000:2.0:1787739050.303117:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665489408 00000100:00100000:2.0:1787739050.303120:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665489408 00000100:00100000:2.0:1787739050.303145:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2343 00000100:00100000:2.0:1787739050.303150:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ecf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:6524:x1874579665489408:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739050.303158:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739050.303169:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aaed3 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aaecc expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.303183:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aaed3 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaecc expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.303198:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aaed3 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaecc expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.303206:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x79cdf7e6 (ffffa07403eebe00) 00010000:00010000:2.0:1787739050.303209:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739050.303211:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c400, rc 0) 00000100:00100000:2.0:1787739050.303235:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ecf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665489408:12345-0@lo:101:mdt00_001.0 Request processed in 85us (387us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.303240:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2343 00000100:00100000:2.0:1787739050.303261:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665489408:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739050.303271:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1aaecc lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aaed3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.303279:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1aaecc lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aaed3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739050.303291:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:2.0:1787739050.303316:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665489536:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739050.303338:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.303352:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e780 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.303365:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665489536 00000100:00100000:2.0:1787739050.303367:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665489536 00000100:00100000:2.0:1787739050.303384:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2344 00000100:00100000:2.0:1787739050.303390:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665489536:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739050.303475:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665489536:12345-0@lo:1000:mdt00_001.0 Request processed in 85us (135us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.303482:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2344 00000100:00100000:2.0:1787739050.303513:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665489536:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739050.303713:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665489664:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739050.303740:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.305140:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e780 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.306065:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665489664 00000100:00100000:3.0:1787739050.306074:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665489664 00000100:00100000:3.0:1787739050.306171:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2345 00000100:00100000:3.0:1787739050.306181:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f70000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665489664:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739050.306303:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f70000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665489664:12345-0@lo:1000:mdt00_001.0 Request processed in 124us (2561us total) trans 0 rc 1/1 00000100:00100000:3.0:1787739050.306310:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2345 00000100:00100000:0.0:1787739050.306372:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665489664:0@lo:1000:mdt00_001.0 00010000:00010000:0.0:1787739050.306675:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1aaeda lrc: 3/0,1 mode: --/EX res: [0x240000402:0x155:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.306686:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1aaeda lrc: 3/0,1 mode: --/EX res: [0x240000402:0x155:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.306697:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1aaeda lrc: 3/0,1 mode: --/EX res: [0x240000402:0x155:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739050.306715:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb6940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665489792:0@lo:101:mdt00_001.0 00000100:00100000:0.0:1787739050.306764:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.306787:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.306828:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665489792 00000100:00100000:2.0:1787739050.306833:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665489792 00000100:00100000:2.0:1787739050.307036:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2346 00000100:00100000:2.0:1787739050.307044:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665489792:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739050.307057:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739050.307074:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aaee1 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x155:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aaeda expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.307097:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aaee1 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x155:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaeda expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.307120:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aaee1 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x155:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaeda expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.307129:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x155:0x0].0x0 (ffffa07403eeb300) 00010000:00010000:2.0:1787739050.307132:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739050.307136:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633dc00, rc 0) 00000100:00100000:2.0:1787739050.307171:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6524:x1874579665489792:12345-0@lo:101:mdt00_001.0 Request processed in 128us (418us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.307177:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2346 00000100:00100000:0.0:1787739050.307315:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb6940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665489792:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739050.307329:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1aaeda lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x155:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aaee1 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.307339:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1aaeda lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x155:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aaee1 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.307364:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1aaee8 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x155:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.307371:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1aaee8 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x155:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.307379:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1aaee8 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x155:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.307395:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1aaeef lrc: 3/0,1 mode: --/EX res: [0x240000400:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.307406:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aaea9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094926 expref: 10 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.307651:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1aaeef lrc: 4/0,1 mode: --/EX res: [0x240000400:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.307677:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aaea9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094926 expref: 10 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.307691:0:6524:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aaea9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094926 expref: 10 pid: 6525 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739050.307702:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb6940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665489920:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739050.307771:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d600 going to sleep for 11 seconds 00010000:00010000:0.0:1787739050.307814:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739050.311183:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739050.311233:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652396160 00000100:00100000:1.0:1787739050.311243:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652396160 00010000:00010000:1.0:1787739050.311282:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aaea9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094926 expref: 11 pid: 6525 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739050.311282:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb6940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665489920:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739050.311308:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1aaeef lrc: 3/0,1 mode: --/EX res: [0x240000400:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739050.311320:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1334 00000100:00100000:1.0:1787739050.311386:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41e580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2408:x1874579652396160:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739050.311400:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739050.311408:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aaea9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094926 expref: 11 pid: 6525 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.311449:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aaea9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094926 expref: 11 pid: 6525 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.311469:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aaea9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094926 expref: 11 pid: 6525 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.311484:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xaf:0x0].0x0 (ffffa074419d5100) 00010000:00010000:1.0:1787739050.311488:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1aaeef lrc: 3/0,1 mode: --/EX res: [0x240000400:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.311497:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1aaeef lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.311503:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1aaeef lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.311523:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739050.311539:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1aaeef lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.311554:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1aaeef lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739050.311577:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41e580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652396160:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 195us (403us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.311587:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1334 00010000:00010000:0.0:1787739050.311592:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1aaef6 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.311599:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1aaef6 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.311609:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1aaef6 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739050.311626:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb6940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665490048:0@lo:101:mdt00_001.0 00000100:00100000:0.0:1787739050.311665:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.311689:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.311696:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665490048 00000100:00100000:2.0:1787739050.311701:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665490048 00000100:00100000:2.0:1787739050.311737:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2347 00000100:00100000:2.0:1787739050.311744:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb70c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6524:x1874579665490048:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739050.311757:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739050.311771:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aaefd lrc: 2/0,0 mode: --/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aaef6 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.311790:0:18535:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1aaeb7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09492d expref: 13 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.311799:0:18535:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aaefd lrc: 4/0,0 mode: --/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaef6 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.311821:0:18535:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1aaeb7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09492d expref: 13 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.311834:0:18535:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1aaeb7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09492d expref: 13 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.311844:0:18535:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f72d00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:18535:1874579665490176:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739050.312148:0:18535:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.314661:0:18535:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f72d00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:18535:1874579665490176:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739050.314690:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aaefd lrc: 3/0,0 mode: --/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaef6 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.314700:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xaf:0x0].0x0 (ffffa0730608bb00) 00010000:00010000:2.0:1787739050.314702:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739050.314705:0:18535:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aaefd lrc: 3/0,0 mode: --/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaef6 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.314712:0:18535:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aaefd lrc: 3/0,0 mode: --/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaef6 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.314716:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633ce00, rc 0) 00000100:00100000:2.0:1787739050.314767:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb70c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665490048:12345-0@lo:101:mdt00_001.0 Request processed in 3023us (3100us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.314774:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2347 00000100:00100000:0.0:1787739050.315179:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb6940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665490048:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739050.315197:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1aaef6 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aaefd expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.315207:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1aaef6 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aaefd expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.317191:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739050.318264:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652396288 00000100:00100000:2.0:1787739050.318277:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652396288 00010000:00010000:2.0:1787739050.318317:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1aaeb7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09492d expref: 14 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.318367:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1335 00000100:00100000:2.0:1787739050.318376:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba5050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652396288:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739050.318391:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739050.318398:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1aaeb7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09492d expref: 14 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.318447:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1aaeb7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09492d expref: 14 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.318470:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1aaeb7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xaf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09492d expref: 14 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.318486:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xaf:0x0].0x0 (ffffa0730608bb00) 00010000:00010000:2.0:1787739050.318493:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aaefd lrc: 2/0,0 mode: --/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaef6 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.318504:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aaefd lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1aaef6 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.318513:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aaefd lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1aaef6 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.318543:0:6508:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aaefd lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaef6 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.318556:0:6508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f72d00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665490304:0@lo:105: 00000100:00100000:2.0:1787739050.318605:0:6508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.318645:0:6508:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e000 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.318688:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665490304 00000100:00100000:3.0:1787739050.318693:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665490304 00000100:00100000:3.0:1787739050.318734:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1017 00000100:00100000:3.0:1787739050.318742:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f78000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665490304:12345-0@lo:105: 00010000:00010000:3.0:1787739050.318755:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1aaef6 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aaefd expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.318774:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1aaef6 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aaefd expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.318783:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1aaef6 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aaefd expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.318791:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1aaef6 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1aaefd expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.318798:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1aaef6 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1aaefd expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.318806:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1aaef6 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1aaefd expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.318831:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609a800) 00000100:00100000:3.0:1787739050.318879:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f78000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665490304:12345-0@lo:105: Request processed in 137us (273us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.318886:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1017 00000100:00100000:2.0:1787739050.319368:0:6508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f72d00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665490304:0@lo:105: 00010000:00010000:2.0:1787739050.319387:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739050.319556:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba5050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:2409:x1874579652396288:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 1178us (2369us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.319566:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1335 00010000:00010000:2.0:1787739050.319630:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1aaef6 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aaefd expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.319641:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1aaef6 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aaefd expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.320336:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f72d00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665490432:0@lo:403:mdt00_001.0 00000100:00100000:2.0:1787739050.320393:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.320438:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ef00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.320459:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665490432 00000100:00100000:2.0:1787739050.320465:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665490432 00000100:00100000:2.0:1787739050.320515:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2348 00000100:00100000:2.0:1787739050.320524:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c382580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665490432:12345-0@lo:403:mdt00_001.0 00000100:00100000:2.0:1787739050.322102:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c382580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665490432:12345-0@lo:403:mdt00_001.0 Request processed in 1574us (1703us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.322162:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f72d00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665490432:0@lo:403:mdt00_001.0 00000100:00100000:2.0:1787739050.323228:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2348 00000040:00080000:1.0:1787739050.325731:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].319, 664 off487360 00000040:00080000:1.0:1787739050.325920:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].320, 744 off533624 00000004:00080000:1.0:1787739050.325927:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730623d000 with version 609 00000004:00080000:3.0:1787739050.326678:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 609 rpc_version 609 00000020:00100000:3.0:1787739050.326716:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739050.326728:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743c41da40 x1874579665490688/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1808/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739050.326764:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41da40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665490688:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739050.326831:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.326898:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739050.327133:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665490688 00000100:00100000:1.0:1787739050.327141:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665490688 00000100:00100000:1.0:1787739050.327197:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2349 00000100:00100000:1.0:1787739050.327207:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665490688:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739050.327611:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665490688:12345-0@lo:1000:osp_up0-1.0 Request processed in 392us (766us total) trans 4294983277 rc 0/0 00000100:00100000:1.0:1787739050.327619:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2349 00000100:00100000:3.0:1787739050.328075:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41da40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665490688:0@lo:1000:osp_up0-1.0 00000004:00200000:1.0:1787739050.328199:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x155:0x0]: returning size 4096 00010000:00010000:1.0:1787739050.328222:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1aaeef lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.328232:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1aaeef lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.328239:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1aaeef lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.328246:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739050.328250:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1aaeef lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.328258:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1aaeef lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.328277:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xaf:0x0].0x0 (ffffa074419d5100) 00010000:00010000:1.0:1787739050.328280:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739050.328285:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1aaeef lrc: 2/0,0 mode: --/EX res: [0x240000400:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.328291:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1aaeef lrc: 0/0,0 mode: --/EX res: [0x240000400:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.328305:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1aaef6 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aaefd expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.328313:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1aaef6 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aaefd expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.328320:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1aaef6 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aaefd expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.328345:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739050.328353:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xaf:0x0] 00010000:00010000:1.0:1787739050.328360:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1aaee8 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x155:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.328366:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1aaee8 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x155:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.328372:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1aaee8 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x155:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.328377:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739050.328380:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1aaee8 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x155:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.328386:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1aaee8 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x155:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.328394:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x155:0x0].0x0 (ffffa074419d5b00) 00010000:00010000:1.0:1787739050.328396:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739050.328399:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1aaee8 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x155:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.328405:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1aaee8 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x155:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.328592:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739050.328598:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1aaef6 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aaefd expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.328606:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739050.328611:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1aaef6 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aaefd expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739050.328618:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1aaef6 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aaefd expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.328648:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1aaef6 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aaefd expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.328654:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739050.328686:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1aaef6 lrc: 1/0,0 mode: --/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aaefd expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.328693:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1aaef6 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aaefd expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.328710:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739050.328712:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739050.328715:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739050.328724:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787739050.328736:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665490816:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739050.328791:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.328839:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665490816 00000100:00100000:3.0:1787739050.328845:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665490816 00000100:00100000:3.0:1787739050.328981:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1128 00000100:00100000:3.0:1787739050.328990:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+10:3651:x1874579665490816:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739050.329003:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739050.329020:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aaefd lrc: 0/0,0 mode: --/EX res: [0x200000401:0xaf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aaef6 expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.329033:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xaf:0x0].0x0 (ffffa0730608bb00) 00010000:00010000:3.0:1787739050.329040:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739050.329070:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+9:3651:x1874579665490816:12345-0@lo:103:ldlm_bl.0 Request processed in 82us (279us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.329076:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1128 00000100:00100000:3.0:1787739050.329117:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665490816:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739050.329701:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1aaeda lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x155:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aaee1 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.329712:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1aaeda lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x155:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aaee1 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.329719:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1aaeda lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x155:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aaee1 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.329777:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739050.329792:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739050.329798:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1aaebe lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aaec5 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.329805:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1aaebe lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aaec5 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.329813:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1aaebe lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aaec5 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.337046:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1787739050.337061:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1aaeda lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x155:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aaee1 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.337074:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739050.337081:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1aaeda lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x155:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aaee1 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739050.337091:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1aaeda lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x155:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aaee1 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.337148:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1aaeda lrc: 2/0,0 mode: --/EX res: [0x240000402:0x155:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aaee1 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.337156:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739050.337183:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1aaeda lrc: 1/0,0 mode: --/EX res: [0x240000402:0x155:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aaee1 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.337190:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1aaeda lrc: 0/0,0 mode: --/EX res: [0x240000402:0x155:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aaee1 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.337215:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739050.337219:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1aaebe lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aaec5 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.337226:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739050.337229:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1aaebe lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aaec5 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739050.337236:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1aaebe lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aaec5 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.337253:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1aaebe lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aaec5 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.337259:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739050.337275:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1aaebe lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aaec5 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.337289:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739050.337290:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739050.337323:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:1.0:1787739050.337332:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1aaebe lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aaec5 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739050.337354:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739050.337361:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1aaecc lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aaed3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.337368:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1aaecc lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aaed3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739050.337480:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17370:x1874579652396032:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 36255us (36500us total) trans 4294968267 rc 0/0 00000100:00100000:1.0:1787739050.337489:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23084 00010000:00010000:2.0:1787739050.338252:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739050.338272:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665491072:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739050.338330:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.338386:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787739050.338395:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665490944:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739050.338440:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739050.338525:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665491072 00000100:00100000:1.0:1787739050.338533:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665491072 00000100:00100000:1.0:1787739050.338578:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665490944 00000100:00100000:1.0:1787739050.338581:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665490944 00000100:00100000:1.0:1787739050.338602:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1129 00000100:00100000:1.0:1787739050.338611:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+10:3652:x1874579665491072:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739050.338625:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739050.338657:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1aaec5 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aaebe expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.338673:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403eebb00) 00010000:00010000:1.0:1787739050.338681:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739050.338700:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1130 00000100:00100000:3.0:1787739050.338709:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c0780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+9:3650:x1874579665490944:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739050.338718:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787739050.338728:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+9:3652:x1874579665491072:12345-0@lo:103:ldlm_bl.0 Request processed in 104us (383us total) trans 0 rc 0/0 00010000:00010000:3.0:1787739050.338729:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aaee1 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x155:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aaeda expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739050.338735:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1129 00010000:00010000:3.0:1787739050.338740:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x155:0x0].0x0 (ffffa07403eeb300) 00010000:00010000:3.0:1787739050.338746:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739050.338798:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665491072:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739050.338800:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c0780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+7:3650:x1874579665490944:12345-0@lo:103:ldlm_bl.0 Request processed in 93us (361us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.338806:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1130 00000100:00100000:1.0:1787739050.339283:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665490944:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739050.339447:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739050.393371:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739050.393480:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652396416 00000100:00100000:2.0:1787739050.393490:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652396416 00000100:00100000:2.0:1787739050.393539:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23085 00000100:00100000:2.0:1787739050.393548:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17371:x1874579652396416:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739050.393571:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739050.393592:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1aaf04 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094934 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.393658:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaf0b lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.393672:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaf0b lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.393683:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaf0b lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.393721:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1aaf12 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.393730:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1aaf12 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.393737:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1aaf12 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.393781:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaf0b lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.393788:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaf0b lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.393799:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaf0b lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.393806:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaf0b lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.393813:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739050.393817:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaf0b lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.393824:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaf0b lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.393835:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074419d5400) 00010000:00010000:2.0:1787739050.393839:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739050.393843:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaf0b lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.393849:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aaf0b lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.393862:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1aaf12 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.393868:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1aaf12 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.393873:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1aaf12 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.393878:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739050.393882:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1aaf12 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.393888:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1aaf12 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.393895:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xc795dd8e (ffffa07403eeb200) 00010000:00010000:2.0:1787739050.393898:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739050.393900:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1aaf12 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.393906:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1aaf12 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.393936:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1aaf04 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094934 expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.393946:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074419d5400) 00010000:00010000:2.0:1787739050.393948:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739050.393951:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1aaf04 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094934 expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.393959:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d400, rc 0) 00000100:00100000:2.0:1787739050.394021:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17371:x1874579652396416:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 474us (659us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739050.394028:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23085 00000100:00100000:3.0:1787739050.400859:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739050.401022:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652396544 00000100:00100000:0.0:1787739050.401032:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652396544 00000100:00100000:0.0:1787739050.401093:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23086 00000100:00100000:0.0:1787739050.401114:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17371:x1874579652396544:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739050.401136:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739050.401164:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aaf19 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09493b expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.401222:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1aaf20 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.401236:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1aaf20 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.401247:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1aaf20 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739050.401610:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:0.0:1787739050.401616:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739050.401658:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aaf19 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09493b expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.401675:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1aaf20 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09493b expref: 14 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.401683:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403a6f000) 00010000:00010000:0.0:1787739050.401687:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739050.401690:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609ac00, rc 0) 00000100:00100000:0.0:1787739050.401843:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17371:x1874579652396544:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 740us (990us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.401853:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23086 00000100:00100000:3.0:1787739050.414746:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739050.414802:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652396800 00000100:00100000:2.0:1787739050.414812:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652396800 00000100:00100000:2.0:1787739050.414870:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23087 00000100:00100000:2.0:1787739050.414880:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17371:x1874579652396800:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:2.0:1787739050.415194:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665491200:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739050.415253:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.415290:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.415696:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665491200 00000100:00100000:3.0:1787739050.415701:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665491200 00000100:00100000:3.0:1787739050.415759:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2350 00000100:00100000:3.0:1787739050.415768:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c03c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665491200:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739050.415857:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c03c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665491200:12345-0@lo:1000:mdt00_001.0 Request processed in 90us (602us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.415881:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2350 00000100:00100000:2.0:1787739050.416070:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665491200:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739050.416126:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665491328:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739050.416150:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.416183:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.417009:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665491328 00000100:00100000:3.0:1787739050.417017:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665491328 00000100:00100000:3.0:1787739050.417053:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2351 00000100:00100000:3.0:1787739050.417059:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c30c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665491328:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739050.417148:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c30c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665491328:12345-0@lo:1000:mdt00_001.0 Request processed in 89us (996us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739050.417155:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2351 00000100:00100000:3.0:1787739050.417184:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665491328:0@lo:1000:mdt00_001.0 00010000:00010000:3.0:1787739050.417234:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1aaf27 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.417245:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1aaf27 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.417256:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1aaf27 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.417269:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665491456:0@lo:101:mdt00_001.0 00000100:00100000:3.0:1787739050.417294:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.417324:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.417335:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665491456 00000100:00100000:3.0:1787739050.417338:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665491456 00000100:00100000:3.0:1787739050.417357:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2352 00000100:00100000:3.0:1787739050.417362:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41cb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665491456:12345-0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787739050.417370:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739050.417382:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aaf2e lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aaf27 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.417404:0:8448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1aae39 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094911 expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.417434:0:8448:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aaf2e lrc: 4/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaf27 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.417456:0:8448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1aae39 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094911 expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.417468:0:8448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1aae39 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094911 expref: 9 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.417478:0:8448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41de00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8448:1874579665491584:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739050.417522:0:8448:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688db40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.422045:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739050.422082:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652396928 00000100:00100000:3.0:1787739050.422089:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652396928 00010000:00010000:3.0:1787739050.422140:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1aae39 lrc: 5/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094911 expref: 10 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.422178:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1336 00000100:00100000:3.0:1787739050.422189:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41e580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652396928:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739050.422204:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739050.422211:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1aae39 lrc: 5/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094911 expref: 10 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.422224:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1aae39 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094911 expref: 10 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.422246:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074419d5400) 00010000:00010000:3.0:1787739050.422253:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aaf2e lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaf27 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.422264:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aaf2e lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1aaf27 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.422273:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aaf2e lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1aaf27 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.422306:0:10340:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aaf2e lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaf27 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.422319:0:10340:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304000780 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665491712:0@lo:105: 00000100:00100000:3.0:1787739050.422375:0:10340:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.422441:0:10340:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688df00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.422461:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665491712 00000100:00100000:3.0:1787739050.422465:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665491712 00000100:00100000:3.0:1787739050.422490:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1018 00000100:00100000:3.0:1787739050.422496:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564da40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665491712:12345-0@lo:105: 00010000:00010000:3.0:1787739050.422505:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1aaf27 lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aaf2e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.422514:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1aaf27 lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aaf2e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.422522:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1aaf27 lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aaf2e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.422529:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1aaf27 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1aaf2e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.422535:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1aaf27 lrc: 5/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1aaf2e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.422542:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1aaf27 lrc: 5/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1aaf2e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.422552:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069aba00) 00000100:00100000:3.0:1787739050.422584:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564da40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665491712:12345-0@lo:105: Request processed in 88us (209us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.422590:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1018 00000100:00100000:3.0:1787739050.422623:0:10340:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304000780 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665491712:0@lo:105: 00010000:00010000:3.0:1787739050.422635:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739050.422664:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41e580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652396928:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 479us (628us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.422670:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1336 00010000:00010000:0.0:1787739050.424983:0:8448:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1aae39 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094911 expref: 9 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739050.425001:0:8448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41de00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8448:1874579665491584:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739050.425023:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aaf2e lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaf27 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.425030:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074419d5400) 00010000:00010000:0.0:1787739050.425032:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739050.425034:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab600, rc 0) 00000100:00100000:0.0:1787739050.425146:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41cb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6524:x1874579665491456:12345-0@lo:101:mdt00_001.0 Request processed in 7782us (7850us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.425154:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2352 00000100:00100000:3.0:1787739050.425264:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665491456:0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787739050.425282:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1aaf27 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aaf2e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739050.425302:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739050.425339:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa200/0x7ee4453e4a1aaf35 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.425347:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa200/0x7ee4453e4a1aaf35 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.425354:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa200/0x7ee4453e4a1aaf35 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.425373:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665491840:0@lo:101:mdt00_001.0 00000100:00100000:3.0:1787739050.425406:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.425542:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739050.426887:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665491840 00000100:00100000:1.0:1787739050.426896:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665491840 00000100:00100000:1.0:1787739050.426945:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2353 00000100:00100000:1.0:1787739050.426954:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6524:x1874579665491840:12345-0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739050.426970:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739050.426997:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aaf3c lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aaf35 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.427022:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aaf3c lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaf35 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.427037:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aaf3c lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaf35 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.427044:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xc795dd8e (ffffa07305eaeb00) 00010000:00010000:1.0:1787739050.427047:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739050.427051:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d800, rc 0) 00000100:00100000:1.0:1787739050.427116:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665491840:12345-0@lo:101:mdt00_001.0 Request processed in 163us (1709us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.427123:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2353 00000100:00100000:0.0:1787739050.428726:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665491840:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739050.428749:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa200/0x7ee4453e4a1aaf35 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aaf3c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.428762:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa200/0x7ee4453e4a1aaf35 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aaf3c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739050.428784:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:0.0:1787739050.428833:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb6940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665491968:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739050.428879:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.428902:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d000 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.428922:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665491968 00000100:00100000:0.0:1787739050.428927:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665491968 00000100:00100000:0.0:1787739050.428962:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2354 00000100:00100000:0.0:1787739050.428969:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb6580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665491968:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739050.429049:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb6580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665491968:12345-0@lo:1000:mdt00_001.0 Request processed in 80us (169us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.429056:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2354 00000100:00100000:0.0:1787739050.429088:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb6940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665491968:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739050.429141:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb6940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665492096:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739050.429164:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.429177:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d000 going to sleep for 11 seconds 00000100:00100000:1.0:1787739050.429224:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665492096 00000100:00100000:1.0:1787739050.429228:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665492096 00000100:00100000:1.0:1787739050.429263:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2355 00000100:00100000:1.0:1787739050.429270:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb7840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665492096:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739050.429354:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb7840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665492096:12345-0@lo:1000:mdt00_001.0 Request processed in 84us (189us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739050.429370:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2355 00000100:00100000:0.0:1787739050.433804:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb6940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665492096:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739050.433918:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb6940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665492224:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739050.433977:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.434013:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d000 going to sleep for 11 seconds 00000100:00100000:1.0:1787739050.435078:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665492224 00000100:00100000:1.0:1787739050.435086:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665492224 00000100:00100000:1.0:1787739050.435149:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2356 00000100:00100000:1.0:1787739050.435158:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665492224:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739050.436160:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665492224:12345-0@lo:1000:mdt00_001.0 Request processed in 1000us (2178us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739050.436170:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2356 00000100:00100000:2.0:1787739050.443221:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb6940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665492224:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739050.443501:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41de00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665492352:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739050.443567:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.443598:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.443773:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665492352 00000100:00100000:2.0:1787739050.443779:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665492352 00000100:00100000:2.0:1787739050.443819:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2357 00000100:00100000:2.0:1787739050.443826:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665492352:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739050.443910:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665492352:12345-0@lo:1000:mdt00_001.0 Request processed in 83us (341us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.443918:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2357 00000100:00100000:3.0:1787739050.443938:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41de00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665492352:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739050.443995:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564e940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665492480:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739050.444027:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.444070:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d600 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.444075:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665492480 00000100:00100000:0.0:1787739050.444081:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665492480 00000100:00100000:0.0:1787739050.444126:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2358 00000100:00100000:0.0:1787739050.444133:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665492480:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739050.444214:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665492480:12345-0@lo:1000:mdt00_001.0 Request processed in 82us (186us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.444221:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2358 00000100:00100000:3.0:1787739050.452861:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564e940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665492480:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739050.452967:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564e940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665492608:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739050.453033:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.453117:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d600 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.454074:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665492608 00000100:00100000:0.0:1787739050.454082:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665492608 00000100:00100000:0.0:1787739050.454145:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2359 00000100:00100000:0.0:1787739050.454155:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665492608:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739050.454273:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665492608:12345-0@lo:1000:mdt00_001.0 Request processed in 119us (1239us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.454281:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2359 00000100:00100000:3.0:1787739050.454667:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564e940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665492608:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739050.454722:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564e940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665492736:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739050.454755:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.454792:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d600 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.454834:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665492736 00000100:00100000:0.0:1787739050.454839:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665492736 00000100:00100000:0.0:1787739050.454872:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2360 00000100:00100000:0.0:1787739050.454879:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665492736:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739050.454948:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665492736:12345-0@lo:1000:mdt00_001.0 Request processed in 69us (192us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.454956:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2360 00000100:00100000:3.0:1787739050.455048:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564e940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665492736:0@lo:1000:mdt00_001.0 00000040:00080000:3.0:1787739050.455959:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].333, 2168 off513136 00000040:00080000:3.0:1787739050.456046:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].332, 2368 off554496 00000004:00080000:3.0:1787739050.456052:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730688df00 with version 635 00000004:00080000:0.0:1787739050.456143:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 635 rpc_version 635 00000020:00100000:0.0:1787739050.456176:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739050.456185:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073056543c0 x1874579665492992/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739050.456204:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073056543c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665492992:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739050.456257:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.456298:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.456611:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665492992 00000100:00100000:2.0:1787739050.456619:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665492992 00000100:00100000:2.0:1787739050.456674:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2361 00000100:00100000:2.0:1787739050.456685:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305655e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665492992:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739050.457307:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305655e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665492992:12345-0@lo:1000:osp_up1-0.0 Request processed in 623us (1047us total) trans 4294968268 rc 0/0 00000100:00100000:2.0:1787739050.457316:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2361 00000100:00100000:1.0:1787739050.457492:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073056543c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665492992:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787739050.457713:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aaf43 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x14eb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.457729:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aaf43 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14eb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.457745:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aaf43 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14eb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.457837:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1aaf4a lrc: 3/0,1 mode: --/PW res: [0x200000400:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.457846:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1aaf4a lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.457853:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1aaf4a lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.457882:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1aaf51 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.457890:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1aaf51 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.457900:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1aaf51 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739050.457917:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073056552c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665493120:0@lo:101:mdt00_001.0 00000100:00100000:0.0:1787739050.457952:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.457981:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739050.458145:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665493120 00000100:00100000:1.0:1787739050.458150:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665493120 00000100:00100000:1.0:1787739050.458190:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2362 00000100:00100000:1.0:1787739050.458198:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305655680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6524:x1874579665493120:12345-0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739050.458213:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739050.458230:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aaf58 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aaf51 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.458252:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aaf58 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaf51 expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.458265:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aaf58 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaf51 expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.458274:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb2:0x0].0x0 (ffffa07305eae200) 00010000:00010000:1.0:1787739050.458278:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739050.458281:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1de00, rc 0) 00000100:00100000:1.0:1787739050.458315:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305655680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:6524:x1874579665493120:12345-0@lo:101:mdt00_001.0 Request processed in 118us (363us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.458322:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2362 00000100:00100000:0.0:1787739050.458633:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073056552c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665493120:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739050.458648:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1aaf51 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aaf58 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.458658:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1aaf51 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aaf58 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739050.458686:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000dff685e0 reply state 00000000d682e0a1 transno 4294983278 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1aaf4a lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.458699:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1aaf4a lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.458706:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb2:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:0.0:1787739050.458709:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739050.458712:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739050.458718:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1aaf51 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aaf58 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.458726:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1aaf51 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aaf58 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739050.458736:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xb2:0x0] 00000004:00010000:0.0:1787739050.458744:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000dff685e0 reply state 00000000d682e0a1 transno 4294983278 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aaf43 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14eb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.458752:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aaf43 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x14eb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.458758:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14eb:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:0.0:1787739050.458760:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739050.458762:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787739050.459210:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14eb:0x0]: returning size 4096 00000004:00080000:0.0:1787739050.459221:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739050.459231:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1aaf27 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aaf2e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.459245:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1aaf27 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aaf2e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.459253:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1aaf27 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aaf2e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.459279:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739050.459283:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739050.459288:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa200/0x7ee4453e4a1aaf35 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aaf3c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.459295:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa200/0x7ee4453e4a1aaf35 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aaf3c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.459330:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739050.459338:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1aaf27 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aaf2e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.459348:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aba00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739050.459354:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1aaf27 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aaf2e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739050.459362:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1aaf27 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aaf2e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739050.459373:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17371:x1874579652396800:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 44492us (44633us total) trans 4294983278 rc 0/0 00000100:00100000:0.0:1787739050.459381:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23087 00010000:00010000:1.0:1787739050.459390:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1aaf27 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aaf2e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.459399:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739050.459444:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1aaf27 lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aaf2e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.459452:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1aaf27 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aaf2e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.459468:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739050.459470:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739050.459472:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739050.459485:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f72d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665493248:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739050.459521:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.459554:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665493248 00000100:00100000:2.0:1787739050.459560:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665493248 00000100:00100000:2.0:1787739050.459603:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1131 00000100:00100000:2.0:1787739050.459611:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+11:3653:x1874579665493248:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739050.459620:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739050.459634:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aaf2e lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aaf27 expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.459645:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074419d5400) 00010000:00010000:2.0:1787739050.459652:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739050.459680:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+10:3653:x1874579665493248:12345-0@lo:103:ldlm_bl.0 Request processed in 71us (158us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.459687:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1131 00000100:00100000:1.0:1787739050.460270:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f72d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665493248:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739050.487763:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739050.489046:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652397184 00000100:00100000:1.0:1787739050.489055:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652397184 00000100:00100000:1.0:1787739050.489781:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23088 00000100:00100000:1.0:1787739050.489793:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17372:x1874579652397184:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:1.0:1787739050.489966:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17372:x1874579652397184:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 175us (2209us total) trans 0 rc -17/-17 00000100:00100000:1.0:1787739050.489975:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23088 00000100:00100000:0.0:1787739050.513992:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739050.514500:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652397312 00000100:00100000:3.0:1787739050.514510:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652397312 00000100:00100000:3.0:1787739050.514569:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23089 00000100:00100000:3.0:1787739050.514580:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073056552c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17373:x1874579652397312:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739050.514601:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739050.514626:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aaf5f lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094942 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.515043:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aaf66 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.515060:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aaf66 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.515072:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aaf66 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.515085:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aaf6d lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.515096:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aaf3c lrc: 2/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaf35 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.515116:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aaf6d lrc: 4/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.515150:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aaf3c lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1aaf35 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.515167:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564c780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665493376:0@lo:104: 00000100:00100000:3.0:1787739050.515223:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.515488:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688db40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.520319:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665493376 00000100:00100000:0.0:1787739050.520333:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665493376 00000100:00100000:0.0:1787739050.520518:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1019 00000100:00100000:0.0:1787739050.520529:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665493376:12345-0@lo:104: 00010000:00010000:0.0:1787739050.520548:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa200/0x7ee4453e4a1aaf35 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aaf3c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.520763:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787739050.520772:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665493376:12345-0@lo:104: Request processed in 245us (5548us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.520781:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1019 00010000:00010000:2.0:1787739050.520847:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739050.520856:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa200/0x7ee4453e4a1aaf35 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aaf3c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.520865:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739050.520871:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa200/0x7ee4453e4a1aaf35 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1aaf3c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739050.520881:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa200/0x7ee4453e4a1aaf35 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1aaf3c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.520916:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa200/0x7ee4453e4a1aaf35 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aaf3c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.520923:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739050.520957:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa200/0x7ee4453e4a1aaf35 lrc: 1/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aaf3c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.520966:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa200/0x7ee4453e4a1aaf35 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aaf3c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.520986:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739050.520988:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739050.520989:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739050.521038:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665493504:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739050.521080:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.521183:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665493504 00000100:00100000:2.0:1787739050.521188:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665493504 00010000:00010000:2.0:1787739050.521208:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aaf3c lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1aaf35 expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.521241:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1337 00000100:00100000:2.0:1787739050.521249:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+10:3650:x1874579665493504:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739050.521260:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787739050.523265:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564c780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665493376:0@lo:104: 00010000:00010000:1.0:1787739050.523290:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aaf6d lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.530700:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1aaecc lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aaed3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739050.530721:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1aaecc lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aaed3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.535025:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1aaecc lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aaed3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.535041:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739050.535068:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1aaecc lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aaed3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787739050.535090:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968268 is committed 00000100:00100000:3.0:1787739050.535099:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665493632:0@lo:103:jbd2/dm-1-8.0 00000001:00080000:2.0:1787739050.535125:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968267 is committed 00000020:00080000:2.0:1787739050.535136:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 325 batchid updated 00010000:00010000:0.0:1787739050.535140:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aae5c lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x155:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739050.535149:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07302e89000 refcount 2 committed 0 result 0 batchid 326 00010000:00010000:0.0:1787739050.535154:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aae5c lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x155:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739050.535155:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c695700 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07302e89300 00000020:00080000:2.0:1787739050.535164:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].319 00000020:00080000:2.0:1787739050.535168:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c695a00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa074184eea00 00010000:00010000:0.0:1787739050.535170:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aae5c lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x155:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739050.535172:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].318 00010000:00010000:0.0:1787739050.535189:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c600 already unused, calling callback (ffffffffc18da4f0) 00000020:00080000:2.0:1787739050.535196:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 326 committed batchid 325 00010000:00010000:0.0:1787739050.535198:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aae5c lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x155:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.535204:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787739050.535214:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aae5c lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x155:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00080000:02000000:2.0:1787739050.535217:0:7475:0:(osd_handler.c:2218:osd_trans_start()) *** cfs_fail_loc=19a, val=0*** 00010000:00010000:0.0:1787739050.535236:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x155:0x0].0x0 (ffffa074419d5b00) 00010000:00010000:0.0:1787739050.535239:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739050.535244:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aae5c lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x155:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.535250:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1aae5c lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x155:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.535270:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aae63 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.535275:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665493632 00000004:00080000:2.0:1787739050.535280:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730610ed80 with version 610 00000100:00100000:3.0:1787739050.535284:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665493632 00010000:00010000:0.0:1787739050.535285:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aae63 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739050.535293:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 325 cancel update log [0x200000401:0x3:0x0].318: rc = 0 00010000:00010000:0.0:1787739050.535306:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aae63 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.535312:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633cc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739050.535317:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aae63 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.535324:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aae63 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.535333:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xaf:0x0].0x0 (ffffa074419d5100) 00010000:00010000:0.0:1787739050.535336:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739050.535340:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aae63 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.535346:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aae63 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xaf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739050.535363:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 325 cancel update log [0x240000400:0x3:0x0].317: rc = 0 00000020:00080000:2.0:1787739050.535374:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 326 committed batchid 325 00010000:00010000:2.0:1787739050.535448:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aaf3c lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1aaf35 expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.535460:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xc795dd8e (ffffa07305eaeb00) 00010000:00010000:2.0:1787739050.535465:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aaf6d lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.535473:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aaf6d lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.535480:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aaf6d lrc: 4/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.535507:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739050.535531:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1132 00000100:00100000:3.0:1787739050.535544:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba7450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+7:3651:x1874579665493632:12345-0@lo:103:jbd2/dm-1-8.0 00000100:00100000:2.0:1787739050.535555:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+6:3650:x1874579665493504:12345-0@lo:103:ldlm_bl.0 Request processed in 14305us (14471us total) trans 0 rc 0/0 00010000:00010000:3.0:1787739050.535558:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1787739050.535567:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1337 00010000:00010000:3.0:1787739050.535578:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aaed3 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x79cdf7e6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aaecc expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.535597:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x79cdf7e6 (ffffa07403eebe00) 00010000:00010000:3.0:1787739050.535605:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739050.535630:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aaf6d lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.535650:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba7450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+6:3651:x1874579665493632:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 109us (448us total) trans 0 rc 0/0 00010000:00010000:0.0:1787739050.535651:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aaf6d lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.535664:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1132 00000100:00100000:2.0:1787739050.535728:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665493632:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:0.0:1787739050.535747:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004f00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665493760:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739050.535782:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.535808:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4db40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.535824:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665493760 00000100:00100000:2.0:1787739050.535829:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665493760 00000100:00100000:0.0:1787739050.535854:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665493504:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739050.535863:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2363 00000004:00080000:3.0:1787739050.536075:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 610 rpc_version 610 00000020:00100000:3.0:1787739050.536093:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739050.536101:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0730564d680 x1874579665494016/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739050.536168:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564d680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665494016:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739050.536170:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:7944:x1874579665493760:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739050.536256:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:7944:x1874579665493760:12345-0@lo:1000:mdt00_003.0 Request processed in 385us (473us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.536264:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.536269:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2363 00000100:00100000:3.0:1787739050.536292:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688db40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.536323:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004f00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665493760:0@lo:1000:mdt00_003.0 00010000:00010000:0.0:1787739050.536365:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1aaf74 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14eb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.536377:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1aaf74 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14eb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.536385:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1aaf74 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14eb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739050.536404:0:7944:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1aaf74 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14eb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.536563:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aaf66 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.536571:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aaf66 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.536578:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aaf66 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.536584:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aaf66 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.536590:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739050.536593:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aaf66 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.536599:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aaf66 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.536734:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b600) 00010000:00010000:0.0:1787739050.536737:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739050.536756:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aaf66 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.536763:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aaf66 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.536774:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aaf6d lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.536781:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aaf6d lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.536786:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aaf6d lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.536791:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739050.536795:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aaf6d lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.536801:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aaf6d lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.536808:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xc795dd8e (ffffa07305eaeb00) 00010000:00010000:0.0:1787739050.536811:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739050.536813:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aaf6d lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.536819:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aaf6d lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.536867:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aaf5f lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094942 expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.536896:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1aaf74 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14eb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094942 expref: 7 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.536904:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14eb:0x0].0x0 (ffffa07403a6fd00) 00010000:00010000:0.0:1787739050.536907:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739050.536909:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609be00, rc 0) 00000100:00100000:0.0:1787739050.536960:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073056552c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17373:x1874579652397312:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 22383us (22977us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.536968:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23089 00000100:00100000:3.0:1787739050.542550:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665494016 00000100:00100000:3.0:1787739050.542566:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665494016 00000100:00100000:3.0:1787739050.542657:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2364 00000100:00100000:3.0:1787739050.542667:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:7467:x1874579665494016:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739050.542864:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665494016:12345-0@lo:1000:osp_up0-1.0 Request processed in 198us (6602us total) trans 4294983279 rc 0/0 00000100:00100000:3.0:1787739050.542872:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2364 00000100:00100000:3.0:1787739050.543047:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564d680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665494016:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739050.560376:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739050.560934:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652397440 00000100:00100000:1.0:1787739050.560944:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652397440 00000100:00100000:1.0:1787739050.560998:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23090 00000100:00100000:1.0:1787739050.561010:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17373:x1874579652397440:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739050.561034:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739050.561057:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aaf7b lrc: 2/0,0 mode: --/CR res: [0x200000402:0x14eb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094949 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.561127:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1aaf82 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14eb:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.561142:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1aaf82 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14eb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.561157:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1aaf82 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14eb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739050.561251:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14eb:0x0]: returning size 4096 00000004:00200000:1.0:1787739050.561256:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739050.561292:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aaf7b lrc: 0/0,0 mode: --/CR res: [0x200000402:0x14eb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094949 expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.561308:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1aaf82 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14eb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094949 expref: 17 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.561316:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14eb:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:1.0:1787739050.561318:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739050.561322:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1ca00, rc 0) 00000100:00100000:1.0:1787739050.561639:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17373:x1874579652397440:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 630us (1271us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.561647:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23090 00000100:00100000:3.0:1787739050.566132:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739050.566227:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652397568 00000100:00100000:0.0:1787739050.566237:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652397568 00000100:00100000:0.0:1787739050.566296:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23091 00000100:00100000:0.0:1787739050.566306:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17373:x1874579652397568:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739050.566324:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739050.566343:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aaf89 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094950 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.566406:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1aaf90 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.566435:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1aaf90 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.566447:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1aaf90 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739050.566494:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xb2:0x0]: returning size 4096 00000004:00200000:0.0:1787739050.566499:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739050.566534:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aaf89 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094950 expref: 19 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.566551:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1aaf90 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094950 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.566559:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb2:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:0.0:1787739050.566562:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739050.566565:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a400, rc 0) 00000100:00100000:0.0:1787739050.566622:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17373:x1874579652397568:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 318us (497us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.566629:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23091 00000100:00100000:0.0:1787739050.573092:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739050.573223:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652397696 00000100:00100000:1.0:1787739050.573234:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652397696 00000100:00100000:1.0:1787739050.573336:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23092 00000100:00100000:1.0:1787739050.573347:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f5e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17373:x1874579652397696:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739050.573368:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739050.573389:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aaf97 lrc: 2/0,0 mode: --/CR res: [0x240000401:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094957 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.573482:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aaf9e lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.573502:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aaf58 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aaf51 expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.573510:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aaf9e lrc: 4/1,0 mode: --/PR res: [0x240000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.573543:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aaf58 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1aaf51 expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739050.573558:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f7103c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665494144:0@lo:104: 00000100:00100000:1.0:1787739050.573614:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739050.573661:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcce40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.574220:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665494144 00000100:00100000:2.0:1787739050.574226:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665494144 00000100:00100000:2.0:1787739050.574267:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1020 00000100:00100000:2.0:1787739050.574274:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f712d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665494144:12345-0@lo:104: 00010000:00010000:2.0:1787739050.574286:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1aaf51 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aaf58 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.574357:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739050.575060:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f712d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665494144:12345-0@lo:104: Request processed in 90us (754us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.575069:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1020 00010000:00010000:2.0:1787739050.575093:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739050.575099:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1aaf51 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aaf58 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.575123:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739050.575128:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1aaf51 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xb2:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1aaf58 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739050.575138:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1aaf51 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xb2:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1aaf58 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.575169:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1aaf51 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xb2:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aaf58 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.575175:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739050.575207:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1aaf51 lrc: 1/0,0 mode: --/PW res: [0x240000401:0xb2:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aaf58 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.575216:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1aaf51 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xb2:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aaf58 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.575233:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739050.575235:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739050.575237:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739050.575244:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787739050.575256:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665494272:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739050.575290:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.575323:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665494272 00000100:00100000:2.0:1787739050.575328:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665494272 00010000:00010000:2.0:1787739050.575346:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aaf58 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1aaf51 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.575367:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1338 00000100:00100000:2.0:1787739050.575374:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c0f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+6:3653:x1874579665494272:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739050.575384:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1787739050.575536:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f7103c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665494144:0@lo:104: 00010000:00010000:0.0:1787739050.575554:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aaf9e lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1787739050.577912:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 326 batchid updated 00000020:00080000:2.0:1787739050.578141:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 326 00010000:00010000:1.0:1787739050.578181:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aaf58 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1aaf51 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739050.578193:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730610e300 with version 611 00010000:00010000:1.0:1787739050.578197:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb2:0x0].0x0 (ffffa07305eae200) 00010000:00010000:1.0:1787739050.578202:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aaf9e lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739050.578203:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 326 cancel update log [0x200000401:0x3:0x0].319: rc = 0 00010000:00010000:1.0:1787739050.578211:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aaf9e lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.578219:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aaf9e lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739050.578222:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 611 rpc_version 611 00010000:00010000:1.0:1787739050.578247:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000020:00100000:3.0:1787739050.578254:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739050.578264:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0730564c000 x1874579665494528/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000020:00080000:2.0:1787739050.578269:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 326 cancel update log [0x240000400:0x3:0x0].318: rc = 0 00000020:00080000:2.0:1787739050.578281:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 326 00000100:00100000:3.0:1787739050.578292:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564c000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665494528:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739050.578336:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c0f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+5:3653:x1874579665494272:12345-0@lo:103:ldlm_bl.0 Request processed in 2953us (3035us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.578344:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1338 00010000:00010000:1.0:1787739050.578378:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aaf9e lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.578386:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aaf9e lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.578395:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00200000:1.0:1787739050.578453:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xb2:0x0]: returning size 4096 00000004:00200000:1.0:1787739050.578458:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739050.578495:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aaf97 lrc: 0/0,0 mode: --/CR res: [0x240000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094957 expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.578509:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aaf9e lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094957 expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.578516:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb2:0x0].0x0 (ffffa07305eae200) 00010000:00010000:1.0:1787739050.578519:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739050.578521:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c800, rc 0) 00000100:00100000:3.0:1787739050.578631:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062489c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.578633:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665494528 00000100:00100000:2.0:1787739050.578640:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665494528 00000100:00100000:3.0:1787739050.578981:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2365 00000100:00100000:3.0:1787739050.578992:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665494528:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739050.579462:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f5e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17373:x1874579652397696:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 6116us (6380us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.579475:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23092 00000100:00100000:2.0:1787739050.579636:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665494272:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739050.579670:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665494528:12345-0@lo:1000:osp_up0-1.0 Request processed in 680us (1276us total) trans 4294983280 rc 0/0 00000100:00100000:3.0:1787739050.579680:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2365 00000100:00100000:1.0:1787739050.579926:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564c000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665494528:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739050.593307:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739050.593358:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652397824 00000100:00100000:2.0:1787739050.593496:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652397824 00000100:00100000:2.0:1787739050.593570:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1133 00000100:00100000:2.0:1787739050.593583:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564cf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2408:x1874579652397824:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:2.0:1787739050.593609:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739050.593634:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1aaf74 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14eb:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094942 expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.593754:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14eb:0x0].0x0 (ffffa07403a6fd00) 00010000:00010000:2.0:1787739050.593765:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739050.593824:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564cf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652397824:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 244us (525us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.593832:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1133 00000100:00100000:3.0:1787739050.594178:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739050.594258:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652398080 00000100:00100000:1.0:1787739050.594263:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652398080 00000100:00100000:1.0:1787739050.594301:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23093 00000100:00100000:1.0:1787739050.594311:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17373:x1874579652398080:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:1.0:1787739050.594355:0:6524:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739050.594372:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1aaf82 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14eb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094949 expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.594392:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14eb:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:1.0:1787739050.594395:0:6524:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739050.594573:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d800/0x7ee4453e4a1aafa5 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.594583:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d800/0x7ee4453e4a1aafa5 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.594592:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d800/0x7ee4453e4a1aafa5 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739050.594611:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f713840 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665494656:0@lo:101:mdt00_001.0 00000100:00100000:1.0:1787739050.594655:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739050.594774:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.595078:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665494656 00000100:00100000:3.0:1787739050.595083:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665494656 00000100:00100000:3.0:1787739050.595212:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2366 00000100:00100000:3.0:1787739050.595221:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f711e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+5:6524:x1874579665494656:12345-0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787739050.595231:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739050.595248:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1aafac lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aafa5 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.595273:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1aafac lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aafa5 expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.595286:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1aafac lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aafa5 expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.595294:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b600) 00010000:00010000:3.0:1787739050.595298:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739050.595301:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa200, rc 0) 00000100:00100000:3.0:1787739050.595361:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f711e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6524:x1874579665494656:12345-0@lo:101:mdt00_001.0 Request processed in 142us (707us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.595368:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2366 00000100:00100000:2.0:1787739050.595448:0:6524:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968268 gen 2 00000004:00080000:2.0:1787739050.595454:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968264 committed transno 4294968268 00000020:00080000:2.0:1787739050.595460:0:6524:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07306248780 refcount 2 committed 0 result 0 batchid 335 00000020:00080000:2.0:1787739050.595464:0:6524:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00300 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07306248480 00000020:00080000:2.0:1787739050.595468:0:6524:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].331 00000020:00080000:2.0:1787739050.595471:0:6524:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00900 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa074184ee000 00000020:00080000:2.0:1787739050.595474:0:6524:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].332 00000004:00080000:2.0:1787739050.595497:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968266 committed transno 4294968268 00000004:00080000:2.0:1787739050.595507:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968268 committed transno 4294968268 00000100:00100000:2.0:1787739050.595531:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f713840 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665494656:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739050.595548:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d800/0x7ee4453e4a1aafa5 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aafac expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.595560:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d800/0x7ee4453e4a1aafa5 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aafac expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739050.595580:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:2.0:1787739050.595605:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1aafb3 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.595612:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1aafb3 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.595621:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1aafb3 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.595634:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c0000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665494784:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1787739050.595666:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.595842:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e900 going to sleep for 11 seconds 00000020:00080000:2.0:1787739050.595855:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 335 committed batchid 334 00000020:00080000:2.0:1787739050.595924:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 334 00000100:00100000:1.0:1787739050.595996:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665494784 00000100:00100000:1.0:1787739050.596001:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665494784 00000100:00100000:1.0:1787739050.596032:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2367 00000100:00100000:1.0:1787739050.596039:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c0780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6524:x1874579665494784:12345-0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739050.596048:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739050.596062:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aafba lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aafb3 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.596079:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aafba lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aafb3 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.596090:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aafba lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aafb3 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.596120:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xc795dd8e (ffffa07433235100) 00010000:00010000:1.0:1787739050.596124:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739050.596126:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1de00, rc 0) 00000100:00100000:1.0:1787739050.596160:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c0780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665494784:12345-0@lo:101:mdt00_001.0 Request processed in 122us (493us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.596166:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2367 00000100:00100000:2.0:1787739050.596356:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c0000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665494784:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739050.596368:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1aafb3 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aafba expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.596378:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1aafb3 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aafba expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739050.596390:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:2.0:1787739050.596506:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c0000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665494912:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739050.596534:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.596577:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e240 going to sleep for 11 seconds 00000100:00100000:1.0:1787739050.596605:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665494912 00000100:00100000:1.0:1787739050.596609:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665494912 00000100:00100000:1.0:1787739050.596668:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2368 00000100:00100000:1.0:1787739050.596676:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c21c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665494912:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739050.596861:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c21c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665494912:12345-0@lo:1000:mdt00_001.0 Request processed in 185us (325us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.596868:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2368 00000100:00100000:2.0:1787739050.596949:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c0000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665494912:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739050.596987:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c0000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665495040:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739050.597011:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.597071:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e240 going to sleep for 11 seconds 00000100:00100000:1.0:1787739050.597116:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665495040 00000100:00100000:1.0:1787739050.597120:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665495040 00000100:00100000:1.0:1787739050.597171:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2369 00000100:00100000:1.0:1787739050.597178:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073056543c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665495040:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739050.597240:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073056543c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665495040:12345-0@lo:1000:mdt00_001.0 Request processed in 62us (228us total) trans 0 rc 1/1 00000100:00100000:1.0:1787739050.597247:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2369 00000100:00100000:2.0:1787739050.597291:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c0000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665495040:0@lo:1000:mdt00_001.0 00010000:00010000:2.0:1787739050.597335:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aafc1 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.597344:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aafc1 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.597352:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aafc1 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.597363:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c0000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665495168:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1787739050.597388:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.597404:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e900 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.597436:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665495168 00000100:00100000:2.0:1787739050.597440:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665495168 00000100:00100000:2.0:1787739050.597464:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2370 00000100:00100000:2.0:1787739050.597472:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665495168:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739050.597481:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739050.597496:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aafc8 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aafc1 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.597514:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aafc8 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aafc1 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.597526:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aafc8 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aafc1 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.597534:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14eb:0x0].0x0 (ffffa07403eeb000) 00010000:00010000:2.0:1787739050.597537:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739050.597539:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633da00, rc 0) 00000100:00100000:2.0:1787739050.597566:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6524:x1874579665495168:12345-0@lo:101:mdt00_001.0 Request processed in 96us (177us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.597572:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2370 00000100:00100000:2.0:1787739050.597594:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c0000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665495168:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739050.597604:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aafc1 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aafc8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.597614:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aafc1 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aafc8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.597634:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aafcf lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.597642:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aafcf lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.597651:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aafcf lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.597669:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aafd6 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.597746:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1aaf90 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094950 expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.597763:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aafd6 lrc: 4/0,1 mode: --/EX res: [0x200000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.597844:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1aaf90 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094950 expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.597860:0:6524:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1aaf90 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094950 expref: 17 pid: 13793 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.597930:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c0000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665495296:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739050.597993:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e900 going to sleep for 11 seconds 00000100:00100000:1.0:1787739050.607236:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739050.607286:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c0000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665495296:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739050.607313:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aafd6 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.607468:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652398208 00000100:00100000:3.0:1787739050.607475:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652398208 00010000:00010000:3.0:1787739050.607510:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1aaf90 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094950 expref: 18 pid: 13793 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.607546:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1339 00000100:00100000:3.0:1787739050.607556:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2407:x1874579652398208:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739050.607571:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739050.607578:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1aaf90 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094950 expref: 18 pid: 13793 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.607589:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1aaf90 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094950 expref: 18 pid: 13793 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.607610:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1aaf90 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094950 expref: 18 pid: 13793 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.607622:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb2:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:3.0:1787739050.607627:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aafd6 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.607636:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aafd6 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.607647:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aafd6 lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.607666:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739050.607949:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aafd6 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.607956:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aafd6 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.608007:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1aafdd lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.608015:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1aafdd lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.608025:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1aafdd lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739050.608043:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f713c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665495424:0@lo:101:mdt00_001.0 00000100:00100000:3.0:1787739050.608129:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2407:x1874579652398208:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 574us (899us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.608138:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1339 00000100:00100000:1.0:1787739050.608621:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739050.608650:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc600 going to sleep for 11 seconds 00000100:00100000:1.0:1787739050.608667:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665495424 00000100:00100000:1.0:1787739050.608671:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665495424 00000100:00100000:1.0:1787739050.609224:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2371 00000100:00100000:1.0:1787739050.609233:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6524:x1874579665495424:12345-0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739050.609253:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739050.609268:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1aafe4 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aafdd expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.609292:0:18535:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aaf9e lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094957 expref: 6 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.609306:0:18535:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1aafe4 lrc: 4/0,0 mode: --/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aafdd expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.609330:0:18535:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aaf9e lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094957 expref: 6 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.609343:0:18535:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aaf9e lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094957 expref: 6 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739050.609354:0:18535:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f712580 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:18535:1874579665495552:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739050.609406:0:18535:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.619219:0:18535:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f712580 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:18535:1874579665495552:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739050.619254:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1aafe4 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aafdd expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.619267:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb2:0x0].0x0 (ffffa07305eae200) 00010000:00010000:0.0:1787739050.619270:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739050.619274:0:18535:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1aafe4 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aafdd expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.619285:0:18535:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1aafe4 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aafdd expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.619292:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d600, rc 0) 00000100:00100000:0.0:1787739050.619369:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665495424:12345-0@lo:101:mdt00_001.0 Request processed in 10137us (10747us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.619377:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2371 00000100:00100000:3.0:1787739050.624319:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f713c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665495424:0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787739050.624350:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1aafdd lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aafe4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.624363:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1aafdd lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aafe4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.625117:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739050.625615:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652398336 00000100:00100000:3.0:1787739050.625623:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652398336 00010000:00010000:3.0:1787739050.625659:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aaf9e lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094957 expref: 7 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.625728:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1340 00000100:00100000:3.0:1787739050.625738:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2409:x1874579652398336:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739050.625763:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739050.625770:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aaf9e lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094957 expref: 7 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.625790:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aaf9e lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094957 expref: 7 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.625811:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aaf9e lrc: 0/0,0 mode: --/PR res: [0x240000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094957 expref: 7 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.625825:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb2:0x0].0x0 (ffffa07305eae200) 00010000:00010000:3.0:1787739050.625832:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1aafe4 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aafdd expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.625842:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1aafe4 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1aafdd expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.625851:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1aafe4 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1aafdd expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.627005:0:6509:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1aafe4 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aafdd expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.627025:0:6509:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564c780 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001_UUID:6509:1874579665495680:0@lo:105: 00000100:00100000:3.0:1787739050.627091:0:6509:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.627153:0:6509:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306248a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739050.628454:0:15907:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787739050.628476:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665495680 00000100:00100000:1.0:1787739050.628484:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665495680 00000100:00100000:1.0:1787739050.628534:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1021 00000100:00100000:1.0:1787739050.628544:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306b9a850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6509:x1874579665495680:12345-0@lo:105: 00010000:00010000:1.0:1787739050.628558:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1aafdd lrc: 5/0,1 mode: --/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aafe4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.628570:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1aafdd lrc: 5/0,1 mode: --/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aafe4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.628578:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1aafdd lrc: 5/0,1 mode: --/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aafe4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.628587:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1aafdd lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1aafe4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.628596:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1aafdd lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1aafe4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.628604:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1aafdd lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1aafe4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.628627:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07441a1ca00) 00000100:00100000:1.0:1787739050.628690:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306b9a850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6509:x1874579665495680:12345-0@lo:105: Request processed in 147us (1598us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.628697:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1021 00010000:00010000:1.0:1787739050.628760:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1aafdd lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aafe4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.628770:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1aafdd lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aafe4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739050.631778:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f713c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665495808:0@lo:403:mdt00_001.0 00000100:00100000:1.0:1787739050.632152:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739050.632244:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc000 going to sleep for 11 seconds 00000100:00100000:1.0:1787739050.632307:0:6509:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564c780 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001_UUID:6509:1874579665495680:0@lo:105: 00010000:00010000:1.0:1787739050.633224:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739050.633288:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2409:x1874579652398336:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 7548us (8180us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.633300:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1340 00000100:00100000:3.0:1787739050.634728:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665495808 00000100:00100000:3.0:1787739050.634739:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665495808 00000100:00100000:2.0:1787739050.637081:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2372 00000100:00100000:2.0:1787739050.637096:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f712d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665495808:12345-0@lo:403:mdt00_001.0 00000100:00100000:2.0:1787739050.638755:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f712d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665495808:12345-0@lo:403:mdt00_001.0 Request processed in 1655us (6596us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.638768:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2372 00000100:00100000:3.0:1787739050.639180:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f713c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665495808:0@lo:403:mdt00_001.0 00000040:00080000:3.0:1787739050.640657:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].334, 952 off514088 00000040:00080000:3.0:1787739050.640838:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].333, 1032 off555528 00000004:00080000:3.0:1787739050.640846:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306248a80 with version 636 00000004:00080000:1.0:1787739050.641344:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 636 rpc_version 636 00000020:00100000:1.0:1787739050.641376:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739050.641387:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743f7121c0 x1874579665496064/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739050.641428:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f7121c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665496064:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739050.641587:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.641707:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fccd80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.641731:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665496064 00000100:00100000:0.0:1787739050.641739:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665496064 00000100:00100000:0.0:1787739050.641793:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2373 00000100:00100000:0.0:1787739050.641802:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07436d8ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665496064:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739050.642532:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07436d8ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665496064:12345-0@lo:1000:osp_up1-0.0 Request processed in 729us (943us total) trans 4294968269 rc 0/0 00000100:00100000:0.0:1787739050.642542:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2373 00000100:00100000:1.0:1787739050.643529:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f7121c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665496064:0@lo:1000:osp_up1-0.0 00000004:00200000:1.0:1787739050.643561:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14eb:0x0]: returning size 4096 00010000:00010000:1.0:1787739050.643584:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aafd6 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643594:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aafd6 lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643601:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aafd6 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643607:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633ce00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739050.643611:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aafd6 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643618:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aafd6 lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643632:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb2:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:1.0:1787739050.643635:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739050.643640:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aafd6 lrc: 2/0,0 mode: --/EX res: [0x200000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643645:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1aafd6 lrc: 0/0,0 mode: --/EX res: [0x200000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643658:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1aafdd lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aafe4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643666:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1aafdd lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aafe4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643672:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1aafdd lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aafe4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643688:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739050.643696:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xb2:0x0] 00010000:00010000:1.0:1787739050.643702:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aafcf lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643708:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aafcf lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643713:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aafcf lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643718:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739050.643720:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aafcf lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643726:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aafcf lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643733:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14eb:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:1.0:1787739050.643735:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739050.643738:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aafcf lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643744:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aafcf lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643755:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aafc1 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aafc8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643763:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aafc1 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aafc8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643770:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aafc1 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aafc8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643782:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:1.0:1787739050.643792:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739050.643797:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d800/0x7ee4453e4a1aafa5 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aafac expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643804:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d800/0x7ee4453e4a1aafa5 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aafac expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643811:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d800/0x7ee4453e4a1aafa5 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aafac expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643827:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:1.0:1787739050.643831:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739050.643837:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1aafb3 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aafba expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643845:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1aafb3 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aafba expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.643846:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1787739050.643856:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1aafdd lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aafe4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.643868:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ca00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739050.643873:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1aafdd lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aafe4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739050.643882:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1aafdd lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aafe4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.643914:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1aafdd lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aafe4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.643921:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1787739050.643926:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17373:x1874579652398080:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 49614us (49747us total) trans 4294983281 rc 0/0 00000100:00100000:1.0:1787739050.643937:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23093 00010000:00010000:0.0:1787739050.643947:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1aafdd lrc: 1/0,0 mode: --/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aafe4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.643957:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1aafdd lrc: 0/0,0 mode: --/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aafe4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643961:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1787739050.643968:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aafc1 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aafc8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.643975:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739050.643976:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739050.643981:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d800/0x7ee4453e4a1aafa5 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aafac expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.643983:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aafc1 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aafc8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.644001:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d800 already unused, calling callback (ffffffffc18e6ee0) 00000004:00010000:1.0:1787739050.644003:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aafc1 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aafc8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.644007:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d800/0x7ee4453e4a1aafa5 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aafac expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739050.644018:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d800/0x7ee4453e4a1aafa5 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aafac expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.644025:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aafc1 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aafc8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.644034:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739050.644089:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d800/0x7ee4453e4a1aafa5 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aafac expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.644097:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739050.644117:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aafc1 lrc: 1/0,0 mode: --/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aafc8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.644119:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d800/0x7ee4453e4a1aafa5 lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aafac expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.644133:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1aafc1 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aafc8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.644148:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739050.644150:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739050.644151:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739050.644224:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d800/0x7ee4453e4a1aafa5 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aafac expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.644240:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739050.644242:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739050.644244:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739050.644260:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665496192:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739050.644301:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.644355:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f711a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665496320:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739050.644379:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.644396:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787739050.644402:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665496448:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739050.644451:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.644645:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665496192 00000100:00100000:3.0:1787739050.644651:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665496192 00000100:00100000:3.0:1787739050.644697:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665496320 00000100:00100000:3.0:1787739050.644700:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665496320 00000100:00100000:3.0:1787739050.644714:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665496448 00000100:00100000:3.0:1787739050.644717:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665496448 00000100:00100000:1.0:1787739050.644974:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1134 00000100:00100000:1.0:1787739050.644986:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f6d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+12:3653:x1874579665496192:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739050.645000:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739050.645020:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1aafe4 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aafdd expref: 12 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.645136:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb2:0x0].0x0 (ffffa07305eae200) 00000100:00100000:0.0:1787739050.645137:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1135 00010000:00010000:1.0:1787739050.645148:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739050.645153:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f7840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+11:3650:x1874579665496320:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739050.645164:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739050.645177:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1aafc8 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14eb:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aafc1 expref: 11 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.645189:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14eb:0x0].0x0 (ffffa07403eeb000) 00010000:00010000:0.0:1787739050.645196:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739050.645228:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f7840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+10:3650:x1874579665496320:12345-0@lo:103:ldlm_bl.0 Request processed in 81us (848us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.645234:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1135 00000100:00100000:0.0:1787739050.645250:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1136 00000100:00100000:0.0:1787739050.645258:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba4c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+9:3650:x1874579665496448:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739050.645266:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739050.645277:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1aafac lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aafa5 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.645287:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b600) 00010000:00010000:0.0:1787739050.645294:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739050.645321:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f6d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+8:3653:x1874579665496192:12345-0@lo:103:ldlm_bl.0 Request processed in 327us (1008us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.645326:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba4c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+8:3650:x1874579665496448:12345-0@lo:103:ldlm_bl.0 Request processed in 67us (872us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.645334:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1134 00010000:00010000:2.0:1787739050.645871:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739050.646160:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1136 00000100:00100000:0.0:1787739050.646237:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665496192:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739050.646267:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665496448:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739050.646276:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f711a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665496320:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739050.669881:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739050.670178:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652398464 00000100:00100000:0.0:1787739050.670187:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652398464 00000100:00100000:0.0:1787739050.670245:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23094 00000100:00100000:0.0:1787739050.670256:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17383:x1874579652398464:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739050.670281:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739050.670303:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1aafeb lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09495e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.670636:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aaff2 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.670653:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aaff2 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.670671:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aaff2 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.670687:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1aaff9 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.670694:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1aaff9 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.670700:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1aaff9 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.670745:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aaff2 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.670753:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aaff2 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.670761:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aaff2 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.670768:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aaff2 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.670775:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609aa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739050.670780:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aaff2 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.670788:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aaff2 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.670799:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403a6f000) 00010000:00010000:0.0:1787739050.670803:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739050.670808:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aaff2 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.670814:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1aaff2 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.670826:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1aaff9 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.670832:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1aaff9 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.670838:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1aaff9 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.670844:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739050.670846:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1aaff9 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.670853:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1aaff9 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.670860:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xa70cbc1d (ffffa07403a6f100) 00010000:00010000:0.0:1787739050.670863:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739050.670866:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1aaff9 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.670872:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1aaff9 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.670904:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1aafeb lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09495e expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.670915:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403a6f000) 00010000:00010000:0.0:1787739050.670918:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739050.670922:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1aafeb lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09495e expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.670931:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b200, rc 0) 00000100:00100000:0.0:1787739050.670994:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17383:x1874579652398464:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 739us (1124us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739050.671002:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23094 00000100:00100000:1.0:1787739050.679786:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739050.679901:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652398592 00000100:00100000:0.0:1787739050.679910:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652398592 00000100:00100000:0.0:1787739050.679995:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23095 00000100:00100000:0.0:1787739050.680006:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f711680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17383:x1874579652398592:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739050.680025:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739050.680046:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab000 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094965 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.680113:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab007 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.680128:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab007 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.680139:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab007 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739050.680183:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x5:0x0]: returning size 4096 00000004:00200000:0.0:1787739050.680189:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739050.680225:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab000 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094965 expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.680241:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab007 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094965 expref: 7 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.680250:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07403a6fd00) 00010000:00010000:0.0:1787739050.680253:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739050.680256:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a800, rc 0) 00000100:00100000:0.0:1787739050.680319:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f711680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17383:x1874579652398592:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 315us (540us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.680327:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23095 00000100:00100000:0.0:1787739050.690015:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739050.690695:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652398848 00000100:00100000:1.0:1787739050.690705:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652398848 00000100:00100000:1.0:1787739050.690759:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23096 00000100:00100000:1.0:1787739050.690769:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f52c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17383:x1874579652398848:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:1.0:1787739050.690880:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f7112c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665496576:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739050.690937:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739050.690991:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc780 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.691548:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665496576 00000100:00100000:2.0:1787739050.691554:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665496576 00000100:00100000:2.0:1787739050.691594:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2374 00000100:00100000:2.0:1787739050.691601:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7130c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6524:x1874579665496576:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739050.691685:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7130c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6524:x1874579665496576:12345-0@lo:1000:mdt00_001.0 Request processed in 84us (747us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.691693:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2374 00000100:00100000:0.0:1787739050.691802:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f7112c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665496576:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739050.691847:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665496704:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739050.691875:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.691906:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739050.694092:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665496704 00000100:00100000:1.0:1787739050.694116:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665496704 00000100:00100000:1.0:1787739050.694173:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2375 00000100:00100000:1.0:1787739050.694233:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f7840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6524:x1874579665496704:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739050.694389:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f7840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6524:x1874579665496704:12345-0@lo:1000:mdt00_001.0 Request processed in 208us (2513us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739050.694462:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665496704:0@lo:1000:mdt00_001.0 00010000:00010000:0.0:1787739050.694530:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab00e lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.694539:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab00e lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.694549:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab00e lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739050.694564:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665496832:0@lo:101:mdt00_001.0 00000100:00100000:0.0:1787739050.694597:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739050.694616:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2375 00000100:00100000:3.0:1787739050.694626:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665496832 00000100:00100000:3.0:1787739050.694632:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665496832 00000100:00100000:0.0:1787739050.694633:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d600 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.694663:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2376 00000100:00100000:3.0:1787739050.694672:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f6d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6524:x1874579665496832:12345-0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787739050.694684:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739050.694700:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab015 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab00e expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.694726:0:17834:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1aaf20 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09493b expref: 16 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.694736:0:17834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab015 lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab00e expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.694761:0:17834:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1aaf20 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09493b expref: 16 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.694824:0:17834:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1aaf20 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09493b expref: 16 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.694842:0:17834:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564fc00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:17834:1874579665496960:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739050.694909:0:17834:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306248180 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.699232:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739050.699906:0:17834:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564fc00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:17834:1874579665496960:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739050.699943:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab015 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab00e expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.699955:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403a6f000) 00010000:00010000:3.0:1787739050.699958:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739050.699962:0:17834:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab015 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab00e expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.699972:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739050.700050:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f6d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665496832:12345-0@lo:101:mdt00_001.0 Request processed in 5379us (5451us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.700059:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2376 00000100:00100000:2.0:1787739050.700181:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652398976 00000100:00100000:2.0:1787739050.700188:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652398976 00010000:00010000:2.0:1787739050.700218:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1aaf20 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09493b expref: 17 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.700257:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1341 00000100:00100000:2.0:1787739050.700266:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2408:x1874579652398976:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739050.700281:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739050.700287:0:6510:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1aaf20 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09493b expref: 17 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.700299:0:6510:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1aaf20 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09493b expref: 17 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.700319:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1aaf20 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09493b expref: 17 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.700331:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403a6f000) 00010000:00010000:2.0:1787739050.700336:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab015 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab00e expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.700345:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab015 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab00e expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.700354:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab015 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab00e expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.700380:0:6510:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab015 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab00e expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.700394:0:6510:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c0f00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6510:1874579665497088:0@lo:105: 00000100:00100000:0.0:1787739050.700546:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665496832:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739050.700564:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab00e lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab015 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.700573:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab00e lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab015 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.700630:0:6510:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.700690:0:6510:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.700721:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665497088 00000100:00100000:0.0:1787739050.700734:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665497088 00000100:00100000:0.0:1787739050.700794:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1022 00000100:00100000:0.0:1787739050.700810:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6510:x1874579665497088:12345-0@lo:105: 00010000:00010000:0.0:1787739050.700819:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab00e lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab015 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.700828:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab00e lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab015 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.700837:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab00e lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab015 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.700844:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab00e lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab015 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.700851:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab00e lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab015 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.700864:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab00e lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ab015 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.700881:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609a200) 00000100:00100000:0.0:1787739050.700912:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6510:x1874579665497088:12345-0@lo:105: Request processed in 112us (283us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.700918:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1022 00000100:00100000:0.0:1787739050.701184:0:6510:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c0f00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6510:1874579665497088:0@lo:105: 00010000:00010000:0.0:1787739050.701198:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739050.701267:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:2408:x1874579652398976:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 1002us (2044us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.701275:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1341 00010000:00010000:0.0:1787739050.701297:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab00e lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab015 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.701305:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab00e lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab015 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739050.701323:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739050.701351:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ab01c lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.701358:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ab01c lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.701366:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ab01c lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739050.701383:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665497216:0@lo:101:mdt00_001.0 00000100:00100000:0.0:1787739050.701408:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.701712:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d600 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.701735:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665497216 00000100:00100000:3.0:1787739050.701742:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665497216 00000100:00100000:0.0:1787739050.701869:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2377 00000100:00100000:0.0:1787739050.701879:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665497216:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739050.701894:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739050.701909:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab023 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab01c expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.701929:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab023 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab01c expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.701940:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab023 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab01c expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.701948:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xa70cbc1d (ffffa0741590dc00) 00010000:00010000:0.0:1787739050.701951:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739050.701953:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b800, rc 0) 00000100:00100000:0.0:1787739050.702076:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6524:x1874579665497216:12345-0@lo:101:mdt00_001.0 Request processed in 198us (666us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.702082:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665497216:0@lo:101:mdt00_001.0 00000100:00100000:0.0:1787739050.702085:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2377 00010000:00010000:1.0:1787739050.702116:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ab01c lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab023 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.702127:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ab01c lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab023 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739050.702141:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:1.0:1787739050.702175:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665497344:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739050.702205:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739050.702233:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc840 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.702303:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665497344 00000100:00100000:0.0:1787739050.702309:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665497344 00000100:00100000:0.0:1787739050.702374:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2378 00000100:00100000:0.0:1787739050.702384:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7112c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6524:x1874579665497344:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739050.702489:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7112c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6524:x1874579665497344:12345-0@lo:1000:mdt00_001.0 Request processed in 108us (282us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.702496:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2378 00000100:00100000:2.0:1787739050.702525:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665497344:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739050.702563:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c03c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665497472:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739050.702591:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.702609:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610eb40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739050.702626:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665497472 00000100:00100000:1.0:1787739050.702632:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665497472 00000100:00100000:1.0:1787739050.702668:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2379 00000100:00100000:1.0:1787739050.702676:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c30c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6524:x1874579665497472:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739050.702751:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c30c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6524:x1874579665497472:12345-0@lo:1000:mdt00_001.0 Request processed in 75us (160us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739050.702758:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2379 00000100:00100000:2.0:1787739050.702819:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c03c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665497472:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739050.702889:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c03c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665497600:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739050.702915:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.702930:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610eb40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.702941:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665497600 00000100:00100000:2.0:1787739050.702945:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665497600 00000100:00100000:2.0:1787739050.702970:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2380 00000100:00100000:2.0:1787739050.702977:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c0780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6524:x1874579665497600:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739050.703042:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c0780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6524:x1874579665497600:12345-0@lo:1000:mdt00_001.0 Request processed in 65us (125us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739050.703049:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2380 00000100:00100000:1.0:1787739050.703157:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c03c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665497600:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739050.703207:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f711a40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665497728:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739050.703238:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739050.703301:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcca80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.703334:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665497728 00000100:00100000:0.0:1787739050.703338:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665497728 00000100:00100000:0.0:1787739050.703369:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2381 00000100:00100000:0.0:1787739050.703377:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6524:x1874579665497728:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739050.703695:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6524:x1874579665497728:12345-0@lo:1000:mdt00_001.0 Request processed in 318us (455us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.703702:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2381 00000100:00100000:1.0:1787739050.703757:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f711a40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665497728:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739050.704950:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f711a40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665497856:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739050.705066:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739050.705095:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcca80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.705115:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665497856 00000100:00100000:2.0:1787739050.705122:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665497856 00000100:00100000:2.0:1787739050.705155:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2382 00000100:00100000:2.0:1787739050.705162:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f711e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6524:x1874579665497856:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739050.705255:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f711e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6524:x1874579665497856:12345-0@lo:1000:mdt00_001.0 Request processed in 93us (188us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.705267:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2382 00000100:00100000:1.0:1787739050.705328:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f711a40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665497856:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739050.705383:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f711a40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665497984:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739050.705407:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.705790:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665497984 00000100:00100000:2.0:1787739050.705795:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665497984 00000100:00100000:1.0:1787739050.705801:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcca80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.705836:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2383 00000100:00100000:2.0:1787739050.705842:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6524:x1874579665497984:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739050.705907:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6524:x1874579665497984:12345-0@lo:1000:mdt00_001.0 Request processed in 64us (498us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.705914:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2383 00000100:00100000:3.0:1787739050.706175:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f711a40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665497984:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739050.706220:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564fc00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665498112:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739050.706250:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.706287:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306248e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.706510:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665498112 00000100:00100000:0.0:1787739050.706515:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665498112 00000100:00100000:0.0:1787739050.706550:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2384 00000100:00100000:0.0:1787739050.706557:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6524:x1874579665498112:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739050.706623:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6524:x1874579665498112:12345-0@lo:1000:mdt00_001.0 Request processed in 66us (370us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.706629:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2384 00000100:00100000:3.0:1787739050.706695:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564fc00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665498112:0@lo:1000:mdt00_001.0 00000040:00080000:3.0:1787739050.708213:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].320, 2168 off489528 00000040:00080000:3.0:1787739050.708293:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].321, 2368 off535992 00000004:00080000:3.0:1787739050.708299:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07306248180 with version 612 00000004:00080000:0.0:1787739050.708513:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 612 rpc_version 612 00000020:00100000:0.0:1787739050.708544:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739050.708551:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074074f4000 x1874579665498368/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739050.708567:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f4000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665498368:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739050.708620:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.708656:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.708712:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665498368 00000100:00100000:3.0:1787739050.708719:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665498368 00000100:00100000:3.0:1787739050.708757:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2385 00000100:00100000:3.0:1787739050.708764:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665498368:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739050.709708:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561ad00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:7467:x1874579665498368:12345-0@lo:1000:osp_up0-1.0 Request processed in 942us (1085us total) trans 4294983282 rc 0/0 00000100:00100000:3.0:1787739050.709718:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2385 00000100:00100000:0.0:1787739050.709828:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f4000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665498368:0@lo:1000:osp_up0-1.0 00010000:00010000:3.0:1787739050.709947:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab02a lrc: 3/0,1 mode: --/PW res: [0x240000402:0x157:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.709963:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab02a lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x157:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.709973:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab02a lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x157:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.709991:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab031 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.709998:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab031 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.710004:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab031 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.710031:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab038 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.710038:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab038 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.710047:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab038 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.710062:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564fc00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665498496:0@lo:101:mdt00_001.0 00000100:00100000:3.0:1787739050.710090:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.710175:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688df00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739050.710266:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665498496 00000100:00100000:1.0:1787739050.710272:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665498496 00000100:00100000:1.0:1787739050.710311:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2386 00000100:00100000:1.0:1787739050.710319:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:6524:x1874579665498496:12345-0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739050.710334:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739050.710381:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ab03f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab038 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.710400:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ab03f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab038 expref: 12 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.710472:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ab03f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab038 expref: 12 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.710483:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb0:0x0].0x0 (ffffa07433235600) 00010000:00010000:1.0:1787739050.710486:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739050.710489:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1dc00, rc 0) 00000100:00100000:1.0:1787739050.710555:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:6524:x1874579665498496:12345-0@lo:101:mdt00_001.0 Request processed in 237us (464us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.710562:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2386 00000100:00100000:3.0:1787739050.710790:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564fc00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665498496:0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787739050.710803:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab038 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab03f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.710812:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab038 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab03f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739050.710832:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008f6a0dbd reply state 000000003cb94f90 transno 4294968270 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab031 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.710843:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab031 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.710849:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb0:0x0].0x0 (ffffa0730608bd00) 00010000:00010000:3.0:1787739050.710852:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739050.710855:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787739050.710861:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab038 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab03f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.710868:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab038 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab03f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739050.710878:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xb0:0x0] 00000004:00010000:3.0:1787739050.710883:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008f6a0dbd reply state 000000003cb94f90 transno 4294968270 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab02a lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x157:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.710891:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab02a lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x157:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.710897:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x157:0x0].0x0 (ffffa0730608b800) 00010000:00010000:3.0:1787739050.710900:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739050.710901:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:3.0:1787739050.710906:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x157:0x0]: returning size 4096 00000004:00080000:3.0:1787739050.710914:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739050.710922:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab00e lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab015 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.710929:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab00e lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab015 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.710936:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab00e lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab015 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.710958:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739050.710963:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739050.710968:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ab01c lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab023 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.710975:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ab01c lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab023 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.711032:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739050.711040:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab00e lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab015 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.711048:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739050.711054:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab00e lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab015 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739050.711061:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab00e lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab015 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.711086:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab00e lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab015 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.711092:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:3.0:1787739050.711122:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f52c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17383:x1874579652398848:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 20353us (21114us total) trans 4294968270 rc 0/0 00000100:00100000:3.0:1787739050.711130:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23096 00000100:00100000:0.0:1787739050.711235:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665498624:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739050.711272:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.711335:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665498624 00000100:00100000:3.0:1787739050.711340:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665498624 00000100:00100000:3.0:1787739050.711374:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1137 00000100:00100000:3.0:1787739050.711380:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c382580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+12:3652:x1874579665498624:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739050.711389:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739050.711405:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab015 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab00e expref: 12 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.711441:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403a6f000) 00010000:00010000:3.0:1787739050.711450:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739050.711507:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab00e lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab015 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.711515:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab00e lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab015 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.711535:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739050.711537:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739050.711539:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739050.712864:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c382580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+11:3652:x1874579665498624:12345-0@lo:103:ldlm_bl.0 Request processed in 1479us (1586us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.712886:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1137 00000100:00100000:0.0:1787739050.713962:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665498624:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739050.729881:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739050.730129:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652399232 00000100:00100000:3.0:1787739050.730137:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652399232 00000100:00100000:3.0:1787739050.730186:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23097 00000100:00100000:3.0:1787739050.730196:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305657c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17384:x1874579652399232:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:3.0:1787739050.730350:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305657c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17384:x1874579652399232:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 154us (483us total) trans 0 rc -17/-17 00000100:00100000:3.0:1787739050.730357:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23097 00000100:00100000:0.0:1787739050.747917:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739050.748013:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652399360 00000100:00100000:1.0:1787739050.748023:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652399360 00000100:00100000:1.0:1787739050.748121:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23098 00000100:00100000:1.0:1787739050.748132:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305656d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17385:x1874579652399360:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739050.748154:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739050.748177:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab046 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09496c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.748239:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab04d lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.748252:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab04d lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.748264:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab04d lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.748274:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab054 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.748284:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab023 lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab01c expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.748292:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab054 lrc: 4/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.748321:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab023 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ab01c expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739050.748336:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f712d00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665498752:0@lo:104: 00000100:00100000:1.0:1787739050.748380:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739050.748465:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.748849:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665498752 00000100:00100000:3.0:1787739050.748854:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665498752 00000100:00100000:3.0:1787739050.748978:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1023 00000100:00100000:3.0:1787739050.748993:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665498752:12345-0@lo:104: 00010000:00010000:3.0:1787739050.749006:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ab01c lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab023 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.749076:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739050.749081:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665498752:12345-0@lo:104: Request processed in 96us (701us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.749087:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1023 00000100:00100000:1.0:1787739050.749196:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f712d00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665498752:0@lo:104: 00010000:00010000:0.0:1787739050.749199:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739050.749205:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ab01c lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab023 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.749214:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab054 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.749214:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739050.749219:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ab01c lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ab023 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739050.749228:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ab01c lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ab023 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.749255:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ab01c lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab023 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.749262:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:3.0:1787739050.749306:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00010000:00010000:0.0:1787739050.749333:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ab01c lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab023 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.749341:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ab01c lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab023 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.749359:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739050.749361:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739050.749363:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739050.749532:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305654000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665498880:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739050.749565:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.749707:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665498880 00000100:00100000:0.0:1787739050.749711:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665498880 00010000:00010000:0.0:1787739050.749730:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab023 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ab01c expref: 11 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739050.749887:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1342 00000100:00100000:0.0:1787739050.749895:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564fc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+11:3652:x1874579665498880:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739050.749905:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739050.751957:0:6502:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1aafb3 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aafba expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739050.751975:0:6502:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1aafb3 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aafba expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.752013:0:6502:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1aafb3 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aafba expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.752020:0:6502:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1787739050.752069:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f712d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665499008:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:0.0:1787739050.752134:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787739050.752201:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1aafb3 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aafba expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787739050.752222:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983282 is committed 00000001:00080000:1.0:1787739050.752236:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983281 is committed 00000020:00080000:1.0:1787739050.752242:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 335 batchid updated 00000020:00080000:1.0:1787739050.752252:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa0730688d600 refcount 2 committed 0 result 0 batchid 336 00000020:00080000:1.0:1787739050.752256:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c695800 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730688df00 00000020:00080000:1.0:1787739050.752259:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].332 00000020:00080000:1.0:1787739050.752261:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c695900 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304f1e200 00000020:00080000:1.0:1787739050.752264:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].333 00010000:00010000:1.0:1787739050.752325:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab023 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ab01c expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.752338:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xa70cbc1d (ffffa0741590dc00) 00010000:00010000:1.0:1787739050.752343:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab054 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.752357:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab054 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.752365:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab054 lrc: 4/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.752389:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739050.752521:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aaf43 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x14eb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.752530:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aaf43 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x14eb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.752536:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aaf43 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x14eb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.752542:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609bc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739050.752545:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aaf43 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x14eb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.752551:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aaf43 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x14eb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.752560:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14eb:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:2.0:1787739050.752562:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739050.752566:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aaf43 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x14eb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.752571:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aaf43 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x14eb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.752586:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1aaf4a lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.752593:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1aaf4a lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.752599:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1aaf4a lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.752604:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739050.752607:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1aaf4a lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.752613:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1aaf4a lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.752620:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb2:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:2.0:1787739050.752623:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739050.752625:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1aaf4a lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.752631:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1aaf4a lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739050.752653:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 336 committed batchid 335 00000004:00080000:2.0:1787739050.752760:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730610e900 with version 637 00000020:00080000:2.0:1787739050.752772:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 335 cancel update log [0x240000401:0x3:0x0].331: rc = 0 00000020:00080000:2.0:1787739050.752812:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 335 cancel update log [0x200000400:0x3:0x0].332: rc = 0 00000020:00080000:2.0:1787739050.752822:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 335 00000004:00080000:2.0:1787739050.752831:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 637 rpc_version 637 00000020:00100000:2.0:1787739050.752850:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739050.752858:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073043c21c0 x1874579665499264/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739050.752873:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665499264:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739050.752911:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.752932:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e180 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.752949:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665499008 00000100:00100000:2.0:1787739050.752954:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665499008 00000100:00100000:2.0:1787739050.752994:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1138 00000100:00100000:2.0:1787739050.753002:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305654f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+7:3650:x1874579665499008:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:2.0:1787739050.753014:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739050.753027:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aafba lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xc795dd8e bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aafb3 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.753039:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xc795dd8e (ffffa07433235100) 00010000:00010000:2.0:1787739050.753045:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739050.753072:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305654f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+6:3650:x1874579665499008:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 71us (937us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.753079:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1138 00000100:00100000:0.0:1787739050.753086:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f712d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665499008:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787739050.753101:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665499264 00000100:00100000:2.0:1787739050.753117:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665499264 00000100:00100000:2.0:1787739050.753150:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2387 00000100:00100000:2.0:1787739050.753157:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c03c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665499264:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739050.753184:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564fc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+6:3652:x1874579665498880:12345-0@lo:103:ldlm_bl.0 Request processed in 3289us (3617us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.753191:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1342 00000100:00100000:1.0:1787739050.753232:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305654000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665498880:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739050.753252:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab054 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.753260:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab054 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.753266:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c03c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665499264:12345-0@lo:1000:osp_up1-0.0 Request processed in 109us (354us total) trans 4294968271 rc 0/0 00000100:00100000:2.0:1787739050.753271:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2387 00000100:00100000:2.0:1787739050.753300:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665499264:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739050.753337:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f712580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665499392:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739050.753365:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739050.753391:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc540 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.753451:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665499392 00000100:00100000:0.0:1787739050.753455:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665499392 00000100:00100000:0.0:1787739050.753482:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2388 00000100:00100000:0.0:1787739050.753489:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665499392:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739050.753548:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665499392:12345-0@lo:1000:mdt00_003.0 Request processed in 59us (181us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.753554:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2388 00000100:00100000:2.0:1787739050.753622:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f712580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665499392:0@lo:1000:mdt00_003.0 00010000:00010000:2.0:1787739050.753657:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab05b lrc: 3/1,0 mode: --/PR res: [0x240000402:0x157:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.753673:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab05b lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x157:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.753681:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab05b lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x157:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739050.753703:0:7944:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab05b lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x157:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.753712:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab04d lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.753716:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab04d lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.753720:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab04d lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.753723:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab04d lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.753727:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739050.753728:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab04d lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.753732:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab04d lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.753737:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235400) 00010000:00010000:2.0:1787739050.753739:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739050.753741:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab04d lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.753745:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab04d lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.753756:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab054 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.753762:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab054 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.753767:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab054 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.753772:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1cc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739050.753775:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab054 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.753780:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab054 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.753787:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xa70cbc1d (ffffa0741590dc00) 00010000:00010000:2.0:1787739050.753790:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739050.753793:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab054 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.753798:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab054 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.753832:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab046 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09496c expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.753848:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab05b lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x157:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09496c expref: 14 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.753856:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x157:0x0].0x0 (ffffa07403eebd00) 00010000:00010000:2.0:1787739050.753858:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739050.753861:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d800, rc 0) 00000100:00100000:2.0:1787739050.753913:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305656d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17385:x1874579652399360:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 5782us (6006us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.753920:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23098 00000100:00100000:2.0:1787739050.757181:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739050.757218:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652399488 00000100:00100000:3.0:1787739050.757228:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652399488 00000100:00100000:3.0:1787739050.757270:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23099 00000100:00100000:3.0:1787739050.757281:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17385:x1874579652399488:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739050.757303:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739050.757320:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab062 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x157:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094973 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.757457:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab069 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x157:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.757470:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab069 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x157:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.757482:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab069 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x157:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739050.757569:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x157:0x0]: returning size 4096 00000004:00200000:3.0:1787739050.757574:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739050.757664:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab062 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x157:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094973 expref: 11 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.757677:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab069 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x157:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094973 expref: 10 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.757682:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x157:0x0].0x0 (ffffa0730608b800) 00010000:00010000:3.0:1787739050.757684:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739050.757687:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aaa00, rc 0) 00000100:00100000:3.0:1787739050.757735:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17385:x1874579652399488:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 456us (560us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.757742:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23099 00000100:00100000:1.0:1787739050.762800:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739050.762892:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652399616 00000100:00100000:3.0:1787739050.762901:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652399616 00000100:00100000:3.0:1787739050.762997:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23100 00000100:00100000:3.0:1787739050.763007:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7103c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17385:x1874579652399616:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739050.763027:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739050.763046:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab070 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09497a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.763115:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab077 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.763130:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab077 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.763141:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab077 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739050.763185:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xb0:0x0]: returning size 4096 00000004:00200000:3.0:1787739050.763191:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739050.763222:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab070 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09497a expref: 12 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.763236:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab077 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09497a expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.763242:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb0:0x0].0x0 (ffffa0730608bd00) 00010000:00010000:3.0:1787739050.763245:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739050.763247:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abc00, rc 0) 00000100:00100000:3.0:1787739050.763313:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7103c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17385:x1874579652399616:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 308us (523us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.763324:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23100 00000100:00100000:0.0:1787739050.766040:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739050.766125:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652399744 00000100:00100000:2.0:1787739050.766133:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652399744 00000100:00100000:2.0:1787739050.766193:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23101 00000100:00100000:2.0:1787739050.766203:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305656580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17385:x1874579652399744:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739050.766220:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739050.766238:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ab07e lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094981 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.766291:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab085 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.766305:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ab03f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab038 expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.766313:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab085 lrc: 4/1,0 mode: --/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.766342:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ab03f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ab038 expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.766361:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb4b40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665499520:0@lo:104: 00000100:00100000:2.0:1787739050.766402:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.766666:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e180 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.766863:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665499520 00000100:00100000:0.0:1787739050.766920:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665499520 00000100:00100000:0.0:1787739050.766964:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1024 00000100:00100000:0.0:1787739050.766972:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb70c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665499520:12345-0@lo:104: 00010000:00010000:0.0:1787739050.766984:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab038 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab03f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.767126:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787739050.767132:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb70c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665499520:12345-0@lo:104: Request processed in 161us (729us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.767141:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1024 00010000:00010000:1.0:1787739050.767199:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739050.767206:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab038 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab03f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.767215:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739050.767219:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab038 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ab03f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739050.767233:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab038 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ab03f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.767260:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab038 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab03f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.767267:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739050.767292:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab038 lrc: 1/0,0 mode: --/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab03f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.767300:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab038 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab03f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.767321:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739050.767322:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739050.767324:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739050.767331:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb4b40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665499520:0@lo:104: 00010000:00010000:2.0:1787739050.767349:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab085 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.767360:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665499648:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739050.767397:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.767602:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665499648 00000100:00100000:2.0:1787739050.767607:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665499648 00010000:00010000:2.0:1787739050.767625:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ab03f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ab038 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.767678:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1343 00000100:00100000:2.0:1787739050.767686:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+6:3651:x1874579665499648:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739050.767696:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00080000:1.0:1787739050.770124:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 336 batchid updated 00000020:00080000:2.0:1787739050.770206:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 336 00000004:00080000:2.0:1787739050.770256:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730610e180 with version 638 00000020:00080000:2.0:1787739050.770271:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 336 cancel update log [0x240000401:0x3:0x0].332: rc = 0 00000004:00080000:0.0:1787739050.770307:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 638 rpc_version 638 00000020:00100000:0.0:1787739050.770336:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000020:00080000:2.0:1787739050.770337:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 336 cancel update log [0x200000400:0x3:0x0].333: rc = 0 00000020:00080000:2.0:1787739050.770345:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 336 00000100:00100000:0.0:1787739050.770345:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07305657480 x1874579665499904/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739050.770365:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305657480 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665499904:0@lo:1000:osp_up1-0.0 00010000:00010000:3.0:1787739050.770530:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ab03f lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ab038 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.770548:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb0:0x0].0x0 (ffffa07433235600) 00010000:00010000:3.0:1787739050.770553:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab085 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.770562:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab085 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.770570:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab085 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.770588:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739050.770829:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab085 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.770910:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665499648:12345-0@lo:103:ldlm_bl.0 Request processed in 3225us (3511us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.770920:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1343 00000100:00100000:0.0:1787739050.770940:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.770962:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665499904 00000100:00100000:0.0:1787739050.770965:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.770968:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665499904 00000100:00100000:3.0:1787739050.770999:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2389 00000100:00100000:3.0:1787739050.771007:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305655a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665499904:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739050.771134:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305655a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665499904:12345-0@lo:1000:osp_up1-0.0 Request processed in 127us (194us total) trans 4294968272 rc 0/0 00000100:00100000:3.0:1787739050.771142:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2389 00010000:00010000:2.0:1787739050.771175:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab085 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739050.771226:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb0:0x0]: returning size 4096 00000004:00200000:2.0:1787739050.771231:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000100:00100000:0.0:1787739050.771250:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305657480 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665499904:0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1787739050.771274:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ab07e lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094981 expref: 16 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.771292:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab085 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094981 expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.771297:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb0:0x0].0x0 (ffffa07433235600) 00010000:00010000:2.0:1787739050.771299:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739050.771301:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633dc00, rc 0) 00000100:00100000:2.0:1787739050.771339:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305656580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17385:x1874579652399744:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 5138us (5306us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.771346:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23101 00000100:00100000:1.0:1787739050.771596:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665499648:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739050.775815:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739050.775852:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739050.775961:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652399872 00000100:00100000:0.0:1787739050.775971:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652399872 00000100:00100000:0.0:1787739050.776039:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1139 00000100:00100000:0.0:1787739050.776049:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2408:x1874579652399872:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:0.0:1787739050.776065:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739050.776090:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab05b lrc: 0/0,0 mode: --/PR res: [0x240000402:0x157:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09496c expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.776119:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x157:0x0].0x0 (ffffa07403eebd00) 00010000:00010000:0.0:1787739050.776129:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739050.776178:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2408:x1874579652399872:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 131us (372us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.776185:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1139 00000100:00100000:3.0:1787739050.779223:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652400128 00000100:00100000:3.0:1787739050.779236:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652400128 00000100:00100000:0.0:1787739050.779467:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23102 00000100:00100000:0.0:1787739050.779479:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17385:x1874579652400128:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:0.0:1787739050.779528:0:6524:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739050.779551:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab069 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x157:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094973 expref: 11 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.779568:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x157:0x0].0x0 (ffffa0730608b800) 00010000:00010000:0.0:1787739050.779573:0:6524:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739050.779626:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab08c lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.779634:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab08c lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.779643:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab08c lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739050.779663:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305656940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665500032:0@lo:101:mdt00_001.0 00000100:00100000:0.0:1787739050.779723:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.779851:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665500032 00000100:00100000:3.0:1787739050.779856:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665500032 00000100:00100000:3.0:1787739050.779900:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2390 00000100:00100000:3.0:1787739050.779907:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305657840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+5:6524:x1874579665500032:12345-0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787739050.779917:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739050.779932:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab093 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab08c expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.779953:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab093 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab08c expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.779966:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab093 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab08c expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.779973:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730608ba00) 00010000:00010000:3.0:1787739050.779976:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739050.779978:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aac00, rc 0) 00000100:00100000:0.0:1787739050.780020:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.783232:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305657840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:6524:x1874579665500032:12345-0@lo:101:mdt00_001.0 Request processed in 3316us (3499us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.783243:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2390 00000100:00100000:0.0:1787739050.783322:0:6524:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983282 gen 2 00000004:00080000:0.0:1787739050.783329:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983277 committed transno 4294983282 00000020:00080000:0.0:1787739050.783335:0:6524:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa0730623d3c0 refcount 2 committed 0 result 0 batchid 327 00000020:00080000:0.0:1787739050.783339:0:6524:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00400 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730623d000 00000020:00080000:0.0:1787739050.783343:0:6524:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].320 00000020:00080000:0.0:1787739050.783346:0:6524:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00500 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa074184ef800 00000020:00080000:0.0:1787739050.783348:0:6524:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].319 00000020:00080000:2.0:1787739050.783385:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 327 committed batchid 326 00000020:00080000:2.0:1787739050.783491:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 326 00000004:00080000:0.0:1787739050.783848:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983279 committed transno 4294983282 00000004:00080000:0.0:1787739050.783858:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983280 committed transno 4294983282 00000004:00080000:0.0:1787739050.783867:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983282 committed transno 4294983282 00000100:00100000:0.0:1787739050.783951:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305656940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665500032:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739050.783976:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab08c lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab093 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.783991:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab08c lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab093 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739050.784009:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739050.784045:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab09a lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.784053:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab09a lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.784063:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab09a lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739050.784083:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305656940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665500160:0@lo:101:mdt00_001.0 00000100:00100000:0.0:1787739050.784138:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.784209:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.784244:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665500160 00000100:00100000:3.0:1787739050.784250:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665500160 00000100:00100000:3.0:1787739050.784579:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2391 00000100:00100000:3.0:1787739050.784589:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305655e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:6524:x1874579665500160:12345-0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787739050.784670:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739050.784687:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab0a1 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab09a expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.784716:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab0a1 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab09a expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.784729:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab0a1 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab09a expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.784738:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xa70cbc1d (ffffa0730608b100) 00010000:00010000:3.0:1787739050.784741:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739050.784744:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739050.784966:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305655e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665500160:12345-0@lo:101:mdt00_001.0 Request processed in 378us (827us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.784973:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2391 00000100:00100000:0.0:1787739050.785036:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305656940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665500160:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739050.785049:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab09a lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab0a1 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.785058:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab09a lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab0a1 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739050.785071:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:0.0:1787739050.785116:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305656940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665500288:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739050.785143:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.785209:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d840 going to sleep for 11 seconds 00000100:00100000:1.0:1787739050.785242:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665500288 00000100:00100000:1.0:1787739050.785246:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665500288 00000100:00100000:1.0:1787739050.786000:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2392 00000100:00100000:1.0:1787739050.786014:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305655680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665500288:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739050.786140:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305655680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665500288:12345-0@lo:1000:mdt00_001.0 Request processed in 130us (994us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.786146:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2392 00000100:00100000:0.0:1787739050.786256:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305656940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665500288:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739050.786305:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305656940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665500416:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739050.786333:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.786397:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d840 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.788477:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665500416 00000100:00100000:2.0:1787739050.788489:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665500416 00000100:00100000:2.0:1787739050.788545:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2393 00000100:00100000:2.0:1787739050.788556:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073056543c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665500416:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739050.788670:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073056543c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665500416:12345-0@lo:1000:mdt00_001.0 Request processed in 116us (2335us total) trans 0 rc 1/1 00000100:00100000:2.0:1787739050.788677:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2393 00000100:00100000:2.0:1787739050.788718:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305656940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665500416:0@lo:1000:mdt00_001.0 00010000:00010000:2.0:1787739050.788780:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1ab0a8 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x157:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.788791:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1ab0a8 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x157:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.788812:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1ab0a8 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x157:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.788829:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb7840 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665500544:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1787739050.788861:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.788893:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e840 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.789024:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665500544 00000100:00100000:0.0:1787739050.789029:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665500544 00000100:00100000:0.0:1787739050.789056:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2394 00000100:00100000:0.0:1787739050.789064:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb6940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665500544:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739050.789075:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739050.789091:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab0af lrc: 2/0,0 mode: --/EX res: [0x240000402:0x157:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab0a8 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.789129:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab0af lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x157:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab0a8 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.789143:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab0af lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x157:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab0a8 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.789152:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x157:0x0].0x0 (ffffa0741590d100) 00010000:00010000:0.0:1787739050.789155:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739050.789158:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b600, rc 0) 00000100:00100000:0.0:1787739050.789204:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb6940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6524:x1874579665500544:12345-0@lo:101:mdt00_001.0 Request processed in 140us (342us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.789210:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2394 00000100:00100000:2.0:1787739050.789777:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb7840 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665500544:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739050.789793:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1ab0a8 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x157:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab0af expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.789818:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1ab0a8 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x157:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab0af expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.789845:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab0b6 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x157:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.789855:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab0b6 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x157:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.789864:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab0b6 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x157:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.789881:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab0bd lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.789896:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab077 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09497a expref: 10 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.789904:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab0bd lrc: 4/0,1 mode: --/EX res: [0x240000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.789925:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab077 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09497a expref: 10 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.789938:0:6524:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab077 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09497a expref: 10 pid: 6523 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.789949:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb7840 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665500672:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739050.790035:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e840 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.793212:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb7840 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665500672:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739050.793238:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab0bd lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.797341:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739050.798278:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652400256 00000100:00100000:1.0:1787739050.798288:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652400256 00010000:00010000:1.0:1787739050.798328:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab077 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09497a expref: 11 pid: 6523 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739050.798368:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1344 00000100:00100000:1.0:1787739050.798378:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba6c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2408:x1874579652400256:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739050.798392:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739050.798400:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab077 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09497a expref: 11 pid: 6523 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.798437:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab077 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09497a expref: 11 pid: 6523 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.798461:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab077 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09497a expref: 11 pid: 6523 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.798477:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb0:0x0].0x0 (ffffa0730608bd00) 00010000:00010000:1.0:1787739050.798482:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab0bd lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.798491:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab0bd lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.798498:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab0bd lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.798619:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739050.798680:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba6c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652400256:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 302us (1356us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.798688:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1344 00010000:00010000:3.0:1787739050.798741:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab0bd lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.798751:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab0bd lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.798796:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ab0c4 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.798803:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ab0c4 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.798812:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ab0c4 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.798830:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564d2c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665500800:0@lo:101:mdt00_001.0 00000100:00100000:3.0:1787739050.798874:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.798922:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.799342:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665500800 00000100:00100000:2.0:1787739050.799348:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665500800 00000100:00100000:2.0:1787739050.800482:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2395 00000100:00100000:2.0:1787739050.800494:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6524:x1874579665500800:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739050.800509:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739050.800524:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab0cb lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab0c4 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.800548:0:18535:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab085 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094981 expref: 13 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.800554:0:18535:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab0cb lrc: 4/0,0 mode: --/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab0c4 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.800575:0:18535:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab085 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094981 expref: 13 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.800584:0:18535:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab085 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094981 expref: 13 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.800593:0:18535:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb7840 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:18535:1874579665500928:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739050.800658:0:18535:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e840 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.805691:0:18535:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb7840 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:18535:1874579665500928:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739050.805730:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab0cb lrc: 3/0,0 mode: --/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab0c4 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.805745:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb0:0x0].0x0 (ffffa07433235600) 00010000:00010000:3.0:1787739050.805748:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739050.805752:0:18535:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab0cb lrc: 3/0,0 mode: --/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab0c4 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.805764:0:18535:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab0cb lrc: 3/0,0 mode: --/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab0c4 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.805780:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633de00, rc 0) 00000100:00100000:3.0:1787739050.805850:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665500800:12345-0@lo:101:mdt00_001.0 Request processed in 5358us (6974us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.805860:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2395 00000100:00100000:0.0:1787739050.805947:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564d2c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665500800:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739050.805964:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ab0c4 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab0cb expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.805975:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ab0c4 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab0cb expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739050.806735:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739050.807129:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652400384 00000100:00100000:3.0:1787739050.807138:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652400384 00010000:00010000:3.0:1787739050.807173:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab085 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094981 expref: 14 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.807210:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1345 00000100:00100000:3.0:1787739050.807220:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2407:x1874579652400384:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739050.807235:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739050.807242:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab085 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094981 expref: 14 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.807256:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab085 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094981 expref: 14 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.807278:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab085 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094981 expref: 14 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.807293:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb0:0x0].0x0 (ffffa07433235600) 00010000:00010000:3.0:1787739050.807298:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab0cb lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab0c4 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.807308:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab0cb lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab0c4 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.807318:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab0cb lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab0c4 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.807351:0:6508:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab0cb lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab0c4 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.807366:0:6508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564c780 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665501056:0@lo:105: 00000100:00100000:3.0:1787739050.807409:0:6508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.807508:0:6508:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739050.809044:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665501056 00000100:00100000:1.0:1787739050.809054:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665501056 00000100:00100000:0.0:1787739050.809151:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1025 00000100:00100000:0.0:1787739050.809161:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564da40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665501056:12345-0@lo:105: 00010000:00010000:0.0:1787739050.809173:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ab0c4 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab0cb expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.809182:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ab0c4 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab0cb expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.809188:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ab0c4 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab0cb expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.809194:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ab0c4 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab0cb expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.809199:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ab0c4 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab0cb expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.809205:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ab0c4 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ab0cb expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.809229:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069aa000) 00000100:00100000:0.0:1787739050.809290:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564da40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665501056:12345-0@lo:105: Request processed in 130us (1879us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.809297:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1025 00000100:00100000:2.0:1787739050.809326:0:6508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564c780 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665501056:0@lo:105: 00010000:00010000:2.0:1787739050.809345:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739050.809393:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:2407:x1874579652400384:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 2174us (2670us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.809401:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1345 00010000:00010000:1.0:1787739050.809464:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ab0c4 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab0cb expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.809479:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ab0c4 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab0cb expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739050.810024:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f712580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665501184:0@lo:403:mdt00_001.0 00000100:00100000:1.0:1787739050.810061:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739050.810085:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcca80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.810119:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665501184 00000100:00100000:0.0:1787739050.810126:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665501184 00000100:00100000:0.0:1787739050.810159:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2396 00000100:00100000:0.0:1787739050.810166:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f712d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665501184:12345-0@lo:403:mdt00_001.0 00000100:00100000:0.0:1787739050.812179:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f712d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665501184:12345-0@lo:403:mdt00_001.0 Request processed in 2007us (2110us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.812190:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2396 00000100:00100000:1.0:1787739050.812349:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f712580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665501184:0@lo:403:mdt00_001.0 00000040:00080000:2.0:1787739050.814166:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].321, 664 off490192 00000040:00080000:2.0:1787739050.814335:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].322, 744 off536736 00000004:00080000:2.0:1787739050.814341:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730610ef00 with version 613 00000004:00080000:0.0:1787739050.814407:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 613 rpc_version 613 00000020:00100000:0.0:1787739050.814575:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739050.814586:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07305654f00 x1874579665501440/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1808/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739050.814607:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305654f00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665501440:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739050.814869:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.814900:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d840 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.815218:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665501440 00000100:00100000:3.0:1787739050.815227:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665501440 00000100:00100000:3.0:1787739050.815285:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2397 00000100:00100000:3.0:1787739050.815295:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305656940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665501440:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739050.816112:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305656940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665501440:12345-0@lo:1000:osp_up0-1.0 Request processed in 802us (1227us total) trans 4294983283 rc 0/0 00000100:00100000:3.0:1787739050.816123:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2397 00000100:00100000:0.0:1787739050.816199:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305654f00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665501440:0@lo:1000:osp_up0-1.0 00000004:00200000:2.0:1787739050.816311:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x157:0x0]: returning size 4096 00010000:00010000:2.0:1787739050.816332:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab0bd lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.816347:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab0bd lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.816354:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab0bd lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.816361:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739050.816365:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab0bd lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.816374:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab0bd lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.816389:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb0:0x0].0x0 (ffffa0730608bd00) 00010000:00010000:2.0:1787739050.816392:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739050.816396:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab0bd lrc: 2/0,0 mode: --/EX res: [0x240000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.816401:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab0bd lrc: 0/0,0 mode: --/EX res: [0x240000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.818753:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ab0c4 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab0cb expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.818772:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ab0c4 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab0cb expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.818781:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ab0c4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab0cb expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.818809:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739050.818818:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xb0:0x0] 00010000:00010000:2.0:1787739050.818827:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab0b6 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x157:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.818833:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab0b6 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x157:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.818839:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab0b6 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x157:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.818845:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633ce00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739050.818848:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab0b6 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x157:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.818855:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab0b6 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x157:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.818868:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x157:0x0].0x0 (ffffa0730608b800) 00010000:00010000:2.0:1787739050.818872:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739050.818876:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab0b6 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x157:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.818882:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab0b6 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x157:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.818893:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1ab0a8 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x157:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab0af expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.818900:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1ab0a8 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x157:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab0af expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.818907:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1ab0a8 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x157:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab0af expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.819035:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:2.0:1787739050.819048:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739050.819054:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab08c lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab093 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.819062:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab08c lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab093 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.819069:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab08c lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab093 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.820063:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1787739050.820072:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ab0c4 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab0cb expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.820084:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739050.820089:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ab0c4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab0cb expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739050.820098:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ab0c4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab0cb expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.820155:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ab0c4 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab0cb expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.820162:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739050.820200:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ab0c4 lrc: 1/0,0 mode: --/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab0cb expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.820206:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ab0c4 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab0cb expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.820225:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787739050.820229:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1ab0a8 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x157:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab0af expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.820235:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739050.820240:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1ab0a8 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x157:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab0af expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739050.820247:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1ab0a8 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x157:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab0af expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.820250:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1787739050.820267:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1ab0a8 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x157:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab0af expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739050.820271:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739050.820276:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739050.820281:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab09a lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab0a1 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.820284:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1ab0a8 lrc: 1/0,0 mode: --/EX res: [0x240000402:0x157:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab0af expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.820297:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab09a lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab0a1 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.820299:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1ab0a8 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x157:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab0af expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.820322:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739050.820325:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab08c lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab093 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.820331:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609bc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739050.820334:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab08c lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab093 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739050.820341:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab08c lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab093 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.820403:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17385:x1874579652400128:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 40922us (44546us total) trans 4294968273 rc 0/0 00000100:00100000:2.0:1787739050.820427:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23102 00010000:00010000:3.0:1787739050.820493:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab08c lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab093 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.820505:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739050.820520:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab08c lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab093 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.820527:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab08c lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab093 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.820541:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739050.820543:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739050.820545:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739050.820556:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787739050.820572:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665501824:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739050.820623:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.820896:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665501696:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739050.820923:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.820943:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665501824 00000100:00100000:2.0:1787739050.820950:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665501824 00000100:00100000:2.0:1787739050.820996:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1140 00000100:00100000:2.0:1787739050.821005:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564fc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+10:3651:x1874579665501824:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739050.821018:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739050.821036:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab093 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab08c expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.821050:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730608ba00) 00000100:00100000:1.0:1787739050.821058:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665501568:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739050.821060:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739050.821114:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564fc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+9:3651:x1874579665501824:12345-0@lo:103:ldlm_bl.0 Request processed in 96us (476us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.821122:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1140 00000100:00100000:3.0:1787739050.821329:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665501696 00000100:00100000:3.0:1787739050.821334:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665501696 00000100:00100000:3.0:1787739050.821363:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1141 00000100:00100000:3.0:1787739050.821369:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+9:3653:x1874579665501696:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739050.821377:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739050.821389:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab0af lrc: 0/0,0 mode: --/EX res: [0x240000402:0x157:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab0a8 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.821400:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x157:0x0].0x0 (ffffa0741590d100) 00010000:00010000:3.0:1787739050.821407:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739050.821462:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+8:3653:x1874579665501696:12345-0@lo:103:ldlm_bl.0 Request processed in 93us (537us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.821473:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1141 00000100:00100000:3.0:1787739050.821510:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665501824:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739050.821538:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665501696:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739050.821600:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.821713:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665501568 00000100:00100000:3.0:1787739050.821717:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665501568 00000100:00100000:3.0:1787739050.821781:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1142 00000100:00100000:3.0:1787739050.821789:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb7840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3652:x1874579665501568:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739050.821797:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739050.821808:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab0cb lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab0c4 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.821820:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb0:0x0].0x0 (ffffa07433235600) 00010000:00010000:3.0:1787739050.821826:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739050.821858:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb7840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1874579665501568:12345-0@lo:103:ldlm_bl.0 Request processed in 70us (257us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.821864:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1142 00000100:00100000:2.0:1787739050.821973:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665501568:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739050.822467:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739050.825618:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739050.839941:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739050.840000:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652400512 00000100:00100000:0.0:1787739050.840010:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652400512 00000100:00100000:0.0:1787739050.840072:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23103 00000100:00100000:0.0:1787739050.840082:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17386:x1874579652400512:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739050.840118:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739050.840138:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab0d2 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094988 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.840201:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab0d9 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.840214:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab0d9 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.840223:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab0d9 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.840236:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab0e0 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.840242:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab0e0 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.840248:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab0e0 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.840284:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab0d9 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.840291:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab0d9 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.840298:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab0d9 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.840304:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab0d9 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.840310:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ba00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739050.840314:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab0d9 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.840322:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab0d9 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.840333:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07403a6fd00) 00010000:00010000:0.0:1787739050.840336:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739050.840340:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab0d9 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.840352:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab0d9 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.840363:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab0e0 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.840369:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab0e0 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.840374:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab0e0 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.840379:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739050.840382:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab0e0 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.840388:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab0e0 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.840394:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x425cbd54 (ffffa0741590d400) 00010000:00010000:0.0:1787739050.840396:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739050.840399:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab0e0 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.840404:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab0e0 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.840580:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab0d2 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094988 expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.840592:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07403a6fd00) 00010000:00010000:0.0:1787739050.840595:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739050.840599:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab0d2 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094988 expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.840618:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a000, rc 0) 00000100:00100000:0.0:1787739050.840674:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17386:x1874579652400512:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 594us (741us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739050.840682:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23103 00000100:00100000:0.0:1787739050.848134:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739050.848243:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652400640 00000100:00100000:1.0:1787739050.848252:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652400640 00000100:00100000:1.0:1787739050.848309:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23104 00000100:00100000:1.0:1787739050.848319:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305654000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17386:x1874579652400640:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739050.848337:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739050.848357:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab0e7 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09498f expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.848409:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab0ee lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.848440:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab0ee lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.848452:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab0ee lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739050.848504:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:1.0:1787739050.848509:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739050.848541:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab0e7 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09498f expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.848556:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab0ee lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09498f expref: 14 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.848564:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae000) 00010000:00010000:1.0:1787739050.848567:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739050.848571:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1de00, rc 0) 00000100:00100000:1.0:1787739050.848628:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305654000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17386:x1874579652400640:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 311us (504us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.848644:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23104 00000100:00100000:3.0:1787739050.854920:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739050.855576:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652400896 00000100:00100000:0.0:1787739050.855585:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652400896 00000100:00100000:0.0:1787739050.855694:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23105 00000100:00100000:0.0:1787739050.855706:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17386:x1874579652400896:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:0.0:1787739050.855845:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c382580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665501952:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739050.855911:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.855943:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.855969:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665501952 00000100:00100000:0.0:1787739050.855973:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665501952 00000100:00100000:0.0:1787739050.856012:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2398 00000100:00100000:0.0:1787739050.856019:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665501952:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739050.856137:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665501952:12345-0@lo:1000:mdt00_004.0 Request processed in 111us (220us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.856145:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2398 00000100:00100000:0.0:1787739050.856275:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c382580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665501952:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739050.856328:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665502080:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739050.856360:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.856378:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.856404:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665502080 00000100:00100000:3.0:1787739050.856409:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665502080 00000100:00100000:3.0:1787739050.856457:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2399 00000100:00100000:3.0:1787739050.856465:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c382580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665502080:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739050.856541:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c382580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665502080:12345-0@lo:1000:mdt00_004.0 Request processed in 77us (179us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739050.856548:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2399 00000100:00100000:0.0:1787739050.857469:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665502080:0@lo:1000:mdt00_004.0 00010000:00010000:0.0:1787739050.857525:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ba00/0x7ee4453e4a1ab0f5 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.857536:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ba00/0x7ee4453e4a1ab0f5 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.857546:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ba00/0x7ee4453e4a1ab0f5 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739050.857569:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665502208:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739050.857607:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.857649:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d840 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.857674:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665502208 00000100:00100000:0.0:1787739050.857678:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665502208 00000100:00100000:0.0:1787739050.857731:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2400 00000100:00100000:0.0:1787739050.857741:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665502208:12345-0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739050.857751:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739050.857773:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab0fc lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab0f5 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.857805:0:17834:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab007 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094965 expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.857815:0:17834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab0fc lrc: 4/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab0f5 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.857843:0:17834:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab007 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094965 expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.857864:0:17834:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab007 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094965 expref: 9 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739050.857875:0:17834:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432d680 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:17834:1874579665502336:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739050.857943:0:17834:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.864953:0:17834:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432d680 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:17834:1874579665502336:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739050.864991:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab0fc lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab0f5 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.865005:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07403a6fd00) 00010000:00010000:0.0:1787739050.865008:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739050.865012:0:17834:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab0fc lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab0f5 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.865023:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609be00, rc 0) 00000100:00100000:0.0:1787739050.865091:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:13793:x1874579665502208:12345-0@lo:101:mdt00_004.0 Request processed in 7352us (7482us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.865112:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2400 00000100:00100000:1.0:1787739050.865217:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665502208:0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739050.865236:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ba00/0x7ee4453e4a1ab0f5 lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab0fc expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.865247:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ba00/0x7ee4453e4a1ab0f5 lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab0fc expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.865497:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739050.865601:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652401024 00000100:00100000:0.0:1787739050.865607:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652401024 00010000:00010000:0.0:1787739050.865646:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab007 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094965 expref: 10 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739050.865680:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1346 00000100:00100000:0.0:1787739050.865690:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652401024:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739050.865706:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739050.865712:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab007 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094965 expref: 10 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.865725:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab007 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094965 expref: 10 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.865746:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab007 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094965 expref: 10 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.865757:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07403a6fd00) 00010000:00010000:0.0:1787739050.865768:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab0fc lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab0f5 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.865778:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab0fc lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab0f5 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.865787:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab0fc lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab0f5 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.865821:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab0fc lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab0f5 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739050.865835:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432d680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665502464:0@lo:105: 00000100:00100000:0.0:1787739050.865870:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.865910:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739050.866010:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665502464 00000100:00100000:1.0:1787739050.866015:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665502464 00000100:00100000:1.0:1787739050.866050:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1026 00000100:00100000:1.0:1787739050.866058:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432ed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665502464:12345-0@lo:105: 00010000:00010000:1.0:1787739050.866068:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ba00/0x7ee4453e4a1ab0f5 lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab0fc expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.866077:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ba00/0x7ee4453e4a1ab0f5 lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab0fc expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.866087:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ba00/0x7ee4453e4a1ab0f5 lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab0fc expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.866095:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ba00/0x7ee4453e4a1ab0f5 lrc: 5/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab0fc expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.866112:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ba00/0x7ee4453e4a1ab0f5 lrc: 6/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab0fc expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.866121:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ba00/0x7ee4453e4a1ab0f5 lrc: 6/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ab0fc expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.866148:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609ba00) 00000100:00100000:1.0:1787739050.866184:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432ed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665502464:12345-0@lo:105: Request processed in 127us (313us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.866190:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1026 00000100:00100000:0.0:1787739050.866767:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432d680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665502464:0@lo:105: 00010000:00010000:0.0:1787739050.866779:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739050.866822:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:2408:x1874579652401024:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 1133us (1325us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.866830:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1346 00010000:00010000:3.0:1787739050.867147:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ba00/0x7ee4453e4a1ab0f5 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab0fc expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.867158:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ba00/0x7ee4453e4a1ab0f5 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab0fc expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739050.867180:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739050.867217:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab103 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.867225:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab103 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.867235:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab103 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739050.867254:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564d680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665502592:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787739050.867289:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.867329:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688df00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739050.867969:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665502592 00000100:00100000:0.0:1787739050.867976:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665502592 00000100:00100000:0.0:1787739050.868020:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2401 00000100:00100000:0.0:1787739050.868028:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:13793:x1874579665502592:12345-0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739050.868042:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739050.868057:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab10a lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab103 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.868078:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab10a lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab103 expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.868091:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab10a lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab103 expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.868099:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x425cbd54 (ffffa0741590d400) 00010000:00010000:0.0:1787739050.868113:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739050.868116:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a000, rc 0) 00000100:00100000:0.0:1787739050.870197:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665502592:12345-0@lo:101:mdt00_004.0 Request processed in 2163us (2901us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.870208:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2401 00000100:00100000:1.0:1787739050.875242:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564d680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665502592:0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739050.875268:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab103 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab10a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.875283:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab103 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab10a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739050.875304:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:1.0:1787739050.875355:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f712580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665502720:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739050.875855:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739050.876820:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fccc00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.879000:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665502720 00000100:00100000:2.0:1787739050.879014:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665502720 00000100:00100000:2.0:1787739050.879074:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2402 00000100:00100000:2.0:1787739050.879084:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f711a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665502720:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739050.879218:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f711a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665502720:12345-0@lo:1000:mdt00_004.0 Request processed in 135us (3807us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.879226:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2402 00000100:00100000:3.0:1787739050.879313:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f712580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665502720:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739050.879382:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564d680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665502848:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739050.879453:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.879488:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688df00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.879543:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665502848 00000100:00100000:2.0:1787739050.879548:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665502848 00000100:00100000:2.0:1787739050.879578:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2403 00000100:00100000:2.0:1787739050.879585:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665502848:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739050.879730:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665502848:12345-0@lo:1000:mdt00_004.0 Request processed in 145us (276us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739050.879740:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2403 00000100:00100000:0.0:1787739050.879768:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564d680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665502848:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739050.879840:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432d680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665502976:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739050.879866:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.879889:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4db40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.879966:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665502976 00000100:00100000:2.0:1787739050.879970:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665502976 00000100:00100000:3.0:1787739050.880000:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2404 00000100:00100000:3.0:1787739050.880008:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665502976:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739050.880196:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665502976:12345-0@lo:1000:mdt00_004.0 Request processed in 188us (328us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739050.880235:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432d680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665502976:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739050.881798:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432d680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665503104:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739050.881846:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.881872:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4db40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.882700:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665503104 00000100:00100000:2.0:1787739050.882707:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665503104 00000100:00100000:2.0:1787739050.883098:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2405 00000100:00100000:2.0:1787739050.883119:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564d680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:13793:x1874579665503104:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739050.883210:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564d680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:13793:x1874579665503104:12345-0@lo:1000:mdt00_004.0 Request processed in 93us (1363us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.883218:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2405 00000100:00100000:2.0:1787739050.883267:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432d680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665503104:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739050.883330:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb4b40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665503232:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739050.883360:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.883379:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665503232 00000100:00100000:2.0:1787739050.883383:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665503232 00000100:00100000:2.0:1787739050.883401:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2406 00000100:00100000:2.0:1787739050.883407:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432d680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:13793:x1874579665503232:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739050.884167:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2404 00000100:00100000:2.0:1787739050.884604:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432d680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665503232:12345-0@lo:1000:mdt00_004.0 Request processed in 1195us (1241us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.884612:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2406 00000100:00100000:2.0:1787739050.884635:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ed80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.884653:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb4b40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665503232:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739050.884707:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb4b40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665503360:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739050.884735:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.884752:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ed80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.884769:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665503360 00000100:00100000:2.0:1787739050.884773:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665503360 00000100:00100000:2.0:1787739050.884801:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2407 00000100:00100000:2.0:1787739050.884808:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665503360:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739050.884862:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665503360:12345-0@lo:1000:mdt00_004.0 Request processed in 54us (126us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.884869:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2407 00000100:00100000:2.0:1787739050.884893:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb4b40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665503360:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739050.884924:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561e940 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665503488:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739050.884946:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.884963:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665503488 00000100:00100000:2.0:1787739050.884966:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665503488 00000100:00100000:2.0:1787739050.884984:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2408 00000100:00100000:2.0:1787739050.884990:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665503488:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739050.885027:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665503488:12345-0@lo:1000:mdt00_004.0 Request processed in 37us (81us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.885034:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2408 00000100:00100000:2.0:1787739050.885048:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ed80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.885059:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561e940 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665503488:0@lo:1000:mdt00_004.0 00000040:00080000:2.0:1787739050.887297:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].335, 2168 off516256 00000040:00080000:2.0:1787739050.887387:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].335, 2368 off559424 00000004:00080000:2.0:1787739050.887394:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730610e3c0 with version 639 00000004:00080000:3.0:1787739050.888076:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 639 rpc_version 639 00000020:00100000:3.0:1787739050.888127:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739050.888137:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07302528780 x1874579665503744/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739050.888159:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302528780 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665503744:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739050.888225:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739050.888268:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.889681:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665503744 00000100:00100000:2.0:1787739050.889693:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665503744 00000100:00100000:2.0:1787739050.889752:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2409 00000100:00100000:2.0:1787739050.889761:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665503744:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739050.891786:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:7740:x1874579665503744:12345-0@lo:1000:osp_up1-0.0 Request processed in 2020us (3554us total) trans 4294968274 rc 0/0 00000100:00100000:2.0:1787739050.891800:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2409 00000100:00100000:3.0:1787739050.892083:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302528780 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665503744:0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1787739050.892236:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab111 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x14ed:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.892252:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab111 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14ed:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.892263:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab111 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14ed:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.892284:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab118 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.892292:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab118 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.892299:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab118 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.892331:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab11f lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.892338:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab11f lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.892347:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab11f lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.892365:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561e940 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665503872:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1787739050.892410:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.892517:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739050.894705:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665503872 00000100:00100000:3.0:1787739050.894718:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665503872 00000100:00100000:3.0:1787739050.895014:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2410 00000100:00100000:3.0:1787739050.895026:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304260f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:13793:x1874579665503872:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739050.895047:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739050.895070:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab126 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab11f expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.895099:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab126 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab11f expref: 12 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.895126:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab126 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab11f expref: 12 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739050.895134:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb3:0x0].0x0 (ffffa0730608bf00) 00010000:00010000:3.0:1787739050.895137:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739050.895141:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa400, rc 0) 00000100:00100000:3.0:1787739050.895223:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304260f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:13793:x1874579665503872:12345-0@lo:101:mdt00_004.0 Request processed in 200us (2812us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739050.895232:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2410 00000100:00100000:0.0:1787739050.895265:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561e940 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665503872:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739050.895284:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab11f lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab126 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.895296:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab11f lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab126 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739050.895322:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009a1dd8db reply state 00000000e203567f transno 4294983284 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab118 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.895335:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab118 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.895342:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb3:0x0].0x0 (ffffa07403eeb600) 00010000:00010000:0.0:1787739050.895346:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739050.895349:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739050.895355:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab11f lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab126 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.895360:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab11f lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab126 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739050.895371:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xb3:0x0] 00000004:00010000:0.0:1787739050.895378:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009a1dd8db reply state 00000000e203567f transno 4294983284 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab111 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14ed:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.895387:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab111 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x14ed:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.895403:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14ed:0x0].0x0 (ffffa07403eebf00) 00010000:00010000:0.0:1787739050.895406:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739050.895409:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787739050.895439:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14ed:0x0]: returning size 4096 00000004:00080000:0.0:1787739050.895450:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739050.895461:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ba00/0x7ee4453e4a1ab0f5 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab0fc expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.895469:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ba00/0x7ee4453e4a1ab0f5 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab0fc expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.895476:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ba00/0x7ee4453e4a1ab0f5 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab0fc expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.895498:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739050.895503:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739050.895509:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab103 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab10a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.895515:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab103 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab10a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739050.895590:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17386:x1874579652400896:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 39885us (40676us total) trans 4294983284 rc 0/0 00000100:00100000:0.0:1787739050.895596:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23105 00010000:00010000:1.0:1787739050.895625:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739050.895755:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ba00/0x7ee4453e4a1ab0f5 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab0fc expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.895766:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ba00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739050.895771:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ba00/0x7ee4453e4a1ab0f5 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab0fc expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739050.895780:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ba00/0x7ee4453e4a1ab0f5 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab0fc expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.895810:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ba00/0x7ee4453e4a1ab0f5 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab0fc expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.895818:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739050.895848:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ba00/0x7ee4453e4a1ab0f5 lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab0fc expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.895855:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ba00/0x7ee4453e4a1ab0f5 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab0fc expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.895879:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739050.895882:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739050.895884:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739050.896168:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665504000:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739050.896215:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739050.897383:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665504000 00000100:00100000:1.0:1787739050.897395:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665504000 00000100:00100000:1.0:1787739050.897461:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1143 00000100:00100000:1.0:1787739050.897470:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561e940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+12:3651:x1874579665504000:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739050.897483:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739050.897502:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab0fc lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab0f5 expref: 12 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.897517:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07403a6fd00) 00010000:00010000:1.0:1787739050.897525:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739050.897573:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561e940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+11:3651:x1874579665504000:12345-0@lo:103:ldlm_bl.0 Request processed in 104us (1358us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.897580:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1143 00000100:00100000:1.0:1787739050.897620:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665504000:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739050.940038:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739050.942754:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652401280 00000100:00100000:0.0:1787739050.942769:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652401280 00000100:00100000:0.0:1787739050.942842:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23106 00000100:00100000:0.0:1787739050.942852:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17387:x1874579652401280:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:0.0:1787739050.943035:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17387:x1874579652401280:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 184us (3004us total) trans 0 rc -17/-17 00000100:00100000:0.0:1787739050.943042:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23106 00000100:00100000:3.0:1787739050.971072:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739050.973004:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652401408 00000100:00100000:1.0:1787739050.973021:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652401408 00000100:00100000:1.0:1787739050.973088:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23107 00000100:00100000:1.0:1787739050.973097:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17388:x1874579652401408:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739050.973128:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739050.973152:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab12d lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094996 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.973225:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab134 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.973240:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab134 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.973252:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab134 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.973264:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab13b lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.973275:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab10a lrc: 2/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab103 expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.973285:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab13b lrc: 4/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.973322:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab10a lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ab103 expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739050.973338:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665504128:0@lo:104: 00000100:00100000:1.0:1787739050.973397:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739050.973459:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc180 going to sleep for 11 seconds 00000100:00100000:1.0:1787739050.978441:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665504128 00000100:00100000:1.0:1787739050.978456:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665504128 00000100:00100000:1.0:1787739050.978516:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1027 00000100:00100000:1.0:1787739050.978529:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f712580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6525:x1874579665504128:12345-0@lo:104: 00010000:00010000:1.0:1787739050.978555:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab103 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab10a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.979808:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787739050.979823:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f712580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6525:x1874579665504128:12345-0@lo:104: Request processed in 1294us (6422us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.979835:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1027 00010000:00010000:1.0:1787739050.980023:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739050.980032:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab103 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab10a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.980044:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739050.980052:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab103 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ab10a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739050.980067:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab103 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ab10a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.980126:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab103 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab10a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.980134:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739050.980159:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab103 lrc: 1/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab10a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.980167:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab103 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab10a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.980188:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739050.980190:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739050.980192:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739050.980201:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787739050.980217:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ee1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665504256:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739050.980276:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739050.980318:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665504256 00000100:00100000:1.0:1787739050.980323:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665504256 00010000:00010000:1.0:1787739050.980350:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab10a lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ab103 expref: 11 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739050.980381:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1347 00000100:00100000:1.0:1787739050.980390:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba5450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+11:3653:x1874579665504256:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739050.980403:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1787739050.982080:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665504128:0@lo:104: 00010000:00010000:2.0:1787739050.982127:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab13b lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.985252:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab09a lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab0a1 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739050.985276:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab09a lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab0a1 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.985325:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab09a lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab0a1 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.985333:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739050.985370:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab09a lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab0a1 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787739050.985392:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968274 is committed 00000001:00080000:1.0:1787739050.985428:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968273 is committed 00000020:00080000:1.0:1787739050.985436:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 327 batchid updated 00000020:00080000:1.0:1787739050.985448:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07306248e40 refcount 2 committed 0 result 0 batchid 328 00000020:00080000:1.0:1787739050.985452:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c695700 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07306248180 00000020:00080000:1.0:1787739050.985456:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].321 00000020:00080000:1.0:1787739050.985458:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c695a00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa074080f4000 00000020:00080000:1.0:1787739050.985460:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].320 00010000:00010000:1.0:1787739050.985486:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab02a lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x157:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.985494:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab02a lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x157:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.985501:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab02a lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x157:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.985512:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739050.985518:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab02a lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x157:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.985525:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab02a lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x157:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.985538:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x157:0x0].0x0 (ffffa0730608b800) 00010000:00010000:1.0:1787739050.985540:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739050.985545:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab02a lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x157:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.985551:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab02a lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x157:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.985568:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab031 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.985574:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab031 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.985579:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab031 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.985584:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739050.985587:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab031 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.985592:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab031 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.985603:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb0:0x0].0x0 (ffffa0730608bd00) 00010000:00010000:1.0:1787739050.985605:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739050.985608:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab031 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.985614:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab031 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739050.986647:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 328 committed batchid 327 00010000:00010000:1.0:1787739050.986839:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab10a lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ab103 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.986855:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x425cbd54 (ffffa0741590d400) 00010000:00010000:1.0:1787739050.986860:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab13b lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739050.986869:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730610e840 with version 614 00000020:00080000:2.0:1787739050.986880:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 327 cancel update log [0x200000401:0x3:0x0].320: rc = 0 00000020:00080000:2.0:1787739050.986923:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 327 cancel update log [0x240000400:0x3:0x0].319: rc = 0 00000020:00080000:2.0:1787739050.986932:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 327 00000004:00080000:2.0:1787739050.986942:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 614 rpc_version 614 00010000:00010000:1.0:1787739050.986947:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab13b lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739050.986956:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab13b lrc: 4/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787739050.986972:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00010000:00010000:1.0:1787739050.986981:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739050.986993:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743562f840 x1874579665504640/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739050.987013:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562f840 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665504640:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739050.988609:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787739050.988647:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665504384:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:2.0:1787739050.991172:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.991222:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e240 going to sleep for 11 seconds 00010000:00010000:2.0:1787739050.991268:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab13b lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.991279:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab13b lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739050.991399:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562de00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665504768:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739050.991494:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739050.991510:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739050.991526:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665504640 00000100:00100000:2.0:1787739050.991532:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665504640 00000100:00100000:2.0:1787739050.991571:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665504768 00000100:00100000:2.0:1787739050.991573:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665504768 00000100:00100000:2.0:1787739050.991592:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2411 00000100:00100000:2.0:1787739050.991599:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665504640:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739050.991953:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665504640:12345-0@lo:1000:osp_up0-1.0 Request processed in 353us (786us total) trans 4294983285 rc 0/0 00000100:00100000:2.0:1787739050.991961:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2411 00000100:00100000:2.0:1787739050.992534:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2412 00000100:00100000:2.0:1787739050.992543:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6525:x1874579665504768:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739050.992675:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6525:x1874579665504768:12345-0@lo:1000:mdt00_002.0 Request processed in 63us (1110us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.992683:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2412 00000100:00100000:2.0:1787739050.992724:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562de00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665504768:0@lo:1000:mdt00_002.0 00010000:00010000:2.0:1787739050.993142:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab142 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14ed:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.993158:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab142 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14ed:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.993170:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab142 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14ed:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739050.993191:0:6525:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab142 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14ed:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.993205:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab134 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.993212:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab134 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.993218:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab134 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.993225:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab134 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.993231:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ca00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739050.993235:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab134 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.993243:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab134 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.993255:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305eaec00) 00010000:00010000:2.0:1787739050.993258:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739050.993262:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab134 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.993268:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab134 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.993279:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab13b lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.993285:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab13b lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.993291:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab13b lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.993297:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739050.993299:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab13b lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.993305:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab13b lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.993312:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x425cbd54 (ffffa0741590d400) 00010000:00010000:2.0:1787739050.993314:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739050.993317:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab13b lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.993321:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab13b lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.993355:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab12d lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094996 expref: 8 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.993371:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab142 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14ed:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094996 expref: 7 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739050.993379:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14ed:0x0].0x0 (ffffa07403eeb100) 00010000:00010000:2.0:1787739050.993382:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739050.993384:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c000, rc 0) 00000100:00100000:2.0:1787739050.993444:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17388:x1874579652401408:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 20348us (22381us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739050.993452:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23107 00000100:00100000:2.0:1787739050.993592:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562f840 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665504640:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739050.994614:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba5450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+6:3653:x1874579665504256:12345-0@lo:103:ldlm_bl.0 Request processed in 14220us (14333us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739050.994624:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1347 00000100:00100000:2.0:1787739050.996514:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ee1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665504256:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739050.996706:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739050.999146:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665504384 00000100:00100000:0.0:1787739050.999158:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665504384 00000100:00100000:0.0:1787739050.999216:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1144 00000100:00100000:0.0:1787739050.999225:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+8:3652:x1874579665504384:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:0.0:1787739050.999239:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739050.999260:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab0a1 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xa70cbc1d bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab09a expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.999277:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xa70cbc1d (ffffa0730608b100) 00010000:00010000:0.0:1787739050.999287:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739050.999455:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+7:3652:x1874579665504384:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 231us (2748us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739050.999463:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1144 00000100:00100000:3.0:1787739050.999548:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665504384:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787739050.999631:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739050.999733:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652401536 00000100:00100000:0.0:1787739050.999737:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652401536 00000100:00100000:0.0:1787739050.999820:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23108 00000100:00100000:0.0:1787739050.999827:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ee940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17388:x1874579652401536:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739050.999844:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739050.999858:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab149 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x14ed:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09499d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739050.999989:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab150 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14ed:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.000001:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab150 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14ed:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.000011:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab150 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14ed:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739051.000286:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14ed:0x0]: returning size 4096 00000004:00200000:0.0:1787739051.000292:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739051.000384:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab149 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x14ed:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09499d expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.000401:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab150 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14ed:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09499d expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.000526:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14ed:0x0].0x0 (ffffa07403eebf00) 00010000:00010000:0.0:1787739051.000529:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739051.000532:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a400, rc 0) 00000100:00100000:3.0:1787739051.000726:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ee940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17388:x1874579652401536:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 898us (1094us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.000733:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23108 00000100:00100000:0.0:1787739051.004866:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739051.005018:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652401664 00000100:00100000:1.0:1787739051.005027:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652401664 00000100:00080000:3.0:1787739051.005127:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1787739051.005165:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23109 00000100:00100000:1.0:1787739051.005177:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17388:x1874579652401664:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739051.005198:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739051.005219:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab157 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0949a4 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.005273:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab15e lrc: 3/1,0 mode: --/PR res: [0x200000400:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.005286:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab15e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.005297:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab15e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739051.005346:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xb3:0x0]: returning size 4096 00000004:00200000:1.0:1787739051.005352:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739051.005387:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab157 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949a4 expref: 19 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.005403:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab15e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949a4 expref: 18 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.005441:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb3:0x0].0x0 (ffffa07403eeb600) 00010000:00010000:1.0:1787739051.005444:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739051.005448:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c600, rc 0) 00000100:00100000:1.0:1787739051.005500:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17388:x1874579652401664:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 327us (642us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.005507:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23109 00000100:00100000:3.0:1787739051.009037:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739051.009348:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652401792 00000100:00100000:0.0:1787739051.009358:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652401792 00000100:00100000:0.0:1787739051.009433:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23110 00000100:00100000:0.0:1787739051.009443:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17388:x1874579652401792:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739051.009461:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739051.009482:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ab165 lrc: 2/0,0 mode: --/CR res: [0x240000401:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0949ab expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.009556:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab16c lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.009570:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab126 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab11f expref: 5 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.009579:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab16c lrc: 4/1,0 mode: --/PR res: [0x240000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.009610:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab126 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ab11f expref: 5 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739051.009625:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665504896:0@lo:104: 00000100:00100000:0.0:1787739051.009692:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739051.009725:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4dcc0 going to sleep for 11 seconds 00000100:00080000:0.0:1787739051.009739:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787739051.009767:0:15907:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787739051.009776:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665504896 00000100:00100000:0.0:1787739051.009779:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665504896 00000100:00100000:0.0:1787739051.009803:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1028 00000100:00100000:0.0:1787739051.009809:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306b9a050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6524:x1874579665504896:12345-0@lo:104: 00010000:00010000:0.0:1787739051.009820:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab11f lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab126 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.009861:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787739051.009866:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306b9a050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6524:x1874579665504896:12345-0@lo:104: Request processed in 58us (174us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.010011:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1028 00010000:00010000:0.0:1787739051.010052:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739051.010059:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab11f lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab126 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.010068:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739051.010073:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab11f lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xb3:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ab126 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739051.010082:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab11f lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xb3:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ab126 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.010116:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab11f lrc: 2/0,0 mode: --/PW res: [0x240000401:0xb3:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab126 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.010122:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739051.010251:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab11f lrc: 1/0,0 mode: --/PW res: [0x240000401:0xb3:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab126 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.010260:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab11f lrc: 0/0,0 mode: --/PW res: [0x240000401:0xb3:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab126 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.010278:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739051.010280:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739051.010282:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739051.010292:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665505024:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739051.010317:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739051.010379:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665504896:0@lo:104: 00010000:00010000:0.0:1787739051.010395:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab16c lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739051.010571:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665505024 00000100:00100000:1.0:1787739051.010576:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665505024 00010000:00010000:1.0:1787739051.010594:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab126 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ab11f expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739051.010613:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1348 00000100:00100000:1.0:1787739051.010618:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432ed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+6:3650:x1874579665505024:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739051.010626:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00080000:1.0:1787739051.012275:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 328 batchid updated 00000020:00080000:2.0:1787739051.012333:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 328 00000004:00080000:2.0:1787739051.012384:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0743c0ef300 with version 615 00000020:00080000:2.0:1787739051.012402:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 328 cancel update log [0x200000401:0x3:0x0].321: rc = 0 00000004:00080000:0.0:1787739051.012476:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 615 rpc_version 615 00000020:00080000:2.0:1787739051.012488:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 328 cancel update log [0x240000400:0x3:0x0].320: rc = 0 00000020:00080000:2.0:1787739051.012497:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 328 00000020:00100000:0.0:1787739051.012502:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739051.012511:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306004f00 x1874579665505280/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739051.012531:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004f00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665505280:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739051.012582:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739051.012612:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4dcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739051.012636:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665505280 00000100:00100000:2.0:1787739051.012642:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665505280 00000100:00100000:2.0:1787739051.012678:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2413 00000100:00100000:2.0:1787739051.012685:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665505280:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739051.012797:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665505280:12345-0@lo:1000:osp_up0-1.0 Request processed in 112us (215us total) trans 4294983286 rc 0/0 00000100:00100000:2.0:1787739051.012804:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2413 00010000:00010000:3.0:1787739051.013150:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab126 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ab11f expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.013164:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb3:0x0].0x0 (ffffa0730608bf00) 00010000:00010000:3.0:1787739051.013169:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab16c lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.013178:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab16c lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.013185:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab16c lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.013213:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739051.013258:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432ed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+5:3650:x1874579665505024:12345-0@lo:103:ldlm_bl.0 Request processed in 2639us (2940us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.013270:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1348 00010000:00010000:2.0:1787739051.013337:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab16c lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.013347:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab16c lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739051.013391:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xb3:0x0]: returning size 4096 00000004:00200000:2.0:1787739051.013396:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739051.013449:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ab165 lrc: 0/0,0 mode: --/CR res: [0x240000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ab expref: 9 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.013464:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab16c lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ab expref: 8 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.013477:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb3:0x0].0x0 (ffffa0730608bf00) 00010000:00010000:2.0:1787739051.013480:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739051.013482:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609be00, rc 0) 00000100:00100000:2.0:1787739051.013519:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17388:x1874579652401792:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 4078us (4493us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.013525:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23110 00000100:00100000:0.0:1787739051.014390:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665505024:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739051.016193:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004f00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665505280:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739051.023487:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739051.023525:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739051.023735:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652402176 00000100:00100000:3.0:1787739051.023797:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652402176 00000100:00100000:3.0:1787739051.023889:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23111 00000100:00100000:3.0:1787739051.023899:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17388:x1874579652402176:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:3.0:1787739051.023953:0:13793:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739051.023980:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab150 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14ed:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09499d expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.023999:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14ed:0x0].0x0 (ffffa07403eebf00) 00010000:00010000:3.0:1787739051.024005:0:13793:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739051.024053:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ab173 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.024061:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ab173 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.024070:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ab173 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739051.024093:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665505408:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787739051.024151:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.024193:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688df00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.024210:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652401920 00000100:00100000:3.0:1787739051.024213:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652401920 00000100:00100000:3.0:1787739051.024242:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1145 00000100:00100000:3.0:1787739051.024248:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562de00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2409:x1874579652401920:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:3.0:1787739051.024257:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739051.024269:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab142 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14ed:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094996 expref: 8 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.024282:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14ed:0x0].0x0 (ffffa07403eeb100) 00010000:00010000:3.0:1787739051.024290:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739051.024336:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562de00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2409:x1874579652401920:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 77us (847us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.024344:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1145 00000100:00100000:2.0:1787739051.024453:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665505408 00000100:00100000:2.0:1787739051.024457:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665505408 00000100:00100000:2.0:1787739051.024490:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2414 00000100:00100000:2.0:1787739051.024497:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+5:13793:x1874579665505408:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739051.024507:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739051.024520:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab17a lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab173 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.024543:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab17a lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab173 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.024555:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab17a lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab173 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.024561:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07403eeb400) 00010000:00010000:2.0:1787739051.024563:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739051.024566:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633de00, rc 0) 00000100:00100000:2.0:1787739051.024599:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:13793:x1874579665505408:12345-0@lo:101:mdt00_004.0 Request processed in 103us (448us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.024605:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2414 00000100:00100000:2.0:1787739051.024634:0:13793:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968274 gen 2 00000004:00080000:2.0:1787739051.024639:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968269 committed transno 4294968274 00000020:00080000:2.0:1787739051.024645:0:13793:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07306248480 refcount 2 committed 0 result 0 batchid 337 00000020:00080000:2.0:1787739051.024649:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c695000 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07306248a80 00000020:00080000:2.0:1787739051.024652:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].333 00000020:00080000:2.0:1787739051.024654:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c695f00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073027dba00 00000020:00080000:2.0:1787739051.024656:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].334 00000004:00080000:2.0:1787739051.024676:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968271 committed transno 4294968274 00000004:00080000:2.0:1787739051.024685:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968272 committed transno 4294968274 00000004:00080000:2.0:1787739051.024693:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968274 committed transno 4294968274 00000100:00100000:2.0:1787739051.024707:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665505408:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739051.024719:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ab173 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab17a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.024729:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ab173 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab17a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739051.024743:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:2.0:1787739051.024762:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1ab181 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.024769:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1ab181 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.024777:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1ab181 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.024792:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665505536:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1787739051.024816:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739051.024831:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0eff00 going to sleep for 11 seconds 00000020:00080000:2.0:1787739051.024841:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 337 committed batchid 336 00000020:00080000:2.0:1787739051.024898:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 336 00000100:00100000:2.0:1787739051.024908:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665505536 00000100:00100000:2.0:1787739051.024912:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665505536 00000100:00100000:2.0:1787739051.024937:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2415 00000100:00100000:2.0:1787739051.024944:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:13793:x1874579665505536:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739051.024952:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739051.024963:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab188 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab181 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.024977:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab188 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab181 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.024988:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab188 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab181 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.024995:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x425cbd54 (ffffa07403eeb700) 00010000:00010000:2.0:1787739051.024997:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739051.024999:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633dc00, rc 0) 00000100:00100000:2.0:1787739051.025028:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665505536:12345-0@lo:101:mdt00_004.0 Request processed in 84us (210us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.025034:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2415 00000100:00100000:2.0:1787739051.025061:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665505536:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739051.025072:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1ab181 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab188 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.025080:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1ab181 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab188 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739051.025091:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:2.0:1787739051.025131:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665505664:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739051.025156:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739051.025177:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.025386:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665505664 00000100:00100000:3.0:1787739051.025390:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665505664 00000100:00100000:3.0:1787739051.025439:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2416 00000100:00100000:3.0:1787739051.025446:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305657480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665505664:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739051.025503:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305657480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665505664:12345-0@lo:1000:mdt00_004.0 Request processed in 58us (347us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.025510:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2416 00000100:00100000:2.0:1787739051.026045:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665505664:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739051.026084:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665505792:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739051.026122:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739051.026149:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.026285:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665505792 00000100:00100000:3.0:1787739051.026290:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665505792 00000100:00100000:3.0:1787739051.026314:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2417 00000100:00100000:3.0:1787739051.026319:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7121c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665505792:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739051.026386:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7121c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665505792:12345-0@lo:1000:mdt00_004.0 Request processed in 66us (264us total) trans 0 rc 1/1 00000100:00100000:3.0:1787739051.026393:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2417 00000100:00100000:2.0:1787739051.026478:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665505792:0@lo:1000:mdt00_004.0 00010000:00010000:2.0:1787739051.026521:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ab18f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.026528:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ab18f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.026535:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ab18f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.026545:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f713c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665505920:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1787739051.026565:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739051.026588:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.026625:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665505920 00000100:00100000:3.0:1787739051.026629:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665505920 00000100:00100000:3.0:1787739051.026654:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2418 00000100:00100000:3.0:1787739051.026660:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f712580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665505920:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739051.026668:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739051.026682:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab196 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab18f expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.026699:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab196 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab18f expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.026711:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab196 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab18f expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.026718:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14ed:0x0].0x0 (ffffa0730608b300) 00010000:00010000:3.0:1787739051.026721:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739051.026723:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa400, rc 0) 00000100:00100000:3.0:1787739051.026765:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f712580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665505920:12345-0@lo:101:mdt00_004.0 Request processed in 105us (199us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.026771:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2418 00000100:00100000:1.0:1787739051.026822:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f713c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665505920:0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739051.026833:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ab18f lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab196 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.026844:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ab18f lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab196 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.026863:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab19d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.026937:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab19d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.026947:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab19d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.026964:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab1a4 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.026975:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab15e lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949a4 expref: 17 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.026983:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab1a4 lrc: 4/0,1 mode: --/EX res: [0x200000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.027006:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab15e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949a4 expref: 17 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.027019:0:13793:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab15e lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949a4 expref: 17 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739051.027029:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ec000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665506048:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739051.027352:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.036133:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ec000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665506048:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739051.036171:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab1a4 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739051.044385:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739051.044775:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652402304 00000100:00100000:0.0:1787739051.044785:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652402304 00010000:00010000:0.0:1787739051.044825:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab15e lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949a4 expref: 18 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739051.044866:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1349 00000100:00100000:0.0:1787739051.044887:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2409:x1874579652402304:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739051.044902:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739051.044909:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab15e lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949a4 expref: 18 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.044923:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab15e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949a4 expref: 18 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.044947:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab15e lrc: 0/0,0 mode: --/PR res: [0x200000400:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949a4 expref: 18 pid: 7944 timeout: 737 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.044961:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb3:0x0].0x0 (ffffa07403eeb600) 00010000:00010000:0.0:1787739051.044965:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab1a4 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.044975:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab1a4 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.044983:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab1a4 lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.045012:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739051.045065:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652402304:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 181us (688us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.045073:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1349 00010000:00010000:3.0:1787739051.045174:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab1a4 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.045182:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab1a4 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.045225:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab1ab lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.045233:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab1ab lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.045242:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab1ab lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739051.045260:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41e580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665506176:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787739051.045299:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.045338:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688df00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739051.045389:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665506176 00000100:00100000:2.0:1787739051.045394:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665506176 00000100:00100000:2.0:1787739051.045443:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2419 00000100:00100000:2.0:1787739051.045450:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665506176:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739051.045465:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739051.045477:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ab1b2 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab1ab expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.045498:0:17834:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab16c lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ab expref: 6 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.045507:0:17834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ab1b2 lrc: 4/0,0 mode: --/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab1ab expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.045529:0:17834:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab16c lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ab expref: 6 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.045541:0:17834:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab16c lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ab expref: 6 pid: 6524 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.045551:0:17834:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710780 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:17834:1874579665506304:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739051.045596:0:17834:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.052300:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739051.052378:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652402432 00000100:00100000:1.0:1787739051.052386:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652402432 00010000:00010000:1.0:1787739051.052447:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab16c lrc: 5/0,0 mode: PR/PR res: [0x240000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ab expref: 7 pid: 6524 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739051.052479:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1350 00000100:00100000:1.0:1787739051.052488:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652402432:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739051.052502:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739051.052508:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab16c lrc: 5/0,0 mode: PR/PR res: [0x240000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ab expref: 7 pid: 6524 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.052521:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab16c lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ab expref: 7 pid: 6524 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.052542:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb3:0x0].0x0 (ffffa0730608bf00) 00010000:00010000:1.0:1787739051.052549:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ab1b2 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab1ab expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.052564:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ab1b2 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab1ab expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.052573:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ab1b2 lrc: 4/0,0 mode: EX/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab1ab expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.052603:0:6508:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ab1b2 lrc: 4/0,0 mode: EX/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab1ab expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739051.052615:0:6508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001_UUID:6508:1874579665506432:0@lo:105: 00000100:00100000:1.0:1787739051.052669:0:6508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.052704:0:6508:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc780 going to sleep for 11 seconds 00000100:00100000:0.0:1787739051.052852:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665506432 00000100:00100000:0.0:1787739051.052857:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665506432 00000100:00100000:0.0:1787739051.052900:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1029 00000100:00100000:0.0:1787739051.052907:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156eda40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6508:x1874579665506432:12345-0@lo:105: 00010000:00010000:0.0:1787739051.052917:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab1ab lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab1b2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.052926:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab1ab lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab1b2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.052934:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab1ab lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab1b2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.052942:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab1ab lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab1b2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.052949:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab1ab lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab1b2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.052957:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab1ab lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ab1b2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.052969:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069ab000) 00010000:00010000:2.0:1787739051.052983:0:17834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab16c lrc: 0/0,0 mode: --/PR res: [0x240000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ab expref: 7 pid: 6524 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.053000:0:17834:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710780 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:17834:1874579665506304:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739051.053000:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156eda40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6508:x1874579665506432:12345-0@lo:105: Request processed in 94us (330us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.053007:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1029 00010000:00010000:2.0:1787739051.053022:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ab1b2 lrc: 4/0,0 mode: EX/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab1ab expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.053036:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb3:0x0].0x0 (ffffa0730608bf00) 00010000:00010000:2.0:1787739051.053039:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739051.053041:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d400, rc 0) 00000100:00100000:2.0:1787739051.053090:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:13793:x1874579665506176:12345-0@lo:101:mdt00_004.0 Request processed in 7641us (7791us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.053098:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2419 00000100:00100000:0.0:1787739051.053248:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41e580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665506176:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739051.053262:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab1ab lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab1b2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739051.053955:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665506560:0@lo:403:mdt00_004.0 00000100:00100000:1.0:1787739051.053977:0:6508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001_UUID:6508:1874579665506432:0@lo:105: 00010000:00010000:1.0:1787739051.053989:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739051.055265:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2408:x1874579652402432:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 2773us (2967us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.055278:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1350 00000100:00100000:0.0:1787739051.055488:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739051.055607:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4dcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.056512:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665506560 00000100:00100000:3.0:1787739051.056521:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665506560 00000100:00100000:0.0:1787739051.056713:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2420 00000100:00100000:0.0:1787739051.056722:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a31a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:13793:x1874579665506560:12345-0@lo:403:mdt00_004.0 00000100:00100000:2.0:1787739051.059040:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a31a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:13793:x1874579665506560:12345-0@lo:403:mdt00_004.0 Request processed in 2312us (3544us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.059052:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2420 00000100:00100000:3.0:1787739051.059159:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665506560:0@lo:403:mdt00_004.0 00000040:00080000:0.0:1787739051.060255:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].336, 664 off516920 00000040:00080000:0.0:1787739051.062841:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].336, 744 off560168 00000004:00080000:0.0:1787739051.062858:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07410b4d0c0 with version 640 00000004:00080000:3.0:1787739051.062988:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 640 rpc_version 640 00000020:00100000:3.0:1787739051.063027:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739051.063037:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743c41e580 x1874579665506816/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2064/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739051.063060:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41e580 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665506816:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739051.063136:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.063182:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739051.063196:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665506816 00000100:00100000:2.0:1787739051.063204:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665506816 00000100:00100000:2.0:1787739051.063255:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2421 00000100:00100000:2.0:1787739051.063265:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304000b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665506816:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739051.063687:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304000b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665506816:12345-0@lo:1000:osp_up1-0.0 Request processed in 422us (549us total) trans 4294968275 rc 0/0 00000100:00100000:2.0:1787739051.063696:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2421 00000100:00100000:3.0:1787739051.064036:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41e580 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665506816:0@lo:1000:osp_up1-0.0 00000004:00200000:0.0:1787739051.064530:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14ed:0x0]: returning size 4096 00010000:00010000:0.0:1787739051.064553:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab1a4 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.064563:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab1a4 lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.064570:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab1a4 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.064577:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739051.064582:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab1a4 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.064589:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab1a4 lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.064604:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb3:0x0].0x0 (ffffa07403eeb600) 00010000:00010000:0.0:1787739051.064607:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739051.064612:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab1a4 lrc: 2/0,0 mode: --/EX res: [0x200000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.064619:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab1a4 lrc: 0/0,0 mode: --/EX res: [0x200000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.064633:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab1ab lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab1b2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.064642:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab1ab lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab1b2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.064649:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab1ab lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab1b2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.064673:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739051.064681:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xb3:0x0] 00010000:00010000:0.0:1787739051.064688:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab19d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.064695:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab19d lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.064701:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab19d lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.064706:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739051.064709:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab19d lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.064715:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab19d lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.064729:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14ed:0x0].0x0 (ffffa07403eebf00) 00010000:00010000:0.0:1787739051.064731:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739051.064734:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab19d lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.064740:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab19d lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.064811:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739051.064818:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab1ab lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab1b2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.064824:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739051.064829:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab1ab lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab1b2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739051.064836:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab1ab lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab1b2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.064981:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ab18f lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab196 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.064990:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ab18f lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab196 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.064997:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ab18f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab196 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.065050:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab1ab lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab1b2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.065058:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:3.0:1787739051.065132:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00010000:00010000:1.0:1787739051.065137:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739051.065149:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:2.0:1787739051.065152:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab1ab lrc: 1/0,0 mode: --/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab1b2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.065155:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ab173 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab17a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.065162:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ab173 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab17a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.065169:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab1ab lrc: 0/0,0 mode: --/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab1b2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.065170:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ab173 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab17a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.065182:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:1.0:1787739051.065186:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739051.065190:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1ab181 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab188 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.065198:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1ab181 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab188 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.065243:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1787739051.065247:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ab18f lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab196 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.065253:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739051.065256:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ab18f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab196 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739051.065263:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ab18f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab196 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739051.065273:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17388:x1874579652402176:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 41373us (41744us total) trans 4294983287 rc 0/0 00010000:00010000:2.0:1787739051.065279:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ab18f lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab196 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739051.065282:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23111 00010000:00010000:2.0:1787739051.065285:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739051.065300:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ab18f lrc: 1/0,0 mode: --/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab196 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.065312:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ab18f lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab196 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.065321:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739051.065324:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739051.065325:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ab173 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab17a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.065326:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739051.065327:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739051.065332:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa600 already unused, calling callback (ffffffffc18e6ee0) 00000100:00100000:2.0:1787739051.065333:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00010000:00010000:1.0:1787739051.065336:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ab173 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab17a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739051.065342:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ab173 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab17a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.065344:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665507072:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739051.065359:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ab173 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab17a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.065365:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739051.065370:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ab173 lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab17a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.065377:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ab173 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab17a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.065388:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787739051.065388:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739051.065389:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739051.065391:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739051.065397:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739051.065452:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665507200:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739051.065478:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739051.065509:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665507072 00000100:00100000:2.0:1787739051.065515:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665507072 00000100:00100000:2.0:1787739051.065557:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665507200 00000100:00100000:2.0:1787739051.065561:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665507200 00000100:00100000:2.0:1787739051.065582:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1146 00000100:00100000:2.0:1787739051.065590:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+11:3653:x1874579665507072:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739051.065601:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739051.065616:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab196 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14ed:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab18f expref: 11 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.065626:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14ed:0x0].0x0 (ffffa0730608b300) 00010000:00010000:2.0:1787739051.065633:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739051.065661:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+10:3653:x1874579665507072:12345-0@lo:103:ldlm_bl.0 Request processed in 71us (272us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.065667:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1146 00000100:00100000:2.0:1787739051.065680:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1147 00000100:00100000:2.0:1787739051.065686:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba4850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+9:3653:x1874579665507200:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739051.065694:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739051.065704:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab17a lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab173 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.065714:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07403eeb400) 00010000:00010000:2.0:1787739051.065720:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739051.065743:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba4850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+8:3653:x1874579665507200:12345-0@lo:103:ldlm_bl.0 Request processed in 57us (264us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.066264:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1147 00000100:00100000:2.0:1787739051.066322:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665507200:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739051.066332:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665507072:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739051.066504:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665506944:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739051.066533:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.066707:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665506944 00000100:00100000:1.0:1787739051.066712:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665506944 00000100:00100000:1.0:1787739051.067753:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1148 00000100:00100000:1.0:1787739051.067766:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073040003c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+8:3650:x1874579665506944:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739051.067780:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739051.067800:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ab1b2 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab1ab expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.067816:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb3:0x0].0x0 (ffffa0730608bf00) 00010000:00010000:1.0:1787739051.067825:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739051.067872:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073040003c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+7:3650:x1874579665506944:12345-0@lo:103:ldlm_bl.0 Request processed in 110us (1338us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.067879:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1148 00000100:00100000:3.0:1787739051.068073:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665506944:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739051.087513:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739051.087550:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652402560 00000100:00100000:3.0:1787739051.087558:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652402560 00000100:00100000:3.0:1787739051.087605:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23112 00000100:00100000:3.0:1787739051.087614:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304000780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17389:x1874579652402560:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739051.087648:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739051.087668:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab1b9 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0949b2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.087727:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab1c0 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.087740:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab1c0 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.087960:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab1c0 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.087976:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab1c7 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.087985:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab1c7 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.087991:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab1c7 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.088032:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab1c0 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.088040:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab1c0 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.088047:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab1c0 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.088057:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab1c0 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.088064:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739051.088068:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab1c0 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.088076:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab1c0 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.088088:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae000) 00010000:00010000:3.0:1787739051.088091:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739051.088095:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab1c0 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.088115:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab1c0 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.088125:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab1c7 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.088130:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab1c7 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.088135:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab1c7 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.088139:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739051.088141:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab1c7 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.088150:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab1c7 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.088156:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xb98062fb (ffffa0730608b900) 00010000:00010000:3.0:1787739051.088225:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739051.088229:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab1c7 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.088235:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab1c7 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.088266:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab1b9 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949b2 expref: 18 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.088277:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae000) 00010000:00010000:3.0:1787739051.088279:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739051.088283:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab1b9 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949b2 expref: 18 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.088292:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aae00, rc 0) 00000100:00100000:3.0:1787739051.088467:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304000780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17389:x1874579652402560:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 852us (960us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739051.088475:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23112 00000100:00100000:0.0:1787739051.098465:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739051.099062:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652402688 00000100:00100000:3.0:1787739051.099072:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652402688 00000100:00100000:3.0:1787739051.099139:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23113 00000100:00100000:3.0:1787739051.099149:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17389:x1874579652402688:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739051.099172:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739051.099193:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab1ce lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0949b9 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.099250:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab1d5 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.099264:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab1d5 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.099276:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab1d5 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739051.099322:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x5:0x0]: returning size 4096 00000004:00200000:3.0:1787739051.099326:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739051.099361:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab1ce lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949b9 expref: 8 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.099375:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab1d5 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949b9 expref: 7 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.099383:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b900) 00010000:00010000:3.0:1787739051.099386:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739051.099389:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab000, rc 0) 00000100:00100000:3.0:1787739051.099463:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17389:x1874579652402688:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 316us (1008us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.099471:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23113 00000100:00100000:0.0:1787739051.123446:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739051.123622:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652402944 00000100:00100000:1.0:1787739051.123696:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652402944 00000100:00100000:1.0:1787739051.124012:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23114 00000100:00100000:1.0:1787739051.124024:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17389:x1874579652402944:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:1.0:1787739051.124143:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665507328:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739051.125332:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.125393:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739051.125483:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665507328 00000100:00100000:0.0:1787739051.125491:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665507328 00000100:00100000:0.0:1787739051.125620:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2422 00000100:00100000:0.0:1787739051.125630:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665507328:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739051.126037:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665507328:12345-0@lo:1000:mdt00_004.0 Request processed in 409us (707us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.126046:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2422 00000100:00100000:0.0:1787739051.126093:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665507328:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739051.127095:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a31e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665507456:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739051.127151:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739051.127741:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730638b9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739051.128257:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665507456 00000100:00100000:1.0:1787739051.128265:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665507456 00000100:00100000:1.0:1787739051.128315:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2423 00000100:00100000:1.0:1787739051.128323:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665507456:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739051.128440:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665507456:12345-0@lo:1000:mdt00_004.0 Request processed in 118us (1288us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739051.128448:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2423 00000100:00100000:0.0:1787739051.129986:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a31e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665507456:0@lo:1000:mdt00_004.0 00010000:00010000:0.0:1787739051.130058:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab1dc lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.130068:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab1dc lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.130079:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab1dc lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739051.130097:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a31e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665507584:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739051.130155:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739051.132003:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730638b000 going to sleep for 11 seconds 00000100:00100000:1.0:1787739051.132687:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665507584 00000100:00100000:1.0:1787739051.132697:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665507584 00000100:00100000:1.0:1787739051.132760:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2424 00000100:00100000:1.0:1787739051.132770:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665507584:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739051.132787:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739051.132806:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab1e3 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab1dc expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.132834:0:8448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab0ee lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09498f expref: 16 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.132845:0:8448:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab1e3 lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab1dc expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.133077:0:8448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab0ee lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09498f expref: 16 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.133094:0:8448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab0ee lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09498f expref: 16 pid: 7944 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739051.133122:0:8448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8448:1874579665507712:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739051.133191:0:8448:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc000 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.147721:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739051.147791:0:8448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8448:1874579665507712:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739051.147824:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab1e3 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab1dc expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.147838:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae000) 00010000:00010000:3.0:1787739051.147841:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739051.147845:0:8448:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab1e3 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab1dc expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.147854:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c000, rc 0) 00000100:00100000:3.0:1787739051.147916:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665507584:12345-0@lo:101:mdt00_004.0 Request processed in 15147us (17759us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.147925:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2424 00000100:00100000:3.0:1787739051.147984:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a31e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665507584:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739051.148000:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab1dc lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab1e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.148009:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab1dc lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab1e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739051.148322:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652403072 00000100:00100000:1.0:1787739051.148329:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652403072 00010000:00010000:1.0:1787739051.148358:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab0ee lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09498f expref: 17 pid: 7944 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739051.148393:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1351 00000100:00100000:1.0:1787739051.148402:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fde00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652403072:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739051.148440:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739051.148447:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab0ee lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09498f expref: 17 pid: 7944 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.148459:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab0ee lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09498f expref: 17 pid: 7944 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.148480:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab0ee lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09498f expref: 17 pid: 7944 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.148492:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae000) 00010000:00010000:1.0:1787739051.148497:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab1e3 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab1dc expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.148506:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab1e3 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab1dc expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.148519:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab1e3 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab1dc expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.148698:0:6509:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab1e3 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab1dc expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739051.148711:0:6509:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665507840:0@lo:105: 00000100:00100000:1.0:1787739051.148760:0:6509:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.148797:0:6509:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc600 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.148886:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665507840 00000100:00100000:3.0:1787739051.148899:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665507840 00000100:00100000:3.0:1787739051.148936:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1030 00000100:00100000:3.0:1787739051.148943:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ec780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665507840:12345-0@lo:105: 00010000:00010000:3.0:1787739051.148954:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab1dc lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab1e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.148962:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab1dc lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab1e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.148971:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab1dc lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab1e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.148979:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab1dc lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab1e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.148985:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab1dc lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab1e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.148993:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab1dc lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ab1e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.149019:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609bc00) 00000100:00100000:3.0:1787739051.149057:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ec780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665507840:12345-0@lo:105: Request processed in 115us (297us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.149063:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1030 00000100:00100000:3.0:1787739051.149091:0:6509:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665507840:0@lo:105: 00010000:00010000:3.0:1787739051.149111:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739051.149144:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fde00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:2409:x1874579652403072:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 743us (1433us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.149151:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1351 00010000:00010000:1.0:1787739051.150275:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab1dc lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab1e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.150288:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab1dc lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab1e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739051.150311:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739051.150349:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ab1ea lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.150356:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ab1ea lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.150366:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ab1ea lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739051.150384:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665507968:0@lo:101:mdt00_004.0 00000100:00100000:1.0:1787739051.150450:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.150477:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.151578:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665507968 00000100:00100000:3.0:1787739051.151586:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665507968 00000100:00100000:3.0:1787739051.151644:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2425 00000100:00100000:3.0:1787739051.151654:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665507968:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739051.151670:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739051.151688:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab1f1 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab1ea expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.151713:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab1f1 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab1ea expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.151726:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab1f1 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab1ea expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.151735:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xb98062fb (ffffa0730608b700) 00010000:00010000:3.0:1787739051.151738:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739051.151741:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aae00, rc 0) 00000100:00100000:3.0:1787739051.151815:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665507968:12345-0@lo:101:mdt00_004.0 Request processed in 163us (1364us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.151821:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2425 00000100:00100000:1.0:1787739051.152096:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665507968:0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739051.152123:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ab1ea lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab1f1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.152134:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ab1ea lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab1f1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739051.152154:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:1.0:1787739051.152195:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665508096:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739051.152226:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.152254:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.152809:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665508096 00000100:00100000:3.0:1787739051.152815:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665508096 00000100:00100000:3.0:1787739051.152897:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2426 00000100:00100000:3.0:1787739051.152905:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ecf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665508096:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739051.152978:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ecf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665508096:12345-0@lo:1000:mdt00_004.0 Request processed in 74us (751us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.152985:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2426 00000100:00100000:3.0:1787739051.156188:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665508096:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739051.156264:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061ff0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665508224:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739051.156320:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.156406:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d540 going to sleep for 11 seconds 00000100:00100000:2.0:1787739051.157664:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665508224 00000100:00100000:2.0:1787739051.157674:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665508224 00000100:00100000:2.0:1787739051.157721:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2427 00000100:00100000:2.0:1787739051.157730:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fc780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665508224:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739051.157851:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fc780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665508224:12345-0@lo:1000:mdt00_004.0 Request processed in 122us (1530us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739051.157877:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2427 00000100:00100000:3.0:1787739051.158677:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061ff0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665508224:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739051.158775:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061ff0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665508352:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739051.158817:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.158846:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.158879:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665508352 00000100:00100000:3.0:1787739051.158883:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665508352 00000100:00100000:3.0:1787739051.158915:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2428 00000100:00100000:3.0:1787739051.158922:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665508352:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739051.158994:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665508352:12345-0@lo:1000:mdt00_004.0 Request processed in 72us (177us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739051.159000:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2428 00000100:00100000:3.0:1787739051.159025:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061ff0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665508352:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739051.159064:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061ff0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665508480:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739051.159086:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.159114:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.159126:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665508480 00000100:00100000:3.0:1787739051.159130:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665508480 00000100:00100000:3.0:1787739051.159150:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2429 00000100:00100000:3.0:1787739051.159156:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665508480:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739051.159202:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665508480:12345-0@lo:1000:mdt00_004.0 Request processed in 45us (114us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.159208:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2429 00000100:00100000:3.0:1787739051.159232:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061ff0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665508480:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739051.159267:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061ff0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665508608:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739051.159288:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.159303:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.159315:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665508608 00000100:00100000:3.0:1787739051.159318:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665508608 00000100:00100000:3.0:1787739051.159337:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2430 00000100:00100000:3.0:1787739051.159343:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fd680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665508608:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739051.159398:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fd680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665508608:12345-0@lo:1000:mdt00_004.0 Request processed in 54us (108us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.159403:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2430 00000100:00100000:3.0:1787739051.159444:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061ff0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665508608:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739051.159478:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061ff0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665508736:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739051.159500:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.159516:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.159527:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665508736 00000100:00100000:3.0:1787739051.159530:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665508736 00000100:00100000:3.0:1787739051.159563:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2431 00000100:00100000:3.0:1787739051.159569:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fd2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665508736:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739051.159613:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fd2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665508736:12345-0@lo:1000:mdt00_004.0 Request processed in 45us (112us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.159620:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2431 00000100:00100000:3.0:1787739051.159641:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061ff0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665508736:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739051.159668:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061ff0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665508864:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739051.159688:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.159701:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.159711:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665508864 00000100:00100000:3.0:1787739051.159713:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665508864 00000100:00100000:3.0:1787739051.159731:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2432 00000100:00100000:3.0:1787739051.159737:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fcb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665508864:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739051.159790:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fcb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665508864:12345-0@lo:1000:mdt00_004.0 Request processed in 52us (101us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.159796:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2432 00000100:00100000:3.0:1787739051.159821:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061ff0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665508864:0@lo:1000:mdt00_004.0 00000040:00080000:3.0:1787739051.160640:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].323, 2168 off493688 00000040:00080000:3.0:1787739051.160713:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].323, 2368 off539104 00000004:00080000:3.0:1787739051.160719:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730688d6c0 with version 616 00000004:00080000:0.0:1787739051.163042:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 616 rpc_version 616 00000020:00100000:0.0:1787739051.163147:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739051.163158:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303a31e00 x1874579665509120/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739051.163181:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a31e00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665509120:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739051.163247:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739051.163283:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730638b840 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.163495:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665509120 00000100:00100000:3.0:1787739051.163503:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665509120 00000100:00100000:0.0:1787739051.164128:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2433 00000100:00100000:0.0:1787739051.164140:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c21c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665509120:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739051.165096:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c21c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:7467:x1874579665509120:12345-0@lo:1000:osp_up0-1.0 Request processed in 958us (1846us total) trans 4294983288 rc 0/0 00000100:00100000:0.0:1787739051.165122:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2433 00000100:00100000:0.0:1787739051.165194:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a31e00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665509120:0@lo:1000:osp_up0-1.0 00010000:00010000:3.0:1787739051.165726:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab1f8 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x159:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.165741:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab1f8 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x159:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.165753:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab1f8 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x159:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.165771:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab1ff lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.165778:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab1ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.165785:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab1ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.165815:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab206 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.165822:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab206 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.165832:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab206 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739051.165851:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061ff0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665509248:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787739051.166342:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.166383:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739051.166647:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665509248 00000100:00100000:1.0:1787739051.166654:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665509248 00000100:00100000:1.0:1787739051.166702:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2434 00000100:00100000:1.0:1787739051.166711:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061ff840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665509248:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739051.166729:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739051.166748:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab20d lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab206 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.166769:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab20d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab206 expref: 12 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.166781:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab20d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab206 expref: 12 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.166790:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb1:0x0].0x0 (ffffa07433235800) 00010000:00010000:1.0:1787739051.166793:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739051.166796:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d800, rc 0) 00000100:00100000:1.0:1787739051.166833:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061ff840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:13793:x1874579665509248:12345-0@lo:101:mdt00_004.0 Request processed in 124us (493us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.166840:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2434 00000100:00100000:3.0:1787739051.169561:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061ff0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665509248:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739051.169590:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab206 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab20d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.169604:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab206 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab20d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739051.169630:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000588a0942 reply state 00000000697f45c0 transno 4294968276 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab1ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.169645:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab1ff lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.169652:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb1:0x0].0x0 (ffffa0730608b500) 00010000:00010000:3.0:1787739051.169655:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739051.169659:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787739051.169675:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab206 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab20d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.169683:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab206 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab20d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739051.169699:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xb1:0x0] 00000004:00010000:3.0:1787739051.169707:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000588a0942 reply state 00000000697f45c0 transno 4294968276 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab1f8 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x159:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.169715:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab1f8 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x159:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.169721:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x159:0x0].0x0 (ffffa0730608bb00) 00010000:00010000:3.0:1787739051.169724:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739051.169726:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:3.0:1787739051.169731:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x159:0x0]: returning size 4096 00000004:00080000:3.0:1787739051.169741:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739051.169749:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab1dc lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab1e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.169757:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab1dc lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab1e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.169764:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab1dc lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab1e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.169794:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739051.169799:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739051.169803:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ab1ea lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab1f1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.169810:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ab1ea lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab1f1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739051.169919:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17389:x1874579652402944:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 45895us (46480us total) trans 4294968276 rc 0/0 00000100:00100000:3.0:1787739051.169932:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23114 00010000:00010000:1.0:1787739051.170143:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739051.170151:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab1dc lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab1e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.170158:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609bc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739051.170163:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab1dc lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab1e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739051.170171:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab1dc lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab1e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.170202:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab1dc lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab1e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.170208:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739051.170236:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab1dc lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab1e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.170242:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ab1dc lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab1e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.170259:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00010000:00010000:1.0:1787739051.170262:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739051.170265:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739051.170267:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739051.170276:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665509376:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739051.170327:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.170497:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665509376 00000100:00100000:1.0:1787739051.170505:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665509376 00000100:00100000:1.0:1787739051.170564:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1149 00000100:00100000:1.0:1787739051.170574:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+12:3651:x1874579665509376:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739051.170587:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739051.170603:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab1e3 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab1dc expref: 12 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.170615:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae000) 00010000:00010000:1.0:1787739051.170622:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739051.170658:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+11:3651:x1874579665509376:12345-0@lo:103:ldlm_bl.0 Request processed in 87us (330us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.170665:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1149 00000100:00100000:2.0:1787739051.170786:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665509376:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739051.218840:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739051.218977:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652403328 00000100:00100000:3.0:1787739051.218987:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652403328 00000100:00100000:3.0:1787739051.219049:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23115 00000100:00100000:3.0:1787739051.219059:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17390:x1874579652403328:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:3.0:1787739051.219362:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ec000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17390:x1874579652403328:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 304us (529us total) trans 0 rc -17/-17 00000100:00100000:3.0:1787739051.219370:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23115 00000100:00100000:0.0:1787739051.241963:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739051.242020:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652403456 00000100:00100000:2.0:1787739051.242030:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652403456 00000100:00100000:2.0:1787739051.242090:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23116 00000100:00100000:2.0:1787739051.242114:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17391:x1874579652403456:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739051.242134:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739051.242158:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab214 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0949c0 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.242217:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab21b lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.242230:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab21b lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.242241:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab21b lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.242252:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab222 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.242262:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab1f1 lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab1ea expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.242273:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab222 lrc: 4/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.242302:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab1f1 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ab1ea expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.242318:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665509504:0@lo:104: 00000100:00100000:2.0:1787739051.242367:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739051.242449:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e540 going to sleep for 11 seconds 00000100:00100000:0.0:1787739051.242460:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665509504 00000100:00100000:0.0:1787739051.242466:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665509504 00000100:00100000:0.0:1787739051.242504:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1031 00000100:00100000:0.0:1787739051.242512:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432ed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6523:x1874579665509504:12345-0@lo:104: 00010000:00010000:0.0:1787739051.242524:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ab1ea lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab1f1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.242581:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787739051.242587:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432ed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6523:x1874579665509504:12345-0@lo:104: Request processed in 76us (219us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.242593:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1031 00010000:00010000:2.0:1787739051.242614:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739051.242622:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ab1ea lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab1f1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739051.242667:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665509504:0@lo:104: 00010000:00010000:3.0:1787739051.242684:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab222 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.242690:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739051.242698:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ab1ea lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ab1f1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739051.242708:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ab1ea lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ab1f1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.242736:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ab1ea lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab1f1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.242743:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1787739051.242834:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665509632:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739051.242854:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ab1ea lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab1f1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.242865:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ab1ea lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab1f1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.242882:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739051.242884:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739051.242886:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739051.242948:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739051.242990:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665509632 00000100:00100000:2.0:1787739051.242995:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665509632 00010000:00010000:2.0:1787739051.243015:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab1f1 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ab1ea expref: 11 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.243052:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1352 00000100:00100000:2.0:1787739051.243059:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d1e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3651:x1874579665509632:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739051.243071:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739051.249501:0:6502:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1ab181 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab188 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739051.249521:0:6502:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1ab181 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab188 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.249565:0:6502:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1ab181 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab188 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.249571:0:6502:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739051.249607:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1ab181 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab188 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787739051.249624:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983288 is committed 00000001:00080000:1.0:1787739051.249637:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983287 is committed 00000020:00080000:1.0:1787739051.249644:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 337 batchid updated 00000020:00080000:1.0:1787739051.249655:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa0730610ed80 refcount 2 committed 0 result 0 batchid 338 00000020:00080000:1.0:1787739051.249658:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2d00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730610e3c0 00000020:00080000:1.0:1787739051.249662:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].335 00000020:00080000:1.0:1787739051.249664:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2600 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07302faa400 00000020:00080000:1.0:1787739051.249667:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].335 00000100:00100000:3.0:1787739051.249706:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665509760:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:3.0:1787739051.249768:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.250561:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665509760 00000100:00100000:1.0:1787739051.250569:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665509760 00000100:00100000:1.0:1787739051.250634:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1150 00000100:00100000:1.0:1787739051.250645:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061ff0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+7:3652:x1874579665509760:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:1.0:1787739051.250660:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739051.250681:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab188 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x425cbd54 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab181 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.250721:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab111 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x14ed:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.250735:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab111 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x14ed:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.250742:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab111 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x14ed:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.250749:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739051.250754:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab111 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x14ed:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.250760:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x425cbd54 (ffffa07403eeb700) 00010000:00010000:2.0:1787739051.250762:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab111 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x14ed:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.250775:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739051.250778:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14ed:0x0].0x0 (ffffa07403eebf00) 00010000:00010000:2.0:1787739051.250783:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739051.250786:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab111 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x14ed:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.250792:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab111 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x14ed:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.250811:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab118 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.250822:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab118 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739051.250829:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061ff0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+6:3652:x1874579665509760:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 185us (1058us total) trans 0 rc 0/0 00010000:00010000:2.0:1787739051.250831:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab118 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.250843:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633cc00 already unused, calling callback (ffffffffc18da4f0) 00000100:00100000:1.0:1787739051.250844:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1150 00010000:00010000:2.0:1787739051.250849:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab118 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.250857:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab118 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.250866:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb3:0x0].0x0 (ffffa07403eeb600) 00010000:00010000:2.0:1787739051.250869:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739051.250872:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab118 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739051.250876:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665509760:0@lo:103:jbd2/dm-0-8.0 00010000:00010000:2.0:1787739051.250882:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab118 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739051.250909:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 338 committed batchid 337 00000004:00080000:2.0:1787739051.251010:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730610e0c0 with version 641 00000020:00080000:2.0:1787739051.251064:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 337 cancel update log [0x240000401:0x3:0x0].333: rc = 0 00000020:00080000:2.0:1787739051.251128:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 337 cancel update log [0x200000400:0x3:0x0].334: rc = 0 00000020:00080000:2.0:1787739051.251138:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 337 00010000:00010000:2.0:1787739051.251187:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab1f1 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ab1ea expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.251204:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xb98062fb (ffffa0730608b700) 00010000:00010000:2.0:1787739051.251208:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab222 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.251216:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab222 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.251223:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab222 lrc: 4/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.251245:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739051.251291:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d1e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+6:3651:x1874579665509632:12345-0@lo:103:ldlm_bl.0 Request processed in 8232us (8343us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.251298:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1352 00010000:00010000:2.0:1787739051.251323:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab222 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.251331:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab222 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739051.251332:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665509632:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739051.251434:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41f0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665509888:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739051.251468:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739051.251489:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739051.251532:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665509888 00000100:00100000:1.0:1787739051.251537:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665509888 00000100:00100000:1.0:1787739051.251573:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2435 00000100:00100000:1.0:1787739051.251581:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07436d8d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6523:x1874579665509888:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739051.251662:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07436d8d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6523:x1874579665509888:12345-0@lo:1000:mdt00_000.0 Request processed in 82us (193us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.251669:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2435 00000100:00100000:2.0:1787739051.252062:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41f0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665509888:0@lo:1000:mdt00_000.0 00010000:00010000:2.0:1787739051.252110:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ab229 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x159:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.252122:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ab229 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x159:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.252130:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ab229 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x159:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739051.252150:0:6523:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ab229 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x159:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.252162:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab21b lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.252168:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab21b lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.252184:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab21b lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.252191:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab21b lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.252197:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739051.252200:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab21b lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.252207:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab21b lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.252216:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403eeb400) 00010000:00010000:2.0:1787739051.252219:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739051.252222:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab21b lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.252228:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab21b lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.252239:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab222 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.252244:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab222 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.252250:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab222 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.252256:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633de00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739051.252268:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab222 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.252275:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab222 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.252283:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xb98062fb (ffffa0730608b700) 00010000:00010000:2.0:1787739051.252285:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739051.252289:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab222 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.252294:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab222 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.252327:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab214 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949c0 expref: 15 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.252344:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ab229 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x159:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949c0 expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.252355:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x159:0x0].0x0 (ffffa07403eeb900) 00010000:00010000:2.0:1787739051.252358:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739051.252360:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d400, rc 0) 00000100:00100000:2.0:1787739051.252429:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17391:x1874579652403456:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 10330us (10477us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.252437:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23116 00000004:00080000:3.0:1787739051.255288:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 641 rpc_version 641 00000020:00100000:2.0:1787739051.255653:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739051.255666:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743c41f0c0 x1874579665510144/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739051.255691:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41f0c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665510144:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739051.255799:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739051.255873:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.255911:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665510144 00000100:00100000:3.0:1787739051.255918:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665510144 00000100:00100000:3.0:1787739051.255998:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2436 00000100:00100000:3.0:1787739051.256007:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665510144:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739051.256184:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665510144:12345-0@lo:1000:osp_up1-0.0 Request processed in 177us (384us total) trans 4294968277 rc 0/0 00000100:00100000:3.0:1787739051.256194:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2436 00000100:00100000:2.0:1787739051.256265:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41f0c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665510144:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739051.258734:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739051.259343:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652403584 00000100:00100000:0.0:1787739051.259352:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652403584 00000100:00100000:0.0:1787739051.259561:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23117 00000100:00100000:0.0:1787739051.259572:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17391:x1874579652403584:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739051.259591:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739051.259608:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab230 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x159:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0949c7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.259658:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ab237 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x159:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.259672:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ab237 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x159:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.259680:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ab237 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x159:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739051.259765:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x159:0x0]: returning size 4096 00000004:00200000:0.0:1787739051.259771:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739051.259809:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab230 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x159:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949c7 expref: 11 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.259825:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ab237 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x159:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949c7 expref: 10 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.259833:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x159:0x0].0x0 (ffffa0730608bb00) 00010000:00010000:0.0:1787739051.259837:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739051.259841:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a600, rc 0) 00000100:00100000:0.0:1787739051.259901:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17391:x1874579652403584:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 331us (1174us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.259910:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23117 00000100:00100000:1.0:1787739051.269987:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739051.270057:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652403712 00000100:00100000:3.0:1787739051.270065:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652403712 00000100:00100000:3.0:1787739051.270130:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23118 00000100:00100000:3.0:1787739051.270141:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17391:x1874579652403712:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739051.270162:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739051.270182:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab23e lrc: 2/0,0 mode: --/CR res: [0x240000400:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0949ce expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.270237:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab245 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.270251:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab245 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.270263:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab245 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739051.270306:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xb1:0x0]: returning size 4096 00000004:00200000:3.0:1787739051.270311:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739051.270339:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab23e lrc: 0/0,0 mode: --/CR res: [0x240000400:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ce expref: 12 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.270355:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab245 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ce expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.270363:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb1:0x0].0x0 (ffffa0730608b500) 00010000:00010000:3.0:1787739051.270366:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739051.270369:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab600, rc 0) 00000100:00100000:3.0:1787739051.270455:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17391:x1874579652403712:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 316us (477us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.270462:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23118 00000100:00100000:2.0:1787739051.275686:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739051.275766:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652403840 00000100:00100000:0.0:1787739051.275776:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652403840 00000100:00100000:0.0:1787739051.275859:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23119 00000100:00100000:0.0:1787739051.275869:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17391:x1874579652403840:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739051.275889:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739051.275909:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab24c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0949d5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.275975:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab253 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.275991:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab20d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab206 expref: 5 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.276001:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab253 lrc: 4/1,0 mode: --/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.276031:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab20d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ab206 expref: 5 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739051.276046:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d30c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665510272:0@lo:104: 00000100:00100000:0.0:1787739051.276096:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739051.276158:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355ed540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.276388:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665510272 00000100:00100000:3.0:1787739051.276393:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665510272 00000100:00100000:3.0:1787739051.276450:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1032 00000100:00100000:3.0:1787739051.276458:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d0780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6523:x1874579665510272:12345-0@lo:104: 00010000:00010000:3.0:1787739051.276471:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab206 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab20d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.276537:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739051.276544:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d0780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6523:x1874579665510272:12345-0@lo:104: Request processed in 87us (448us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.276552:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1032 00010000:00010000:3.0:1787739051.276575:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739051.276582:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab206 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab20d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.276591:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739051.276597:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab206 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ab20d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739051.276607:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab206 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ab20d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.276651:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab206 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab20d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.276664:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739051.276697:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab206 lrc: 1/0,0 mode: --/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab20d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.276734:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab206 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab20d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.276823:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739051.276825:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739051.276827:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739051.276833:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fc3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665510400:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739051.276870:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739051.276904:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665510400 00000100:00100000:2.0:1787739051.276909:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665510400 00010000:00010000:2.0:1787739051.276927:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab20d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ab206 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.276951:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1353 00000100:00100000:2.0:1787739051.276958:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+6:3653:x1874579665510400:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739051.276969:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787739051.277210:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d30c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665510272:0@lo:104: 00010000:00010000:3.0:1787739051.277233:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab253 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:1.0:1787739051.284317:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 338 batchid updated 00010000:00010000:3.0:1787739051.284482:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab20d lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ab206 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.284500:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb1:0x0].0x0 (ffffa07433235800) 00010000:00010000:3.0:1787739051.284506:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab253 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.284517:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab253 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.284525:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab253 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.284556:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739051.284642:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+5:3653:x1874579665510400:12345-0@lo:103:ldlm_bl.0 Request processed in 7684us (7770us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.284653:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1353 00000100:00100000:3.0:1787739051.284711:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fc3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665510400:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739051.285476:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab253 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.285486:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab253 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739051.285544:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb1:0x0]: returning size 4096 00000004:00200000:1.0:1787739051.285550:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739051.285597:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab24c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949d5 expref: 16 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.285615:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab253 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949d5 expref: 15 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.285623:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb1:0x0].0x0 (ffffa07433235800) 00010000:00010000:1.0:1787739051.285626:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739051.285629:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a200, rc 0) 00000100:00100000:1.0:1787739051.285689:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17391:x1874579652403840:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 9821us (10010us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.285698:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23119 00000020:00080000:2.0:1787739051.286347:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 338 00000004:00080000:2.0:1787739051.286401:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730610ee40 with version 642 00000020:00080000:2.0:1787739051.286425:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 338 cancel update log [0x240000401:0x3:0x0].335: rc = 0 00080000:02000000:2.0:1787739051.286457:0:6555:0:(osd_handler.c:2218:osd_trans_start()) *** cfs_fail_loc=19a, val=0*** 00000020:00080000:2.0:1787739051.286464:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 338 cancel update log [0x200000400:0x3:0x0].335: rc = -5 00000020:00080000:2.0:1787739051.286470:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 338 00000004:00080000:2.0:1787739051.286478:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 642 rpc_version 642 00000020:00100000:2.0:1787739051.286502:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739051.286511:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743c41de00 x1874579665510656/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739051.286531:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41de00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665510656:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739051.286584:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739051.286621:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ed80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739051.286710:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665510656 00000100:00100000:1.0:1787739051.286717:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665510656 00000100:00100000:1.0:1787739051.286773:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2437 00000100:00100000:1.0:1787739051.286782:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665510656:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739051.287180:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665510656:12345-0@lo:1000:osp_up1-0.0 Request processed in 398us (594us total) trans 4294968278 rc 0/0 00000100:00100000:1.0:1787739051.287188:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2437 00000100:00100000:3.0:1787739051.287265:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41de00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665510656:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739051.303638:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739051.303679:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739051.303755:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652403968 00000100:00100000:1.0:1787739051.303765:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652403968 00000100:00100000:1.0:1787739051.303860:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1151 00000100:00100000:1.0:1787739051.303871:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba6850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2406:x1874579652403968:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:1.0:1787739051.303888:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739051.303911:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ab229 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x159:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949c0 expref: 15 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.303929:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x159:0x0].0x0 (ffffa07403eeb900) 00010000:00010000:1.0:1787739051.303951:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739051.304009:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba6850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2406:x1874579652403968:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 140us (328us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.304017:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1151 00000100:00100000:3.0:1787739051.304125:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652404224 00000100:00100000:3.0:1787739051.304130:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652404224 00000100:00100000:3.0:1787739051.304162:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23120 00000100:00100000:3.0:1787739051.304170:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17391:x1874579652404224:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:3.0:1787739051.304213:0:6524:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739051.304227:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ab237 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x159:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949c7 expref: 11 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.304241:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x159:0x0].0x0 (ffffa0730608bb00) 00010000:00010000:3.0:1787739051.304244:0:6524:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739051.304294:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab25a lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.304302:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab25a lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.304310:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab25a lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739051.304330:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fc3c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665510784:0@lo:101:mdt00_001.0 00000100:00100000:3.0:1787739051.304372:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.304403:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4540 going to sleep for 11 seconds 00000100:00100000:0.0:1787739051.304623:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665510784 00000100:00100000:0.0:1787739051.304629:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665510784 00000100:00100000:0.0:1787739051.304673:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2438 00000100:00100000:0.0:1787739051.304948:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fcf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+5:6524:x1874579665510784:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739051.304960:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739051.304975:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab261 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab25a expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.305000:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab261 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab25a expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.305013:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab261 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab25a expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.305022:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0741590d600) 00010000:00010000:0.0:1787739051.305025:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739051.305028:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609bc00, rc 0) 00000100:00100000:1.0:1787739051.305228:0:6524:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983288 gen 2 00000004:00080000:1.0:1787739051.305235:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983283 committed transno 4294983288 00000020:00080000:1.0:1787739051.305241:0:6524:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa0730610ec00 refcount 2 committed 0 result 0 batchid 329 00000020:00080000:1.0:1787739051.305245:0:6524:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2a00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730610ef00 00000020:00080000:1.0:1787739051.305247:0:6524:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].322 00000020:00080000:1.0:1787739051.305249:0:6524:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2000 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa074184ee800 00000020:00080000:1.0:1787739051.305252:0:6524:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].321 00000020:00080000:2.0:1787739051.305282:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 329 committed batchid 328 00000100:00100000:0.0:1787739051.305381:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fcf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:6524:x1874579665510784:12345-0@lo:101:mdt00_001.0 Request processed in 130us (701us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.305388:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2438 00000020:00080000:2.0:1787739051.305409:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 328 00000004:00080000:1.0:1787739051.305500:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983285 committed transno 4294983288 00000004:00080000:1.0:1787739051.305508:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983286 committed transno 4294983288 00000004:00080000:1.0:1787739051.305517:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983288 committed transno 4294983288 00000100:00100000:1.0:1787739051.305534:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fc3c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665510784:0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739051.305551:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab25a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab261 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.305563:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab25a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab261 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739051.305580:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739051.305606:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ab268 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.305614:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ab268 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.305622:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ab268 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739051.305635:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665510912:0@lo:101:mdt00_001.0 00000100:00100000:1.0:1787739051.305662:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.305685:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcce40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739051.305734:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665510912 00000100:00100000:0.0:1787739051.305740:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665510912 00000100:00100000:0.0:1787739051.305774:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2439 00000100:00100000:0.0:1787739051.305781:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ecf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:6524:x1874579665510912:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739051.305791:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739051.305803:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ab26f lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab268 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.305820:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ab26f lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab268 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.305828:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ab26f lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab268 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.305834:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xb98062fb (ffffa0741590db00) 00010000:00010000:0.0:1787739051.305835:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739051.305837:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609aa00, rc 0) 00000100:00100000:0.0:1787739051.306118:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ecf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665510912:12345-0@lo:101:mdt00_001.0 Request processed in 337us (454us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.306124:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2439 00000100:00100000:2.0:1787739051.306219:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665510912:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739051.306231:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ab268 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab26f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.306238:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ab268 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab26f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739051.306250:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:2.0:1787739051.306276:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562c000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665511040:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739051.306298:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739051.306317:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739051.306329:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665511040 00000100:00100000:0.0:1787739051.306333:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665511040 00000100:00100000:0.0:1787739051.306399:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2440 00000100:00100000:0.0:1787739051.306407:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665511040:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739051.306492:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665511040:12345-0@lo:1000:mdt00_001.0 Request processed in 86us (191us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.306498:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2440 00000100:00100000:2.0:1787739051.306521:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562c000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665511040:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739051.306556:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562c000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665511168:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739051.306578:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.307591:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665511168 00000100:00100000:3.0:1787739051.307600:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665511168 00000100:00100000:2.0:1787739051.307609:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.307637:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2441 00000100:00100000:3.0:1787739051.307645:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665511168:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739051.308052:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665511168:12345-0@lo:1000:mdt00_001.0 Request processed in 407us (1472us total) trans 0 rc 1/1 00000100:00100000:3.0:1787739051.308060:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2441 00000100:00100000:2.0:1787739051.308072:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562c000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665511168:0@lo:1000:mdt00_001.0 00010000:00010000:2.0:1787739051.308143:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1ab276 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x159:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.308152:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1ab276 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x159:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.308161:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1ab276 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x159:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.308174:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562c000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665511296:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1787739051.308199:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739051.308223:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e540 going to sleep for 11 seconds 00000100:00100000:0.0:1787739051.308348:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665511296 00000100:00100000:0.0:1787739051.308353:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665511296 00000100:00100000:0.0:1787739051.308383:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2442 00000100:00100000:0.0:1787739051.308390:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562f840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665511296:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739051.308399:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739051.308432:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab27d lrc: 2/0,0 mode: --/EX res: [0x240000402:0x159:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab276 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.308452:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab27d lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x159:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab276 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.308464:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab27d lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x159:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab276 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.308472:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x159:0x0].0x0 (ffffa0741590da00) 00010000:00010000:0.0:1787739051.308475:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739051.308479:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b600, rc 0) 00000100:00100000:0.0:1787739051.308512:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562f840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6524:x1874579665511296:12345-0@lo:101:mdt00_001.0 Request processed in 122us (312us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.308620:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562c000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665511296:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739051.308633:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1ab276 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x159:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab27d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.308643:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1ab276 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x159:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab27d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.308665:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab284 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x159:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.308674:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab284 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x159:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.308855:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab284 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x159:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.308874:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab28b lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.308886:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab245 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ce expref: 10 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.308893:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab28b lrc: 4/0,1 mode: --/EX res: [0x240000400:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.308914:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab245 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ce expref: 10 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.308928:0:6524:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab245 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ce expref: 10 pid: 6525 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.308940:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562c000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665511424:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739051.308942:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2442 00000100:00100000:2.0:1787739051.308999:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739051.316744:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562c000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665511424:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739051.316770:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab28b lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.318250:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739051.318299:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652404352 00000100:00100000:1.0:1787739051.318309:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652404352 00010000:00010000:1.0:1787739051.318356:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab245 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ce expref: 11 pid: 6525 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739051.318402:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1354 00000100:00100000:1.0:1787739051.318426:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2408:x1874579652404352:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739051.318440:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739051.318446:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab245 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ce expref: 11 pid: 6525 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.318461:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab245 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ce expref: 11 pid: 6525 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.318479:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab245 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ce expref: 11 pid: 6525 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.318489:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb1:0x0].0x0 (ffffa0730608b500) 00010000:00010000:1.0:1787739051.318493:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab28b lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.318499:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab28b lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.318505:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab28b lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.318543:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739051.318584:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab28b lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.318593:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab28b lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739051.318596:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652404352:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 185us (350us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.318608:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1354 00010000:00010000:2.0:1787739051.318642:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab292 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.318655:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab292 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.318665:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab292 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.318694:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d1a40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665511552:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1787739051.318739:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.318834:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665511552 00000100:00100000:1.0:1787739051.318839:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665511552 00000100:00100000:2.0:1787739051.318846:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e240 going to sleep for 11 seconds 00000100:00100000:1.0:1787739051.318881:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2443 00000100:00100000:1.0:1787739051.318889:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6524:x1874579665511552:12345-0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739051.318904:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739051.318918:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab299 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab292 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.318942:0:8447:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab253 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949d5 expref: 13 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.318950:0:8447:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab299 lrc: 4/0,0 mode: --/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab292 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.318972:0:8447:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab253 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949d5 expref: 13 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.318996:0:8447:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab253 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949d5 expref: 13 pid: 6523 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739051.319008:0:8447:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ee1c0 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8447:1874579665511680:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739051.319066:0:8447:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739051.324826:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739051.324950:0:8447:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ee1c0 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8447:1874579665511680:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739051.324992:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab299 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab292 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.325006:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb1:0x0].0x0 (ffffa07433235800) 00010000:00010000:1.0:1787739051.325009:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739051.325014:0:8447:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab299 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab292 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.325023:0:8447:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab299 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab292 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.325036:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c200, rc 0) 00000100:00100000:1.0:1787739051.325119:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665511552:12345-0@lo:101:mdt00_001.0 Request processed in 6230us (6380us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.325128:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2443 00000100:00100000:2.0:1787739051.325272:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d1a40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665511552:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739051.325292:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab292 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab299 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.325302:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab292 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab299 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739051.329350:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652404480 00000100:00100000:3.0:1787739051.329366:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652404480 00010000:00010000:3.0:1787739051.329442:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab253 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949d5 expref: 14 pid: 6523 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739051.329492:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1355 00000100:00100000:3.0:1787739051.329505:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652404480:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739051.329524:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739051.329532:0:6510:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab253 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949d5 expref: 14 pid: 6523 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.329545:0:6510:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab253 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949d5 expref: 14 pid: 6523 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.329567:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab253 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949d5 expref: 14 pid: 6523 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.329581:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb1:0x0].0x0 (ffffa07433235800) 00010000:00010000:3.0:1787739051.329589:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab299 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab292 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.329600:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab299 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab292 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.329610:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab299 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab292 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.329646:0:6510:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab299 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab292 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739051.329661:0:6510:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fda40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6510:1874579665511808:0@lo:105: 00000100:00100000:3.0:1787739051.329730:0:6510:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.329792:0:6510:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4540 going to sleep for 11 seconds 00000100:00100000:0.0:1787739051.329907:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665511808 00000100:00100000:0.0:1787739051.329913:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665511808 00000100:00100000:0.0:1787739051.329953:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1033 00000100:00100000:0.0:1787739051.329962:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fe1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6510:x1874579665511808:12345-0@lo:105: 00010000:00010000:0.0:1787739051.329973:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab292 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab299 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.329984:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab292 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab299 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.329993:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab292 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab299 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.330000:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab292 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab299 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.330007:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab292 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab299 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.330015:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab292 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ab299 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.330043:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633da00) 00000100:00100000:0.0:1787739051.330080:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fe1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6510:x1874579665511808:12345-0@lo:105: Request processed in 120us (350us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.330087:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1033 00010000:00010000:1.0:1787739051.330147:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab292 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab299 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.330157:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab292 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab299 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739051.330833:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ee1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665511936:0@lo:403:mdt00_001.0 00000100:00100000:1.0:1787739051.330881:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.330910:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc600 going to sleep for 11 seconds 00000100:00100000:0.0:1787739051.331008:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665511936 00000100:00100000:0.0:1787739051.331014:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665511936 00000100:00100000:0.0:1787739051.331054:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2444 00000100:00100000:0.0:1787739051.331061:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665511936:12345-0@lo:403:mdt00_001.0 00000100:00100000:0.0:1787739051.332884:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665511936:12345-0@lo:403:mdt00_001.0 Request processed in 1813us (1991us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.333252:0:6510:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fda40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6510:1874579665511808:0@lo:105: 00010000:00010000:3.0:1787739051.333276:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739051.333325:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:2409:x1874579652404480:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 3821us (8507us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.333337:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1355 00000100:00100000:1.0:1787739051.334516:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ee1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665511936:0@lo:403:mdt00_001.0 00000040:00080000:1.0:1787739051.335464:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].324, 664 off494352 00000040:00080000:1.0:1787739051.335668:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].324, 744 off539848 00000004:00080000:1.0:1787739051.335675:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07305fcc600 with version 617 00000004:00080000:3.0:1787739051.335895:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 617 rpc_version 617 00000020:00100000:3.0:1787739051.335927:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739051.335936:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073061fda40 x1874579665512192/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1808/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739051.335962:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fda40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665512192:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739051.336032:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.336086:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4540 going to sleep for 11 seconds 00000100:00100000:0.0:1787739051.338884:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2444 00000100:00100000:0.0:1787739051.338928:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665512192 00000100:00100000:0.0:1787739051.338989:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665512192 00000100:00100000:0.0:1787739051.339224:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2445 00000100:00100000:0.0:1787739051.339235:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061ffc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665512192:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739051.339976:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061ffc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665512192:12345-0@lo:1000:osp_up0-1.0 Request processed in 740us (3940us total) trans 4294983289 rc 0/0 00000100:00100000:0.0:1787739051.339985:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2445 00000100:00100000:3.0:1787739051.340624:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fda40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665512192:0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1787739051.340725:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00200000:2.0:1787739051.340913:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x159:0x0]: returning size 4096 00010000:00010000:2.0:1787739051.340944:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab28b lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.340954:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab28b lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.340961:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab28b lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.340967:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739051.340973:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab28b lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.340983:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab28b lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.340999:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb1:0x0].0x0 (ffffa0730608b500) 00010000:00010000:2.0:1787739051.341002:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739051.341007:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab28b lrc: 2/0,0 mode: --/EX res: [0x240000400:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.341014:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab28b lrc: 0/0,0 mode: --/EX res: [0x240000400:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.341028:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab292 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab299 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.341036:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab292 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab299 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.341044:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab292 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab299 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.341070:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739051.341081:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xb1:0x0] 00010000:00010000:2.0:1787739051.341088:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab284 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x159:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.341094:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab284 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x159:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.341110:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab284 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x159:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.341118:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739051.341121:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab284 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x159:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.341126:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab284 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x159:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.341133:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x159:0x0].0x0 (ffffa0730608bb00) 00010000:00010000:2.0:1787739051.341135:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739051.341138:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab284 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x159:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.341143:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab284 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x159:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.341154:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1ab276 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x159:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab27d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.341160:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1ab276 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x159:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab27d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.341166:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1ab276 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x159:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab27d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.341185:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:2.0:1787739051.341195:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739051.341200:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab25a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab261 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.341211:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab25a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab261 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.341217:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab25a lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab261 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.341226:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:2.0:1787739051.341230:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739051.341234:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ab268 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab26f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.341240:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ab268 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab26f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.341334:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17391:x1874579652404224:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 37162us (37702us total) trans 4294968279 rc 0/0 00000100:00100000:2.0:1787739051.341344:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23120 00010000:00010000:3.0:1787739051.341797:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1787739051.341805:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab292 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab299 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.341817:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633da00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739051.341822:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab292 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab299 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739051.341830:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab292 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab299 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.341866:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab292 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab299 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.341874:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739051.341910:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab292 lrc: 1/0,0 mode: --/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab299 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.341916:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab292 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab299 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.341934:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787739051.341937:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1ab276 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x159:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab27d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.341943:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633cc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739051.341947:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1ab276 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x159:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab27d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739051.341954:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1ab276 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x159:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab27d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.341976:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1ab276 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x159:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab27d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.341983:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739051.341989:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1ab276 lrc: 1/0,0 mode: --/EX res: [0x240000402:0x159:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab27d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.341996:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1ab276 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x159:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab27d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.342007:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739051.342010:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab25a lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab261 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.342016:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739051.342020:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab25a lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab261 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739051.342026:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab25a lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab261 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.342043:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab25a lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab261 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.342048:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739051.342065:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab25a lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab261 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.342076:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ab25a lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab261 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.342087:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739051.342089:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739051.342090:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739051.342107:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739051.342125:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fe580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665512448:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739051.342178:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.342226:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787739051.342231:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fe940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665512320:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739051.342249:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.342275:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fc3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665512576:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739051.342295:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.342330:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665512448 00000100:00100000:3.0:1787739051.342337:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665512448 00000100:00100000:3.0:1787739051.342398:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665512320 00000100:00100000:3.0:1787739051.342402:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665512320 00000100:00100000:3.0:1787739051.342449:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665512576 00000100:00100000:3.0:1787739051.342453:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665512576 00000100:00100000:3.0:1787739051.342471:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1152 00000100:00100000:3.0:1787739051.342480:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fde00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+12:3651:x1874579665512448:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739051.342496:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739051.342516:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab27d lrc: 0/0,0 mode: --/EX res: [0x240000402:0x159:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab276 expref: 12 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.342530:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x159:0x0].0x0 (ffffa0741590da00) 00010000:00010000:3.0:1787739051.342540:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739051.342576:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fde00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+11:3651:x1874579665512448:12345-0@lo:103:ldlm_bl.0 Request processed in 96us (397us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.342584:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1152 00000100:00100000:3.0:1787739051.342601:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1153 00000100:00100000:3.0:1787739051.342607:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061ff0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+10:3651:x1874579665512320:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739051.342616:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739051.342627:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab299 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab292 expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.342637:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb1:0x0].0x0 (ffffa07433235800) 00010000:00010000:3.0:1787739051.342643:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739051.342667:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061ff0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3651:x1874579665512320:12345-0@lo:103:ldlm_bl.0 Request processed in 60us (417us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.342673:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1153 00000100:00100000:3.0:1787739051.342685:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1154 00000100:00100000:3.0:1787739051.342690:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fc780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+8:3652:x1874579665512576:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739051.342698:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739051.342708:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab261 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab25a expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.342718:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0741590d600) 00010000:00010000:3.0:1787739051.342724:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739051.342751:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fc780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+7:3652:x1874579665512576:12345-0@lo:103:ldlm_bl.0 Request processed in 61us (455us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.342756:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1154 00000100:00100000:3.0:1787739051.342801:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fc3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665512576:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739051.342829:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fe940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665512320:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739051.342839:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fe580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665512448:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739051.342994:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739051.401830:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739051.401867:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652404608 00000100:00100000:3.0:1787739051.401876:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652404608 00000100:00100000:3.0:1787739051.401928:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23121 00000100:00100000:3.0:1787739051.401937:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fe580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17397:x1874579652404608:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739051.401957:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739051.401995:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab2a0 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0949dc expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.402066:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab2a7 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.402079:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab2a7 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.402090:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab2a7 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.402111:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab2ae lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.402118:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab2ae lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.402125:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab2ae lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.402167:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab2a7 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.402175:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab2a7 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.402181:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab2a7 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.402188:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab2a7 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.402194:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739051.402198:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab2a7 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.402206:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab2a7 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.402219:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2700) 00010000:00010000:3.0:1787739051.402221:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739051.402226:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab2a7 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.402231:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab2a7 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.402243:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab2ae lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.402249:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab2ae lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.402255:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab2ae lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.402259:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739051.402261:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab2ae lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.402266:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab2ae lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.402273:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x1009fee1 (ffffa074331c2d00) 00010000:00010000:3.0:1787739051.402275:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739051.402278:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab2ae lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.402283:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab2ae lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.402329:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab2a0 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949dc expref: 14 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.402338:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2700) 00010000:00010000:3.0:1787739051.402341:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739051.402345:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab2a0 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949dc expref: 14 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.402355:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa400, rc 0) 00000100:00100000:3.0:1787739051.402407:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fe580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17397:x1874579652404608:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 471us (587us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739051.402443:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23121 00000100:00100000:0.0:1787739051.414188:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739051.414254:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652404736 00000100:00100000:1.0:1787739051.414264:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652404736 00000100:00100000:1.0:1787739051.414385:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23122 00000100:00100000:1.0:1787739051.414395:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17397:x1874579652404736:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739051.414437:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739051.414460:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab2b5 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0949e3 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.414529:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab2bc lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.414543:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab2bc lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.414553:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab2bc lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.414565:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ab2c3 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.414571:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ab2c3 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.414577:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ab2c3 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.414618:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab2bc lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.414625:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab2bc lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.414632:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab2bc lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.414638:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab2bc lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.414644:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739051.414649:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab2bc lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.414656:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab2bc lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.414667:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235700) 00010000:00010000:1.0:1787739051.414670:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739051.414674:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab2bc lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.414680:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab2bc lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.414692:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ab2c3 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.414698:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ab2c3 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.414704:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ab2c3 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.414709:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ce00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739051.414712:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ab2c3 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.414718:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ab2c3 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.414725:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x1009fee1 (ffffa07433235000) 00010000:00010000:1.0:1787739051.414727:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739051.414731:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ab2c3 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.414736:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ab2c3 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.414764:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab2b5 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949e3 expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.414774:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235700) 00010000:00010000:1.0:1787739051.414776:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739051.414780:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab2b5 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949e3 expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.414791:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1ca00, rc 0) 00000100:00100000:1.0:1787739051.414838:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17397:x1874579652404736:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 444us (660us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739051.414845:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23122 00000100:00100000:1.0:1787739051.441367:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739051.441451:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652404864 00000100:00100000:2.0:1787739051.441459:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652404864 00000100:00100000:2.0:1787739051.441516:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23123 00000100:00100000:2.0:1787739051.441526:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17397:x1874579652404864:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739051.441545:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739051.441568:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab2ca lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0949ea expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.441627:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab2d1 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.441641:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab2d1 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.441652:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab2d1 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739051.441696:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:2.0:1787739051.441701:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739051.441733:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab2ca lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ea expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.441750:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab2d1 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ea expref: 14 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.441759:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403eebf00) 00010000:00010000:2.0:1787739051.441762:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739051.441765:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c600, rc 0) 00000100:00100000:2.0:1787739051.441818:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17397:x1874579652404864:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 294us (462us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.441825:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23123 00000100:00100000:0.0:1787739051.461226:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739051.461399:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652405120 00000100:00100000:2.0:1787739051.461407:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652405120 00000100:00100000:2.0:1787739051.461487:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23124 00000100:00100000:2.0:1787739051.461496:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17397:x1874579652405120:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:2.0:1787739051.461547:0:13793:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739051.461572:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab2d1 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ea expref: 14 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.461597:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403eebf00) 00010000:00010000:2.0:1787739051.461606:0:13793:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739051.461656:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab2d8 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.461668:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab2d8 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.461678:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab2d8 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.461689:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab2df lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.461695:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab2df lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.461701:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab2df lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1787739051.462484:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].337, 2168 off519088 00000040:00080000:2.0:1787739051.462555:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].337, 2368 off562536 00000004:00080000:2.0:1787739051.462561:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730610e3c0 with version 643 00000004:00080000:2.0:1787739051.465112:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 643 rpc_version 643 00000020:00100000:2.0:1787739051.465164:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739051.465176:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074080d1680 x1874579665512832/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 3976/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739051.465200:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d1680 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665512832:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739051.465269:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739051.465442:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ef00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739051.465976:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2:3653:1874579665512960:0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.466028:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.466059:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3653:1874579665513472:0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.466085:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.466115:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3653:1874579665513984:0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.466136:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.466156:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1787739051.466161:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3653:1874579665513344:0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.466178:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.466191:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3653:1874579665513856:0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.466207:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.466229:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3651:1874579665513216:0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.466250:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.466261:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c382580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665513728:0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.466283:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.466310:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665514240:0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.466331:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.466348:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:1.0:1787739051.466354:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3651:1874579665513088:0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.466376:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.466389:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665513600:0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.466408:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.466465:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665514112:0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.466487:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.466517:0:6517:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665512960 00000100:00100000:1.0:1787739051.466525:0:6517:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665512960 00000100:00100000:1.0:1787739051.466574:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 444 00000100:00100000:1.0:1787739051.466582:0:6517:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410c86450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2+24:3653:x1874579665512960:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.466637:0:6517:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410c86450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2+24:3653:x1874579665512960:12345-0@lo:400:kworker.0 Request processed in 55us (608us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.466644:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 444 00000100:00100000:1.0:1787739051.466676:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665513472 00000100:00100000:1.0:1787739051.466680:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665513472 00000100:00100000:1.0:1787739051.466706:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665513984 00000100:00100000:1.0:1787739051.466709:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665513984 00000100:00100000:1.0:1787739051.466721:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665513344 00000100:00100000:1.0:1787739051.466723:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665513344 00000100:00100000:1.0:1787739051.466735:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665513856 00000100:00100000:1.0:1787739051.466737:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665513856 00000100:00100000:1.0:1787739051.466748:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665513216 00000100:00100000:1.0:1787739051.466750:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665513216 00000100:00100000:1.0:1787739051.466760:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665513088 00000100:00100000:1.0:1787739051.466762:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665513088 00000100:00100000:1.0:1787739051.466773:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 642 00000100:00100000:1.0:1787739051.466779:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3653:x1874579665513472:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.466809:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3653:x1874579665513472:12345-0@lo:400:kworker.0 Request processed in 31us (723us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.466816:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 642 00000100:00100000:1.0:1787739051.466827:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 643 00000100:00100000:1.0:1787739051.466832:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874579665513984:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.466854:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874579665513984:12345-0@lo:400:kworker.0 Request processed in 23us (717us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.466859:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 643 00000100:00100000:1.0:1787739051.466869:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 644 00000100:00100000:1.0:1787739051.466876:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874579665513344:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.466902:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874579665513344:12345-0@lo:400:kworker.0 Request processed in 26us (723us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.466907:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 644 00000100:00100000:1.0:1787739051.466918:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 645 00000100:00100000:1.0:1787739051.466924:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3653:x1874579665513856:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.466951:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3653:x1874579665513856:12345-0@lo:400:kworker.0 Request processed in 28us (743us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.466960:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 645 00000100:00100000:1.0:1787739051.466971:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 646 00000100:00100000:1.0:1787739051.466976:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305654f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1874579665513216:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.467006:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305654f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1874579665513216:12345-0@lo:400:kworker.0 Request processed in 30us (755us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.467012:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 646 00000100:00100000:1.0:1787739051.467023:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 647 00000100:00100000:1.0:1787739051.467028:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305655e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1874579665513088:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.467055:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305655e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1874579665513088:12345-0@lo:400:kworker.0 Request processed in 26us (676us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.467060:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 647 00000100:00100000:1.0:1787739051.467078:0:8438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665513728 00000100:00100000:1.0:1787739051.467081:0:8438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665513728 00000100:00100000:1.0:1787739051.467118:0:8438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665514240 00000100:00100000:1.0:1787739051.467121:0:8438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665514240 00000100:00100000:1.0:1787739051.467139:0:8438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665513600 00000100:00100000:1.0:1787739051.467142:0:8438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665513600 00000100:00100000:1.0:1787739051.467159:0:8438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665514112 00000100:00100000:1.0:1787739051.467162:0:8438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665514112 00000100:00100000:1.0:1787739051.467181:0:8438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 337 00000100:00100000:1.0:1787739051.467186:0:8438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305655a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665513728:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.467216:0:8438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305655a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665513728:12345-0@lo:400:kworker.0 Request processed in 30us (932us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.467222:0:8438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 337 00000100:00100000:1.0:1787739051.467231:0:8438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 338 00000100:00100000:1.0:1787739051.467236:0:8438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305657840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665514240:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.467286:0:8438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305657840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665514240:12345-0@lo:400:kworker.0 Request processed in 49us (954us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.467292:0:8438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 338 00000100:00100000:1.0:1787739051.467302:0:8438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 339 00000100:00100000:1.0:1787739051.467308:0:8438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305655680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665513600:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.467333:0:8438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305655680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665513600:12345-0@lo:400:kworker.0 Request processed in 26us (925us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.467339:0:8438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 339 00000100:00100000:1.0:1787739051.467349:0:8438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 340 00000100:00100000:1.0:1787739051.467356:0:8438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073056543c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665514112:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.467382:0:8438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073056543c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665514112:12345-0@lo:400:kworker.0 Request processed in 27us (894us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.467388:0:8438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 340 00000100:00100000:1.0:1787739051.467508:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3651:1874579665513088:0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.467519:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561ad00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665513600:0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.467528:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665514112:0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.467537:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3651:1874579665513216:0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.467544:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c382580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665513728:0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.467552:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665514240:0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.467591:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3653:1874579665513344:0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.467600:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3653:1874579665513856:0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.467608:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2:3653:1874579665512960:0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.467617:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3653:1874579665513472:0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.467625:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3653:1874579665513984:0@lo:400:kworker.0 00000100:00100000:1.0:1787739051.470187:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305656940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665514368:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1787739051.470252:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739051.470435:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665512832 00000100:00100000:0.0:1787739051.470444:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665512832 00000100:00100000:0.0:1787739051.470489:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2446 00000100:00100000:0.0:1787739051.470497:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d1a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665512832:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739051.471095:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665514368 00000100:00100000:1.0:1787739051.471109:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665514368 00000100:00100000:1.0:1787739051.471152:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 348 00000100:00100000:1.0:1787739051.471161:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305657480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874579665514368:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1787739051.471181:0:8441:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 25431488 pending 0 free 3923496960 avail 3691974656 00000100:00100000:1.0:1787739051.471233:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305657480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874579665514368:12345-0@lo:13:osp-pre-1-1.0 Request processed in 74us (981us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.471239:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 348 00000100:00100000:1.0:1787739051.471281:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305656940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665514368:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1787739051.472138:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d1a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665512832:12345-0@lo:1000:osp_up1-0.0 Request processed in 1639us (6866us total) trans 4294968280 rc 0/0 00000100:00100000:0.0:1787739051.472147:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2446 00000100:00080000:0.0:1787739051.472182:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787739051.472188:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787739051.472191:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787739051.472194:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787739051.472318:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d1680 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665512832:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787739051.472500:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab2e6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x14ef:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.472516:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab2e6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14ef:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.472531:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab2e6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14ef:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.472549:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab2ed lrc: 3/0,1 mode: --/PW res: [0x200000400:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.472556:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab2ed lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.472563:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab2ed lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.472594:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ab2f4 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.472600:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ab2f4 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.472609:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ab2f4 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739051.472626:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d1680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665514496:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739051.472659:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739051.472682:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739051.472748:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665514496 00000100:00100000:1.0:1787739051.472754:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665514496 00000100:00100000:1.0:1787739051.472780:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2447 00000100:00100000:1.0:1787739051.472787:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d12c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:13793:x1874579665514496:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739051.472801:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739051.472814:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ab2fb lrc: 2/0,0 mode: --/PW res: [0x240000401:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab2f4 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.472833:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ab2fb lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab2f4 expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.472850:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ab2fb lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab2f4 expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.472858:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb4:0x0].0x0 (ffffa07433235700) 00010000:00010000:1.0:1787739051.472862:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739051.472866:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1ce00, rc 0) 00000100:00100000:1.0:1787739051.472897:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d12c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665514496:12345-0@lo:101:mdt00_004.0 Request processed in 110us (237us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.472903:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2447 00000100:00100000:2.0:1787739051.473119:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787739051.473129:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665514624:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787739051.473160:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739051.473190:0:10361:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665514624 00000100:00100000:2.0:1787739051.473199:0:10361:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665514624 00000100:00100000:2.0:1787739051.473227:0:10361:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 349 00000100:00100000:2.0:1787739051.473234:0:10361:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874579665514624:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787739051.473246:0:10361:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 25431488 pending 0 free 3923496960 avail 3691974656 00000100:00100000:2.0:1787739051.473278:0:10361:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874579665514624:12345-0@lo:13:osp-pre-1-0.0 Request processed in 45us (117us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.473284:0:10361:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 349 00000100:00100000:2.0:1787739051.473316:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665514624:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787739051.473389:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d1680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665514496:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739051.473403:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ab2f4 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab2fb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.473436:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ab2f4 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab2fb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739051.473458:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000eb279c9b reply state 00000000176f6c2d transno 4294983290 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab2ed lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.473470:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab2ed lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.473477:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb4:0x0].0x0 (ffffa0741590d200) 00010000:00010000:0.0:1787739051.473480:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739051.473483:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739051.473488:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ab2f4 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab2fb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.473495:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ab2f4 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab2fb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739051.473505:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xb4:0x0] 00000004:00010000:0.0:1787739051.473512:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000eb279c9b reply state 00000000176f6c2d transno 4294983290 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab2e6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14ef:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.473520:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab2e6 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x14ef:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.473525:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14ef:0x0].0x0 (ffffa0741590d500) 00010000:00010000:0.0:1787739051.473528:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739051.473530:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787739051.473534:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14ef:0x0]: returning size 4096 00010000:00010000:0.0:1787739051.473544:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab2d8 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.473550:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab2d8 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.473557:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab2d8 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.473563:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab2d8 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.473569:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739051.473573:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab2d8 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.473581:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab2d8 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.473590:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403eebd00) 00010000:00010000:0.0:1787739051.473592:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739051.473596:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab2d8 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.473601:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab2d8 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739051.473619:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000eb279c9b reply state 00000000176f6c2d transno 4294983290 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab2df lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.473627:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab2df lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.473633:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x1009fee1 (ffffa07403eeba00) 00010000:00010000:0.0:1787739051.473635:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739051.473637:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 2 00000100:00100000:0.0:1787739051.473702:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17397:x1874579652405120:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 12204us (12484us total) trans 4294983290 rc 0/0 00000100:00100000:0.0:1787739051.473710:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23124 00000100:00100000:3.0:1787739051.497794:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739051.498474:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652405376 00000100:00100000:0.0:1787739051.498485:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652405376 00000100:00100000:0.0:1787739051.498538:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23125 00000100:00100000:0.0:1787739051.498547:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fc3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17402:x1874579652405376:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:0.0:1787739051.498851:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fc3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17402:x1874579652405376:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 303us (1062us total) trans 0 rc -17/-17 00000100:00100000:0.0:1787739051.498858:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23125 00000100:00100000:2.0:1787739051.529552:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739051.529981:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652405504 00000100:00100000:0.0:1787739051.529991:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652405504 00000100:00100000:0.0:1787739051.530052:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23126 00000100:00100000:0.0:1787739051.530061:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17403:x1874579652405504:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739051.530080:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739051.530112:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab302 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0949f1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530180:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab309 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530194:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab309 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530206:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab309 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530217:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab310 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530229:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab310 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530234:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab310 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530321:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab317 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14ef:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530330:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab317 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14ef:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530339:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab317 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14ef:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739051.530455:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14ef:0x0]: returning size 4096 00000004:00200000:0.0:1787739051.530462:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:0.0:1787739051.530486:0:6523:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab317 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14ef:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530501:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab309 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530508:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab309 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530515:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab309 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530522:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab309 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530532:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739051.530537:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab309 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530544:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab309 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530556:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0741590da00) 00010000:00010000:0.0:1787739051.530559:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739051.530564:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab309 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530570:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab309 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530582:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab310 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530588:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab310 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530594:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab310 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530598:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739051.530601:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab310 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530608:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab310 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530618:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x1009fee1 (ffffa07403eeba00) 00010000:00010000:0.0:1787739051.530621:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739051.530624:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab310 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530629:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab310 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530655:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab302 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949f1 expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530674:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab317 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14ef:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949f1 expref: 16 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.530682:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14ef:0x0].0x0 (ffffa0741590d500) 00010000:00010000:0.0:1787739051.530684:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787739051.530686:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609bc00, rc 0) 00000100:00100000:0.0:1787739051.530749:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17403:x1874579652405504:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 688us (1210us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.531114:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23126 00000100:00100000:0.0:1787739051.535312:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739051.537960:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652405632 00000100:00100000:3.0:1787739051.537974:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652405632 00000100:00100000:3.0:1787739051.538044:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23127 00000100:00100000:3.0:1787739051.538054:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17403:x1874579652405632:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739051.538075:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739051.538093:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab31e lrc: 2/0,0 mode: --/CR res: [0x200000400:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0949f8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.538160:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab325 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.538174:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab325 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.538185:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab325 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739051.538233:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xb4:0x0]: returning size 4096 00000004:00200000:3.0:1787739051.538238:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739051.538274:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab31e lrc: 0/0,0 mode: --/CR res: [0x200000400:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949f8 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.538305:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab325 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949f8 expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.538315:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb4:0x0].0x0 (ffffa0741590d200) 00010000:00010000:3.0:1787739051.538318:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739051.538322:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa400, rc 0) 00000100:00100000:3.0:1787739051.538386:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17403:x1874579652405632:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 333us (3082us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.538394:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23127 00000100:00100000:3.0:1787739051.543617:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739051.543732:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652405760 00000100:00100000:1.0:1787739051.543740:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652405760 00000100:00100000:1.0:1787739051.543786:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23128 00000100:00100000:1.0:1787739051.543792:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061ff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17403:x1874579652405760:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739051.543811:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739051.543827:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab32c lrc: 2/0,0 mode: --/CR res: [0x240000401:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0949ff expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.543883:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab333 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.543901:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ab2fb lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab2f4 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.543908:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab333 lrc: 4/1,0 mode: --/PR res: [0x240000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.543930:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ab2fb lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ab2f4 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739051.543944:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305656940 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665514752:0@lo:104: 00000100:00100000:1.0:1787739051.544311:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.544354:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.544398:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665514752 00000100:00100000:3.0:1787739051.544403:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665514752 00000100:00100000:3.0:1787739051.544460:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1034 00000100:00100000:3.0:1787739051.544468:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6525:x1874579665514752:12345-0@lo:104: 00010000:00010000:3.0:1787739051.544482:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ab2f4 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab2fb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.544541:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739051.544547:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6525:x1874579665514752:12345-0@lo:104: Request processed in 80us (237us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.544553:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1034 00010000:00010000:3.0:1787739051.544573:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739051.544579:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ab2f4 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab2fb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.544587:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739051.544592:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ab2f4 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xb4:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ab2fb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739051.544600:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ab2f4 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xb4:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ab2fb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.544625:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ab2f4 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xb4:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab2fb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.544632:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739051.544651:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ab2f4 lrc: 1/0,0 mode: --/PW res: [0x240000401:0xb4:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab2fb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.544658:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ab2f4 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xb4:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab2fb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.544677:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739051.544679:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739051.544681:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739051.544692:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665514880:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739051.544721:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.544763:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665514880 00000100:00100000:3.0:1787739051.544767:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665514880 00010000:00010000:3.0:1787739051.544782:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ab2fb lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ab2f4 expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739051.544801:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1356 00000100:00100000:3.0:1787739051.544807:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fd680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+10:3650:x1874579665514880:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739051.544815:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787739051.545259:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305656940 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665514752:0@lo:104: 00010000:00010000:3.0:1787739051.545278:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab333 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.553705:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ab268 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab26f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739051.553723:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ab268 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab26f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.553772:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ab268 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab26f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.553779:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:3.0:1787739051.553833:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665515008:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:3.0:1787739051.553914:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739051.554006:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665515008 00000100:00100000:0.0:1787739051.554012:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665515008 00000100:00100000:0.0:1787739051.554055:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1155 00000100:00100000:0.0:1787739051.554062:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba2c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+7:3651:x1874579665515008:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:0.0:1787739051.554075:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739051.554092:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ab26f lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab268 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.554118:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xb98062fb (ffffa0741590db00) 00010000:00010000:0.0:1787739051.554130:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739051.554309:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ab268 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xb98062fb bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab26f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787739051.554325:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968280 is committed 00000001:00080000:2.0:1787739051.554342:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968279 is committed 00000020:00080000:2.0:1787739051.554349:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 329 batchid updated 00000020:00080000:2.0:1787739051.554361:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa0730688d540 refcount 2 committed 0 result 0 batchid 330 00000020:00080000:2.0:1787739051.554365:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c695a00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730688d6c0 00000020:00080000:2.0:1787739051.554368:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].323 00000020:00080000:2.0:1787739051.554369:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c695b00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073069bee00 00000020:00080000:2.0:1787739051.554372:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].323 00010000:00010000:2.0:1787739051.554393:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab1f8 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x159:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.554401:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab1f8 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x159:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.554408:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab1f8 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x159:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.554433:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739051.554437:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab1f8 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x159:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.554444:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab1f8 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x159:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.554454:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x159:0x0].0x0 (ffffa0730608bb00) 00010000:00010000:2.0:1787739051.554457:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739051.554461:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab1f8 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x159:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.554468:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab1f8 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x159:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.554485:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab1ff lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.554491:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab1ff lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.554496:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab1ff lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.554501:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aba00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739051.554504:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab1ff lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.554509:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab1ff lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.554515:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb1:0x0].0x0 (ffffa0730608b500) 00010000:00010000:2.0:1787739051.554518:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739051.554520:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab1ff lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.554525:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab1ff lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xb1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739051.554546:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 330 committed batchid 329 00000004:00080000:2.0:1787739051.554685:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730610ed80 with version 618 00000020:00080000:2.0:1787739051.554697:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 329 cancel update log [0x200000401:0x3:0x0].322: rc = 0 00000020:00080000:2.0:1787739051.554737:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 329 cancel update log [0x240000400:0x3:0x0].321: rc = 0 00000020:00080000:2.0:1787739051.554744:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 329 00010000:00010000:2.0:1787739051.554793:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ab2fb lrc: 0/0,0 mode: --/PW res: [0x240000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ab2f4 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.554803:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb4:0x0].0x0 (ffffa07433235700) 00010000:00010000:2.0:1787739051.554807:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab333 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.554817:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab333 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.554824:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab333 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.554846:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739051.554883:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fd680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+5:3650:x1874579665514880:12345-0@lo:103:ldlm_bl.0 Request processed in 10075us (10160us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.555134:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1356 00000100:00100000:2.0:1787739051.559887:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665514880:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739051.565513:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba2c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+6:3651:x1874579665515008:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 11440us (11588us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.565528:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1155 00000004:00080000:0.0:1787739051.566593:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 618 rpc_version 618 00000020:00100000:0.0:1787739051.566631:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739051.566640:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073060d0b40 x1874579665515264/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739051.566665:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0b40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665515264:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739051.566731:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739051.566779:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404780 going to sleep for 11 seconds 00000100:00100000:2.0:1787739051.566835:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665515264 00000100:00100000:2.0:1787739051.566842:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665515264 00000100:00100000:2.0:1787739051.566903:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2448 00000100:00100000:2.0:1787739051.566912:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d3840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:7467:x1874579665515264:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739051.567075:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d3840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665515264:12345-0@lo:1000:osp_up0-1.0 Request processed in 163us (343us total) trans 4294983291 rc 0/0 00000100:00100000:2.0:1787739051.567082:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2448 00000100:00100000:0.0:1787739051.567259:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0b40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665515264:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739051.568563:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665515008:0@lo:103:jbd2/dm-1-8.0 00010000:00010000:3.0:1787739051.568707:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab333 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.568718:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab333 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739051.568780:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xb4:0x0]: returning size 4096 00000004:00200000:3.0:1787739051.568785:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739051.568827:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab32c lrc: 0/0,0 mode: --/CR res: [0x240000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ff expref: 9 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.568845:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab333 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ff expref: 8 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.568854:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb4:0x0].0x0 (ffffa07433235700) 00010000:00010000:3.0:1787739051.568857:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739051.568861:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d000, rc 0) 00000100:00100000:3.0:1787739051.568928:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061ff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17403:x1874579652405760:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 25134us (25320us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.568937:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23128 00000100:00100000:3.0:1787739051.577712:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739051.578204:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652406016 00000100:00100000:2.0:1787739051.578215:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652406016 00000100:00100000:2.0:1787739051.578324:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23129 00000100:00100000:2.0:1787739051.578336:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fd2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17403:x1874579652406016:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:2.0:1787739051.578395:0:6524:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739051.578451:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab317 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14ef:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949f1 expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.578468:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14ef:0x0].0x0 (ffffa0741590d500) 00010000:00010000:2.0:1787739051.578473:0:6524:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739051.578496:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab33a lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.578508:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab33a lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.578518:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab33a lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.578529:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab341 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.578535:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab341 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.578542:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab341 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.578621:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab348 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14ef:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.578629:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab348 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14ef:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.578636:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab348 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14ef:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.578653:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab34f lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.578663:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab325 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949f8 expref: 16 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.578672:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab34f lrc: 4/0,1 mode: --/EX res: [0x200000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.578702:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab325 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949f8 expref: 16 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.578717:0:6524:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab325 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949f8 expref: 16 pid: 13793 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.578731:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383d2c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665515392:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739051.578822:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739051.583596:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739051.583644:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739051.583672:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739051.583702:0:6518:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652406144 00000100:00100000:0.0:1787739051.583711:0:6518:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652406144 00000100:00100000:0.0:1787739051.583773:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 445 00000100:00100000:0.0:1787739051.583784:0:6518:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410c85c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:53427813-ebb7-48fc-94b6-ef03330d7ee5+10:2407:x1874579652406144:12345-192.168.203.15@tcp:400:kworker.0 00000100:00100000:0.0:1787739051.583853:0:6518:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410c85c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:53427813-ebb7-48fc-94b6-ef03330d7ee5+10:2407:x1874579652406144:12345-192.168.203.15@tcp:400:kworker.0 Request processed in 71us (208us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.583861:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 445 00000100:00100000:0.0:1787739051.583904:0:14679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652406272 00000100:00100000:0.0:1787739051.583909:0:14679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652406272 00000100:00100000:0.0:1787739051.583946:0:14679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652406400 00000100:00100000:0.0:1787739051.583949:0:14679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652406400 00000100:00100000:0.0:1787739051.583985:0:14679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 341 00000100:00100000:0.0:1787739051.583992:0:14679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2407:x1874579652406272:12345-192.168.203.15@tcp:400:kworker.0 00000100:00100000:0.0:1787739051.584023:0:14679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2407:x1874579652406272:12345-192.168.203.15@tcp:400:kworker.0 Request processed in 32us (437us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.584029:0:14679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 341 00000100:00100000:0.0:1787739051.584041:0:14679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 342 00000100:00100000:0.0:1787739051.584047:0:14679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d03c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2407:x1874579652406400:12345-192.168.203.15@tcp:400:kworker.0 00000100:00100000:0.0:1787739051.584069:0:14679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d03c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2407:x1874579652406400:12345-192.168.203.15@tcp:400:kworker.0 Request processed in 23us (396us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.584075:0:14679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 342 00000100:00080000:0.0:1787739051.584107:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787739051.584111:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787739051.594829:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383d2c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665515392:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739051.594857:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab34f lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739051.595149:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739051.595220:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652406528 00000100:00100000:2.0:1787739051.595226:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652406528 00010000:00010000:2.0:1787739051.595259:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab325 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949f8 expref: 17 pid: 13793 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.595304:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1357 00000100:00100000:2.0:1787739051.595313:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652406528:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739051.595328:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739051.595335:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab325 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949f8 expref: 17 pid: 13793 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.595345:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab325 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949f8 expref: 17 pid: 13793 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.595366:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab325 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949f8 expref: 17 pid: 13793 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.595380:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb4:0x0].0x0 (ffffa0741590d200) 00010000:00010000:2.0:1787739051.595385:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab34f lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.595394:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab34f lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.595402:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab34f lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.595437:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739051.595481:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:2409:x1874579652406528:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 169us (334us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.595489:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1357 00010000:00010000:2.0:1787739051.595508:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab34f lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.595514:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab34f lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.595552:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ab356 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.595561:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ab356 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.595570:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ab356 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.595587:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383d2c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665515520:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1787739051.595625:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739051.595650:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739051.595663:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665515520 00000100:00100000:2.0:1787739051.595666:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665515520 00000100:00100000:2.0:1787739051.595690:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2449 00000100:00100000:2.0:1787739051.595697:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+5:6524:x1874579665515520:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739051.595709:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739051.595721:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab35d lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab356 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.595739:0:8447:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab333 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ff expref: 7 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.595758:0:8447:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab35d lrc: 4/0,0 mode: --/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab356 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.595780:0:8447:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab333 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ff expref: 7 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.595797:0:8447:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab333 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ff expref: 7 pid: 6525 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.595807:0:8447:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383ed00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8447:1874579665515648:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739051.595841:0:8447:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739051.604564:0:8447:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383ed00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8447:1874579665515648:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739051.604603:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab35d lrc: 3/0,0 mode: --/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab356 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.604617:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb4:0x0].0x0 (ffffa07433235700) 00010000:00010000:0.0:1787739051.604620:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739051.604623:0:8447:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab35d lrc: 3/0,0 mode: --/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab356 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.604633:0:8447:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab35d lrc: 3/0,0 mode: --/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab356 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.604645:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d000, rc 0) 00000100:00100000:0.0:1787739051.604718:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6524:x1874579665515520:12345-0@lo:101:mdt00_001.0 Request processed in 9021us (9092us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.604728:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2449 00000100:00100000:1.0:1787739051.604858:0:6524:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968280 gen 2 00000004:00080000:1.0:1787739051.604865:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968275 committed transno 4294968280 00000020:00080000:1.0:1787739051.604880:0:6524:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07410b4db40 refcount 2 committed 0 result 0 batchid 339 00000020:00080000:1.0:1787739051.604884:0:6524:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa074051e2700 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07410b4d0c0 00000020:00080000:1.0:1787739051.604888:0:6524:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].336 00000020:00080000:1.0:1787739051.604890:0:6524:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa074051e2d00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304f1e400 00000020:00080000:1.0:1787739051.604893:0:6524:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].336 00000004:00080000:1.0:1787739051.604920:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968277 committed transno 4294968280 00000004:00080000:1.0:1787739051.604930:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968278 committed transno 4294968280 00000004:00080000:1.0:1787739051.604936:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968280 committed transno 4294968280 00000100:00100000:1.0:1787739051.604947:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383d2c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665515520:0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739051.604964:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ab356 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab35d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.604984:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ab356 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab35d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00080000:0.0:1787739051.605364:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 339 committed batchid 338 00000020:00080000:0.0:1787739051.605446:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 338 00000100:00100000:1.0:1787739051.605939:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739051.606248:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652406656 00000100:00100000:0.0:1787739051.606256:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652406656 00010000:00010000:0.0:1787739051.606300:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab333 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ff expref: 8 pid: 6525 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739051.606334:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1358 00000100:00100000:0.0:1787739051.606344:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41de00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2408:x1874579652406656:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739051.606359:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739051.606366:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab333 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ff expref: 8 pid: 6525 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.606379:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab333 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ff expref: 8 pid: 6525 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.606399:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab333 lrc: 0/0,0 mode: --/PR res: [0x240000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949ff expref: 8 pid: 6525 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.606431:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb4:0x0].0x0 (ffffa07433235700) 00010000:00010000:0.0:1787739051.606437:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab35d lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab356 expref: 5 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.606447:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab35d lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab356 expref: 5 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.606456:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab35d lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab356 expref: 5 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.606485:0:10340:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab35d lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab356 expref: 5 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739051.606498:0:10340:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0f00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665515776:0@lo:105: 00000100:00100000:0.0:1787739051.606542:0:10340:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739051.606571:0:10340:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404000 going to sleep for 11 seconds 00000100:00100000:0.0:1787739051.606593:0:6507:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787739051.606603:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665515776 00000100:00100000:0.0:1787739051.606606:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665515776 00000100:00100000:0.0:1787739051.606633:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1035 00000100:00100000:0.0:1787739051.606640:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07436dfe450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665515776:12345-0@lo:105: 00010000:00010000:0.0:1787739051.606649:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ab356 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab35d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.606658:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ab356 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab35d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.606667:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ab356 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab35d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.606674:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ab356 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab35d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.606681:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ab356 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab35d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.606689:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ab356 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ab35d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.606706:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633c000) 00000100:00100000:0.0:1787739051.606739:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07436dfe450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665515776:12345-0@lo:105: Request processed in 98us (196us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.607040:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1035 00000100:00100000:0.0:1787739051.607115:0:10340:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0f00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665515776:0@lo:105: 00010000:00010000:0.0:1787739051.607128:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739051.607167:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41de00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652406656:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 824us (1229us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.607175:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1358 00010000:00010000:0.0:1787739051.607638:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ab356 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab35d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.607648:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ab356 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab35d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739051.609662:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665515904:0@lo:403:mdt00_001.0 00000100:00100000:0.0:1787739051.609733:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739051.609784:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404000 going to sleep for 11 seconds 00000100:00100000:1.0:1787739051.609881:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665515904 00000100:00100000:1.0:1787739051.609888:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665515904 00000100:00100000:1.0:1787739051.609938:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2450 00000100:00100000:1.0:1787739051.609947:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d0780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6524:x1874579665515904:12345-0@lo:403:mdt00_001.0 00000100:00100000:1.0:1787739051.611607:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d0780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6524:x1874579665515904:12345-0@lo:403:mdt00_001.0 Request processed in 1657us (1870us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.611619:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2450 00000100:00100000:0.0:1787739051.612233:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665515904:0@lo:403:mdt00_001.0 00000040:00080000:0.0:1787739051.615799:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].338, 952 off520040 00000040:00080000:0.0:1787739051.617641:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].338, 1032 off563568 00000004:00080000:0.0:1787739051.617652:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07410404000 with version 644 00000004:00080000:1.0:1787739051.617840:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 644 rpc_version 644 00000020:00100000:1.0:1787739051.617943:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739051.617954:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743c41f0c0 x1874579665516160/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1992/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739051.618040:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41f0c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665516160:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739051.618118:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.618189:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc780 going to sleep for 11 seconds 00000100:00100000:0.0:1787739051.625464:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665516160 00000100:00100000:0.0:1787739051.625480:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665516160 00000100:00100000:0.0:1787739051.625540:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2451 00000100:00100000:0.0:1787739051.625551:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665516160:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739051.626144:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665516160:12345-0@lo:1000:osp_up1-0.0 Request processed in 595us (8024us total) trans 4294968281 rc 0/0 00000100:00100000:0.0:1787739051.626154:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2451 00000100:00100000:1.0:1787739051.629687:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41f0c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665516160:0@lo:1000:osp_up1-0.0 00000004:00200000:0.0:1787739051.632530:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14ef:0x0]: returning size 4096 00010000:00010000:1.0:1787739051.642194:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab34f lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642212:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab34f lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642219:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab34f lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642226:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739051.642230:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab34f lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642238:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab34f lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642254:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb4:0x0].0x0 (ffffa0741590d200) 00010000:00010000:1.0:1787739051.642258:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739051.642263:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab34f lrc: 2/0,0 mode: --/EX res: [0x200000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642269:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab34f lrc: 0/0,0 mode: --/EX res: [0x200000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642296:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ab356 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab35d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642305:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ab356 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab35d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642313:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ab356 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab35d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642339:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739051.642347:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xb4:0x0] 00010000:00010000:1.0:1787739051.642355:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab348 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14ef:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642361:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab348 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x14ef:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642367:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab348 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14ef:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642372:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633cc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739051.642375:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab348 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14ef:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642380:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab348 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x14ef:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642391:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14ef:0x0].0x0 (ffffa0741590d500) 00010000:00010000:1.0:1787739051.642394:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739051.642397:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab348 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14ef:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642403:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab348 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14ef:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642447:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab33a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642455:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab33a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642461:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab33a lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642466:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab33a lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642471:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739051.642473:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab33a lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642480:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab33a lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642488:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403eebf00) 00010000:00010000:1.0:1787739051.642490:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739051.642493:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab33a lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642503:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab33a lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739051.642519:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000083e15bc0 reply state 00000000accf75b7 transno 4294983292 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab341 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642531:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab341 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.642539:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x1009fee1 (ffffa07403eeba00) 00010000:00010000:1.0:1787739051.642542:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739051.642545:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787739051.642633:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fd2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17403:x1874579652406016:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 64295us (64925us total) trans 4294983292 rc 0/0 00000100:00100000:1.0:1787739051.642641:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23129 00010000:00010000:3.0:1787739051.642870:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739051.642878:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ab356 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab35d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.642886:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739051.642890:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ab356 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab35d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739051.642904:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ab356 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab35d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.642934:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ab356 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab35d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.642940:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739051.642965:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ab356 lrc: 1/0,0 mode: --/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab35d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.642982:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ab356 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab35d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.642998:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739051.643000:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739051.643002:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739051.643369:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fcb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665516288:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739051.643447:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.644138:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665516288 00000100:00100000:3.0:1787739051.644145:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665516288 00000100:00100000:3.0:1787739051.644211:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1156 00000100:00100000:3.0:1787739051.644221:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+7:3652:x1874579665516288:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739051.644233:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739051.644252:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab35d lrc: 0/0,0 mode: --/EX res: [0x240000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab356 expref: 7 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.644267:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb4:0x0].0x0 (ffffa07433235700) 00010000:00010000:3.0:1787739051.644275:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739051.644335:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+6:3652:x1874579665516288:12345-0@lo:103:ldlm_bl.0 Request processed in 115us (888us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.644343:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1156 00000100:00100000:1.0:1787739051.647041:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fcb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665516288:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739051.700526:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739051.700605:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652406784 00000100:00100000:1.0:1787739051.700616:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652406784 00000100:00100000:1.0:1787739051.700676:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23130 00000100:00100000:1.0:1787739051.700687:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17404:x1874579652406784:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739051.700711:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739051.700734:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ab364 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094a06 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.700806:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab36b lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.700821:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab36b lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.700834:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab36b lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.700851:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab372 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.700860:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab372 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.700867:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab372 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.700903:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab36b lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.700910:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab36b lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.700917:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab36b lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.700923:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab36b lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.700929:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739051.700932:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab36b lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.700940:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab36b lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.700951:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae900) 00010000:00010000:1.0:1787739051.700954:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739051.700958:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab36b lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.700964:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab36b lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.700984:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab372 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.700991:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab372 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.700996:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab372 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.701002:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739051.701004:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab372 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.701009:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab372 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.701017:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x99a77dc0 (ffffa07305eae400) 00010000:00010000:1.0:1787739051.701020:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739051.701023:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab372 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.701028:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab372 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.701062:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ab364 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a06 expref: 18 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.701072:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae900) 00010000:00010000:1.0:1787739051.701075:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739051.701078:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ab364 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a06 expref: 18 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.701090:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1dc00, rc 0) 00000100:00100000:1.0:1787739051.701152:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17404:x1874579652406784:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 467us (634us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739051.701160:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23130 00000100:00100000:1.0:1787739051.709896:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739051.709968:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652406912 00000100:00100000:3.0:1787739051.710040:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652406912 00000100:00100000:3.0:1787739051.710113:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23131 00000100:00100000:3.0:1787739051.710123:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c382580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17404:x1874579652406912:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739051.710144:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739051.710166:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab379 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094a0d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.710229:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab380 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.710242:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab380 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.710253:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab380 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.710266:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab387 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.710273:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab387 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.710280:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab387 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.711133:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab380 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.711144:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab380 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.711152:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab380 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.711160:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab380 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.711167:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739051.711172:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab380 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.711180:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab380 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.711194:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2700) 00010000:00010000:3.0:1787739051.711197:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739051.711201:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab380 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.711208:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab380 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.711223:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab387 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.711230:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab387 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.711236:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab387 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.711241:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739051.711245:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab387 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.711251:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab387 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.711259:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x99a77dc0 (ffffa074331c2d00) 00010000:00010000:3.0:1787739051.711262:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739051.711265:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab387 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.711270:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab387 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.711315:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab379 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a0d expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.711327:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2700) 00010000:00010000:3.0:1787739051.711329:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739051.711333:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab379 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a0d expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.711345:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aac00, rc 0) 00000100:00100000:3.0:1787739051.711409:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c382580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17404:x1874579652406912:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 1286us (1522us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739051.711432:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23131 00000100:00100000:3.0:1787739051.722808:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739051.722881:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652407040 00000100:00100000:1.0:1787739051.722890:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652407040 00000100:00100000:1.0:1787739051.723795:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23132 00000100:00100000:1.0:1787739051.723809:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061ff840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17404:x1874579652407040:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739051.723829:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739051.723851:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ab38e lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094a14 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.723909:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab395 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.723923:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab395 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.723939:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab395 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739051.723996:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:1.0:1787739051.724001:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739051.724036:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ab38e lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a14 expref: 19 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.724054:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab395 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a14 expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.724062:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae900) 00010000:00010000:1.0:1787739051.724065:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739051.724068:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d600, rc 0) 00000100:00100000:1.0:1787739051.724135:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061ff840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17404:x1874579652407040:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 330us (1334us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.724142:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23132 00000100:00100000:2.0:1787739051.734553:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739051.735151:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652407296 00000100:00100000:1.0:1787739051.735160:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652407296 00000100:00100000:1.0:1787739051.735218:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23133 00000100:00100000:1.0:1787739051.735229:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17404:x1874579652407296:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:1.0:1787739051.735343:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f6d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665516416:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739051.735396:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.735464:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739051.735481:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665516416 00000100:00100000:1.0:1787739051.735485:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665516416 00000100:00100000:1.0:1787739051.735513:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2452 00000100:00100000:1.0:1787739051.735521:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f7840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665516416:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739051.735588:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f7840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665516416:12345-0@lo:1000:mdt00_000.0 Request processed in 67us (191us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.735595:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2452 00000100:00100000:1.0:1787739051.735636:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f6d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665516416:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739051.735681:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f6d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665516544:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739051.735704:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.735719:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739051.736386:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665516544 00000100:00100000:2.0:1787739051.736393:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665516544 00000100:00100000:2.0:1787739051.737134:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2453 00000100:00100000:2.0:1787739051.737147:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665516544:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739051.737255:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665516544:12345-0@lo:1000:mdt00_000.0 Request processed in 112us (1550us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739051.737263:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2453 00000100:00100000:2.0:1787739051.737315:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f6d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665516544:0@lo:1000:mdt00_000.0 00010000:00010000:2.0:1787739051.737382:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1ab39c lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.737394:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1ab39c lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.737404:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1ab39c lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.737439:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383da40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665516672:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1787739051.737471:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739051.737492:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739051.737507:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665516672 00000100:00100000:2.0:1787739051.737511:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665516672 00000100:00100000:2.0:1787739051.737540:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2454 00000100:00100000:2.0:1787739051.737547:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665516672:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739051.737555:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739051.737570:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ab3a3 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab39c expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.737595:0:8448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab395 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a14 expref: 17 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.737605:0:8448:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ab3a3 lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab39c expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.737635:0:8448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab395 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a14 expref: 17 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.737648:0:8448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab395 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a14 expref: 17 pid: 6524 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.737659:0:8448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383ed00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8448:1874579665516800:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739051.737719:0:8448:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ecc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739051.759501:0:8448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383ed00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8448:1874579665516800:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739051.759554:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ab3a3 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab39c expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.759567:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae900) 00010000:00010000:2.0:1787739051.759571:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739051.759575:0:8448:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ab3a3 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab39c expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.759591:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c200, rc 0) 00000100:00100000:2.0:1787739051.759694:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665516672:12345-0@lo:101:mdt00_000.0 Request processed in 22147us (22222us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.759702:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2454 00000100:00100000:0.0:1787739051.759952:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383da40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665516672:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739051.759980:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1ab39c lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab3a3 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.759991:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1ab39c lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab3a3 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.764000:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739051.765470:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652407424 00000100:00100000:3.0:1787739051.765497:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652407424 00010000:00010000:3.0:1787739051.765568:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab395 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a14 expref: 18 pid: 6524 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739051.765657:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1359 00000100:00100000:3.0:1787739051.765667:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383ed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2409:x1874579652407424:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739051.765689:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739051.765696:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab395 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a14 expref: 18 pid: 6524 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.765725:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab395 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a14 expref: 18 pid: 6524 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.765766:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab395 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a14 expref: 18 pid: 6524 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.765789:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae900) 00010000:00010000:3.0:1787739051.765804:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ab3a3 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab39c expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.765815:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ab3a3 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab39c expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.765825:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ab3a3 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab39c expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.765878:0:6509:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ab3a3 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab39c expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739051.765905:0:6509:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fcb40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665516928:0@lo:105: 00000100:00100000:3.0:1787739051.765986:0:6509:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739051.766048:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665516928 00000100:00100000:0.0:1787739051.766053:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665516928 00000100:00100000:3.0:1787739051.766081:0:6509:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4540 going to sleep for 11 seconds 00000100:00100000:0.0:1787739051.766091:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1036 00000100:00100000:0.0:1787739051.766113:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fe1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665516928:12345-0@lo:105: 00010000:00010000:0.0:1787739051.766124:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1ab39c lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab3a3 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.766134:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1ab39c lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab3a3 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.766143:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1ab39c lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab3a3 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.766151:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1ab39c lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab3a3 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.766158:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1ab39c lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab3a3 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.766165:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1ab39c lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ab3a3 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.766190:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633c400) 00000100:00100000:0.0:1787739051.766231:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fe1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665516928:12345-0@lo:105: Request processed in 133us (244us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.766239:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1036 00000100:00100000:0.0:1787739051.766279:0:6509:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fcb40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665516928:0@lo:105: 00010000:00010000:0.0:1787739051.766305:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739051.766343:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383ed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652407424:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 678us (2361us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.766350:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1359 00010000:00010000:3.0:1787739051.770805:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1ab39c lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab3a3 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.770826:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1ab39c lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab3a3 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739051.772124:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739051.772177:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab3aa lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.772212:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab3aa lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.772226:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab3aa lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739051.772274:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fde00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665517056:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739051.772431:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.772501:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739051.774256:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665517056 00000100:00100000:2.0:1787739051.774268:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665517056 00000100:00100000:2.0:1787739051.774382:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2455 00000100:00100000:2.0:1787739051.774393:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061ff0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665517056:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739051.774451:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739051.774484:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab3b1 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab3aa expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.774526:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab3b1 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab3aa expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.774554:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab3b1 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab3aa expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.774577:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x99a77dc0 (ffffa07403eebe00) 00010000:00010000:2.0:1787739051.774580:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739051.774584:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c000, rc 0) 00000100:00100000:2.0:1787739051.774694:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061ff0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665517056:12345-0@lo:101:mdt00_000.0 Request processed in 302us (2288us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.774719:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2455 00000100:00100000:0.0:1787739051.774790:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fde00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665517056:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739051.774811:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab3aa lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab3b1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.774825:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab3aa lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab3b1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739051.774847:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:0.0:1787739051.774894:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d1a40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665517184:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739051.774930:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739051.774957:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d600 going to sleep for 11 seconds 00000100:00100000:1.0:1787739051.775939:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665517184 00000100:00100000:1.0:1787739051.775947:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665517184 00000100:00100000:1.0:1787739051.776005:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2456 00000100:00100000:1.0:1787739051.776014:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d21c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665517184:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739051.776130:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d21c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665517184:12345-0@lo:1000:mdt00_000.0 Request processed in 117us (1199us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.776149:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2456 00000100:00100000:0.0:1787739051.776215:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d1a40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665517184:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739051.776266:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d1a40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665517312:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739051.776294:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739051.776317:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739051.776576:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665517312 00000100:00100000:2.0:1787739051.776581:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665517312 00000100:00100000:2.0:1787739051.776643:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2457 00000100:00100000:2.0:1787739051.776651:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665517312:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739051.776857:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665517312:12345-0@lo:1000:mdt00_000.0 Request processed in 206us (561us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739051.776863:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2457 00000100:00100000:2.0:1787739051.776931:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d1a40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665517312:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739051.777065:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383c780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665517440:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739051.777118:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739051.777136:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739051.777162:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665517440 00000100:00100000:2.0:1787739051.777166:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665517440 00000100:00100000:2.0:1787739051.777216:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2458 00000100:00100000:2.0:1787739051.777223:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665517440:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739051.777295:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665517440:12345-0@lo:1000:mdt00_000.0 Request processed in 72us (176us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739051.777314:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2458 00000100:00100000:2.0:1787739051.777355:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383c780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665517440:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739051.777505:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383c780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665517568:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739051.777543:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739051.777571:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739051.777594:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665517568 00000100:00100000:2.0:1787739051.777598:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665517568 00000100:00100000:2.0:1787739051.777629:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2459 00000100:00100000:2.0:1787739051.777635:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665517568:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739051.777700:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665517568:12345-0@lo:1000:mdt00_000.0 Request processed in 64us (157us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.777706:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2459 00000100:00100000:2.0:1787739051.777751:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383c780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665517568:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739051.777794:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be43c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665517696:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739051.777826:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739051.777852:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739051.777862:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665517696 00000100:00100000:2.0:1787739051.777865:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665517696 00000100:00100000:2.0:1787739051.777895:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2460 00000100:00100000:2.0:1787739051.777901:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665517696:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739051.777978:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665517696:12345-0@lo:1000:mdt00_000.0 Request processed in 67us (141us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.777984:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2460 00000100:00100000:2.0:1787739051.778016:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be43c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665517696:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739051.778056:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be43c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665517824:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739051.778086:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739051.778123:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739051.778135:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665517824 00000100:00100000:2.0:1787739051.778138:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665517824 00000100:00100000:2.0:1787739051.778173:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2461 00000100:00100000:2.0:1787739051.778192:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665517824:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739051.778266:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665517824:12345-0@lo:1000:mdt00_000.0 Request processed in 87us (179us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.778284:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2461 00000100:00100000:2.0:1787739051.778320:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be43c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665517824:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739051.778376:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be43c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665517952:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739051.778445:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739051.778460:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739051.778485:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665517952 00000100:00100000:2.0:1787739051.778489:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665517952 00000100:00100000:2.0:1787739051.778527:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2462 00000100:00100000:2.0:1787739051.778534:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665517952:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739051.778631:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665517952:12345-0@lo:1000:mdt00_000.0 Request processed in 86us (188us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.778638:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2462 00000100:00100000:2.0:1787739051.778677:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be43c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665517952:0@lo:1000:mdt00_000.0 00000040:00080000:2.0:1787739051.779953:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].325, 2168 off496520 00000040:00080000:2.0:1787739051.780087:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].325, 2368 off542216 00000004:00080000:2.0:1787739051.781286:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730610ecc0 with version 619 00000004:00080000:1.0:1787739051.782048:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 619 rpc_version 619 00000020:00100000:1.0:1787739051.782176:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739051.782322:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074074f61c0 x1874579665518208/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739051.782749:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665518208:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739051.783061:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739051.783289:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665518208 00000100:00100000:2.0:1787739051.783297:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665518208 00000100:00100000:1.0:1787739051.783440:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcca80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739051.783636:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2463 00000100:00100000:2.0:1787739051.783646:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f6d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665518208:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739051.789744:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f6d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665518208:12345-0@lo:1000:osp_up0-1.0 Request processed in 6093us (6676us total) trans 4294983293 rc 0/0 00000100:00100000:2.0:1787739051.789766:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2463 00000100:00100000:3.0:1787739051.793650:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665518208:0@lo:1000:osp_up0-1.0 00010000:00010000:2.0:1787739051.794681:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab3b8 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x15a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.794713:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab3b8 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x15a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.794733:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab3b8 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x15a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.794753:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab3bf lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.794760:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab3bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.794766:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab3bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.794807:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab3c6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.794823:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab3c6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.794832:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab3c6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.794862:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be43c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665518336:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1787739051.794965:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739051.795009:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739051.795046:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665518336 00000100:00100000:2.0:1787739051.795052:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665518336 00000100:00100000:2.0:1787739051.795129:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2464 00000100:00100000:2.0:1787739051.795148:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665518336:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739051.795184:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739051.795219:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab3cd lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab3c6 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.795268:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab3cd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab3c6 expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.795280:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab3cd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab3c6 expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.795288:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb2:0x0].0x0 (ffffa07403eebf00) 00010000:00010000:2.0:1787739051.795305:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739051.795308:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633de00, rc 0) 00000100:00100000:2.0:1787739051.795399:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665518336:12345-0@lo:101:mdt00_000.0 Request processed in 248us (428us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.795406:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2464 00000100:00100000:2.0:1787739051.795500:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be43c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665518336:0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739051.795528:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab3c6 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab3cd expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.795538:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab3c6 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab3cd expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739051.795587:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b3008f80 reply state 00000000d1f3e499 transno 4294968282 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab3bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.795601:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab3bf lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.795626:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb2:0x0].0x0 (ffffa07403eeb300) 00010000:00010000:2.0:1787739051.795629:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739051.795632:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787739051.795647:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab3c6 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab3cd expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.795655:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab3c6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab3cd expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739051.795672:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xb2:0x0] 00000004:00010000:2.0:1787739051.795679:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b3008f80 reply state 00000000d1f3e499 transno 4294968282 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab3b8 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x15a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.795686:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab3b8 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x15a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.795692:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15a:0x0].0x0 (ffffa07403eebb00) 00010000:00010000:2.0:1787739051.795694:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739051.795696:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:2.0:1787739051.795711:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x15a:0x0]: returning size 4096 00000004:00080000:2.0:1787739051.795718:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739051.795725:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1ab39c lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab3a3 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.795744:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1ab39c lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab3a3 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.795750:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1ab39c lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab3a3 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.795772:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739051.795776:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739051.795780:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab3aa lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab3b1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.795788:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab3aa lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab3b1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.795884:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17404:x1874579652407296:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 60655us (61348us total) trans 4294968282 rc 0/0 00000100:00100000:2.0:1787739051.795902:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23133 00010000:00010000:2.0:1787739051.795933:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739051.795939:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1ab39c lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab3a3 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.795957:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739051.795963:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1ab39c lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab3a3 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739051.795972:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1ab39c lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab3a3 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.796007:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1ab39c lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab3a3 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.796028:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739051.796065:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1ab39c lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab3a3 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.796073:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1ab39c lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab3a3 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.796110:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739051.796112:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739051.796114:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739051.796138:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665518464:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739051.796186:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739051.796672:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665518464 00000100:00100000:1.0:1787739051.796679:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665518464 00000100:00100000:1.0:1787739051.796735:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1157 00000100:00100000:1.0:1787739051.796756:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f752c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3653:x1874579665518464:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739051.796767:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739051.796791:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ab3a3 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab39c expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.796805:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae900) 00010000:00010000:1.0:1787739051.796811:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739051.796856:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f752c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+10:3653:x1874579665518464:12345-0@lo:103:ldlm_bl.0 Request processed in 114us (670us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.796872:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1157 00000100:00100000:2.0:1787739051.806800:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665518464:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739051.836810:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739051.842331:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652407680 00000100:00100000:3.0:1787739051.842344:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652407680 00000100:00100000:3.0:1787739051.842439:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23134 00000100:00100000:3.0:1787739051.842452:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17405:x1874579652407680:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:3.0:1787739051.842622:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17405:x1874579652407680:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 172us (5821us total) trans 0 rc -17/-17 00000100:00100000:3.0:1787739051.842630:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23134 00000100:00100000:0.0:1787739051.894165:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739051.894236:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652407808 00000100:00100000:3.0:1787739051.894245:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652407808 00000100:00100000:3.0:1787739051.894298:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23135 00000100:00100000:3.0:1787739051.894307:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17406:x1874579652407808:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739051.894360:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739051.894379:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab3d4 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094a1b expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.894461:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab3db lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.894474:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab3db lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.894484:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab3db lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.894493:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab3e2 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.894503:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab3b1 lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab3aa expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.894511:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab3e2 lrc: 4/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.894548:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab3b1 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ab3aa expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739051.894562:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fc780 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665518592:0@lo:104: 00000100:00100000:3.0:1787739051.894607:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739051.894658:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665518592 00000100:00100000:0.0:1787739051.894663:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665518592 00000100:00100000:0.0:1787739051.894689:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1037 00000100:00100000:0.0:1787739051.894695:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fd680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665518592:12345-0@lo:104: 00010000:00010000:0.0:1787739051.894707:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab3aa lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab3b1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739051.894784:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4540 going to sleep for 11 seconds 00010000:00010000:0.0:1787739051.895326:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787739051.895333:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fd680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665518592:12345-0@lo:104: Request processed in 637us (725us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.895341:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1037 00000100:00100000:3.0:1787739051.895397:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fc780 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665518592:0@lo:104: 00010000:00010000:3.0:1787739051.895429:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab3e2 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.895481:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739051.895488:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab3aa lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab3b1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.895503:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aac00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739051.895510:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab3aa lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ab3b1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739051.895519:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab3aa lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ab3b1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.895550:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab3aa lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab3b1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.895557:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739051.895580:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab3aa lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab3b1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.895587:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab3aa lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab3b1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739051.895609:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665518720:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739051.895610:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739051.895613:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739051.895616:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739051.895644:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739051.895739:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665518720 00000100:00100000:0.0:1787739051.895744:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665518720 00010000:00010000:0.0:1787739051.895763:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab3b1 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ab3aa expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739051.895790:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1360 00000100:00100000:0.0:1787739051.895796:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fc780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+10:3650:x1874579665518720:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739051.895809:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:3.0:1787739051.898817:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983293 is committed 00000001:00080000:3.0:1787739051.898852:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983292 is committed 00000020:00080000:3.0:1787739051.898862:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 339 batchid updated 00000020:00080000:3.0:1787739051.898872:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa0730610eb40 refcount 2 committed 0 result 0 batchid 340 00000020:00080000:3.0:1787739051.898876:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2800 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730610e3c0 00000020:00080000:3.0:1787739051.898880:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].337 00000020:00080000:3.0:1787739051.898883:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2c00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073043b1c00 00000020:00080000:3.0:1787739051.898885:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].337 00010000:00010000:0.0:1787739051.898985:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab2df lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.898996:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab2df lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899004:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab2df lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899010:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633ce00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739051.899015:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab2df lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899024:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab2df lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899040:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x1009fee1 (ffffa07403eeba00) 00010000:00010000:0.0:1787739051.899043:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739051.899048:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab2df lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899053:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab2df lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899064:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab2e6 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x14ef:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899068:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab2e6 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x14ef:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899071:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab2e6 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x14ef:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899075:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609be00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739051.899077:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab2e6 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x14ef:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899080:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab2e6 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x14ef:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899085:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14ef:0x0].0x0 (ffffa0741590d500) 00010000:00010000:0.0:1787739051.899086:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739051.899088:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab2e6 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x14ef:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899091:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab2e6 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x14ef:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899120:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab2ed lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899126:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab2ed lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899131:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab2ed lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899136:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739051.899139:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab2ed lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899146:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab2ed lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899162:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb4:0x0].0x0 (ffffa0741590d200) 00010000:00010000:0.0:1787739051.899165:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739051.899168:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab2ed lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899174:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab2ed lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899191:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab341 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899197:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab341 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899203:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab341 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899208:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739051.899211:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab341 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899216:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab341 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899224:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x1009fee1 (ffffa07403eeba00) 00010000:00010000:0.0:1787739051.899227:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739051.899230:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab341 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899236:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab341 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x1009fee1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:0.0:1787739051.899263:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 340 committed batchid 339 00000004:00080000:0.0:1787739051.899370:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07410b4dcc0 with version 645 00000020:00080000:0.0:1787739051.899434:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 339 cancel update log [0x240000401:0x3:0x0].336: rc = 0 00000020:00080000:0.0:1787739051.899484:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 339 cancel update log [0x200000400:0x3:0x0].336: rc = 0 00000020:00080000:0.0:1787739051.899494:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 339 00000004:00080000:3.0:1787739051.899498:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 645 rpc_version 645 00000020:00100000:3.0:1787739051.899529:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739051.899539:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073061fed00 x1874579665518976/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739051.899567:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fed00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665518976:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787739051.899610:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab3b1 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ab3aa expref: 7 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899622:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x99a77dc0 (ffffa07403eebe00) 00010000:00010000:0.0:1787739051.899627:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab3e2 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739051.899631:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787739051.899636:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab3e2 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899644:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab3e2 lrc: 4/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899663:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739051.899666:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.899684:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665518976 00000100:00100000:0.0:1787739051.899706:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fc780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+6:3650:x1874579665518720:12345-0@lo:103:ldlm_bl.0 Request processed in 3908us (4060us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.899716:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1360 00010000:00010000:0.0:1787739051.899753:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab3e2 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.899761:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab3e2 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739051.899835:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d1a40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665519104:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739051.899862:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739051.899879:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d600 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.899881:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665518976 00000100:00100000:2.0:1787739051.899912:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665519104 00000100:00100000:2.0:1787739051.899918:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665519104 00000100:00100000:3.0:1787739051.899931:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2465 00000100:00100000:3.0:1787739051.899941:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fde00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665518976:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739051.899952:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2466 00000100:00100000:0.0:1787739051.899958:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d1e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665519104:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739051.900004:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665518720:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739051.900018:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d1e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6524:x1874579665519104:12345-0@lo:1000:mdt00_001.0 Request processed in 59us (154us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.900026:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2466 00000100:00100000:0.0:1787739051.900057:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d1a40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665519104:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739051.900062:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fde00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665518976:12345-0@lo:1000:osp_up1-0.0 Request processed in 124us (432us total) trans 4294968283 rc 0/0 00000100:00100000:3.0:1787739051.900069:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2465 00000100:00100000:2.0:1787739051.900093:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fed00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665518976:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787739051.900111:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab3e9 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x15a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.900122:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab3e9 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.900131:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab3e9 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739051.900150:0:6524:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab3e9 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.900160:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab3db lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.900164:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab3db lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.900168:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab3db lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.900171:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab3db lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.900176:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739051.900178:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab3db lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.900181:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab3db lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.900186:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2200) 00010000:00010000:0.0:1787739051.900188:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739051.900190:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab3db lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.900193:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab3db lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.900199:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab3e2 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.900203:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab3e2 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.900207:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab3e2 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.900210:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739051.900211:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab3e2 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.900214:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab3e2 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.900220:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x99a77dc0 (ffffa07403eebe00) 00010000:00010000:0.0:1787739051.900222:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739051.900223:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab3e2 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.900227:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab3e2 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.900250:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab3d4 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a1b expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.900260:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab3e9 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x15a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a1b expref: 14 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.900264:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15a:0x0].0x0 (ffffa0741590db00) 00010000:00010000:0.0:1787739051.900266:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739051.900267:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a800, rc 0) 00000100:00100000:0.0:1787739051.900307:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17406:x1874579652407808:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 6003us (6154us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.900313:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23135 00000100:00100000:3.0:1787739051.904247:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739051.904303:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652407936 00000100:00100000:2.0:1787739051.904317:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652407936 00000100:00100000:2.0:1787739051.904515:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23136 00000100:00100000:2.0:1787739051.904525:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fcb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17406:x1874579652407936:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739051.904614:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739051.904638:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ab3f0 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x15a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094a22 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.904699:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab3f7 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x15a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.904712:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab3f7 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.904724:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab3f7 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739051.904813:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x15a:0x0]: returning size 4096 00000004:00200000:2.0:1787739051.904818:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739051.904855:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ab3f0 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x15a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a22 expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.904874:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab3f7 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x15a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a22 expref: 10 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.904881:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15a:0x0].0x0 (ffffa07403eebb00) 00010000:00010000:2.0:1787739051.904883:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739051.904885:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d600, rc 0) 00000100:00100000:2.0:1787739051.904965:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fcb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17406:x1874579652407936:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 442us (732us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.904971:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23136 00000100:00100000:2.0:1787739051.909043:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739051.909082:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652408064 00000100:00100000:2.0:1787739051.909091:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652408064 00000100:00100000:2.0:1787739051.909222:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23137 00000100:00100000:2.0:1787739051.909235:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17406:x1874579652408064:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739051.909255:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739051.909467:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab3fe lrc: 2/0,0 mode: --/CR res: [0x240000400:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094a29 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.909530:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab405 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.909545:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab405 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.909557:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab405 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739051.909609:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xb2:0x0]: returning size 4096 00000004:00200000:2.0:1787739051.909614:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739051.909648:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab3fe lrc: 0/0,0 mode: --/CR res: [0x240000400:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a29 expref: 12 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.909666:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab405 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a29 expref: 11 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.909675:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb2:0x0].0x0 (ffffa07403eeb300) 00010000:00010000:2.0:1787739051.909678:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739051.909681:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c400, rc 0) 00000100:00100000:2.0:1787739051.909735:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17406:x1874579652408064:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 503us (699us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.909743:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23137 00000100:00100000:0.0:1787739051.925210:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739051.928742:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652408192 00000100:00100000:1.0:1787739051.928756:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652408192 00000100:00100000:0.0:1787739051.929306:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23138 00000100:00100000:0.0:1787739051.931200:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17406:x1874579652408192:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739051.931255:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739051.931291:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab40c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094a30 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.931475:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab413 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.931502:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab3cd lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab3c6 expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.931512:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab413 lrc: 4/1,0 mode: --/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.931545:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab3cd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ab3c6 expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739051.931560:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d30c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665519232:0@lo:104: 00000100:00100000:0.0:1787739051.931648:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739051.931715:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4db40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739051.932243:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665519232 00000100:00100000:2.0:1787739051.932251:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665519232 00000100:00100000:2.0:1787739051.932320:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1038 00000100:00100000:2.0:1787739051.932329:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d2580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665519232:12345-0@lo:104: 00010000:00010000:2.0:1787739051.932352:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab3c6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab3cd expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.932432:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739051.932439:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d2580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665519232:12345-0@lo:104: Request processed in 112us (793us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.932446:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1038 00010000:00010000:2.0:1787739051.932468:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739051.932474:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab3c6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab3cd expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.932483:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633da00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739051.932489:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab3c6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ab3cd expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739051.932498:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab3c6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ab3cd expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.932525:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab3c6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab3cd expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.932532:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739051.932552:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab3c6 lrc: 1/0,0 mode: --/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab3cd expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.932559:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ab3c6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab3cd expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.932575:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739051.932577:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739051.932580:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739051.932593:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f76d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665519360:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739051.932630:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739051.932637:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d30c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665519232:0@lo:104: 00010000:00010000:0.0:1787739051.932677:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab413 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739051.932706:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665519360 00000100:00100000:0.0:1787739051.932720:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665519360 00010000:00010000:0.0:1787739051.932757:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab3cd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ab3c6 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739051.932800:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1361 00000100:00100000:0.0:1787739051.932817:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba1450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+6:3651:x1874579665519360:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739051.932829:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00080000:1.0:1787739051.936159:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 340 batchid updated 00010000:00010000:1.0:1787739051.936508:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab3cd lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ab3c6 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.936527:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb2:0x0].0x0 (ffffa07403eebf00) 00010000:00010000:1.0:1787739051.936533:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab413 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.936543:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab413 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.936551:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab413 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.936575:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739051.936642:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba1450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665519360:12345-0@lo:103:ldlm_bl.0 Request processed in 3826us (4011us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.936652:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1361 00000100:00100000:1.0:1787739051.936733:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f76d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665519360:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739051.936757:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab413 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.936765:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab413 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739051.936816:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb2:0x0]: returning size 4096 00000004:00200000:1.0:1787739051.936821:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739051.936863:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab40c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a30 expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.936878:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab413 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a30 expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.936886:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb2:0x0].0x0 (ffffa07403eebf00) 00010000:00010000:1.0:1787739051.936889:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739051.936891:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609be00, rc 0) 00000100:00100000:1.0:1787739051.936934:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17406:x1874579652408192:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 7617us (11839us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739051.936941:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23138 00000020:00080000:3.0:1787739051.940649:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 340 00000004:00080000:3.0:1787739051.940721:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa073068b4900 with version 646 00000020:00080000:3.0:1787739051.940733:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 340 cancel update log [0x240000401:0x3:0x0].337: rc = 0 00000020:00080000:3.0:1787739051.940811:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 340 cancel update log [0x200000400:0x3:0x0].337: rc = 0 00000020:00080000:3.0:1787739051.940821:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 340 00000004:00080000:3.0:1787739051.940969:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 646 rpc_version 646 00000020:00100000:3.0:1787739051.941002:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739051.941013:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073061fc780 x1874579665519616/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739051.941036:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fc780 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665519616:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739051.941118:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.941163:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739051.941163:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665519616 00000100:00100000:1.0:1787739051.941172:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665519616 00000100:00100000:1.0:1787739051.941227:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2467 00000100:00100000:1.0:1787739051.941236:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305656940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665519616:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739051.941547:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305656940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665519616:12345-0@lo:1000:osp_up1-0.0 Request processed in 312us (428us total) trans 4294968284 rc 0/0 00000100:00100000:1.0:1787739051.941556:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2467 00000100:00100000:3.0:1787739051.941603:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fc780 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665519616:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739051.947712:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739051.947919:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652408320 00000100:00100000:0.0:1787739051.947929:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652408320 00000100:00100000:0.0:1787739051.947992:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1158 00000100:00100000:0.0:1787739051.948003:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2409:x1874579652408320:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:0.0:1787739051.948020:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739051.948045:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab3e9 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x15a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a1b expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.948063:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15a:0x0].0x0 (ffffa0741590db00) 00010000:00010000:0.0:1787739051.948074:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739051.948147:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652408320:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 146us (444us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.948156:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1158 00000100:00100000:1.0:1787739051.949059:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739051.949140:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652408576 00000100:00100000:3.0:1787739051.949148:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652408576 00000100:00100000:3.0:1787739051.949199:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23139 00000100:00100000:3.0:1787739051.949209:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17406:x1874579652408576:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:3.0:1787739051.949262:0:6523:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739051.949284:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab3f7 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x15a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a22 expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.949308:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15a:0x0].0x0 (ffffa07403eebb00) 00010000:00010000:3.0:1787739051.949312:0:6523:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739051.949359:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1ab41a lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.949369:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1ab41a lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.949379:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1ab41a lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739051.949401:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665519744:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739051.949921:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.950149:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739051.950299:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665519744 00000100:00100000:0.0:1787739051.950305:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665519744 00000100:00100000:0.0:1787739051.950380:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2468 00000100:00100000:0.0:1787739051.950390:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7130c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+5:6523:x1874579665519744:12345-0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739051.950402:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739051.950635:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab421 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab41a expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.950663:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab421 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab41a expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.950678:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab421 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab41a expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.950685:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0741590d200) 00010000:00010000:0.0:1787739051.950689:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739051.950692:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b200, rc 0) 00000100:00100000:0.0:1787739051.950738:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7130c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:6523:x1874579665519744:12345-0@lo:101:mdt00_000.0 Request processed in 350us (818us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.950757:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2468 00000100:00100000:3.0:1787739051.950956:0:6523:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983293 gen 2 00000004:00080000:3.0:1787739051.950963:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983289 committed transno 4294983293 00000020:00080000:3.0:1787739051.950969:0:6523:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07305fcc000 refcount 2 committed 0 result 0 batchid 331 00000020:00080000:3.0:1787739051.950973:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00400 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07305fcc600 00000020:00080000:3.0:1787739051.950976:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].324 00000020:00080000:3.0:1787739051.950979:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00500 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306aadc00 00000020:00080000:3.0:1787739051.950981:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].324 00000004:00080000:3.0:1787739051.951014:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983291 committed transno 4294983293 00000004:00080000:3.0:1787739051.951024:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983293 committed transno 4294983293 00000100:00100000:3.0:1787739051.951043:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665519744:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739051.951060:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1ab41a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab421 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.951075:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1ab41a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab421 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739051.951090:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739051.951129:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab428 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.951137:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab428 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.951146:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab428 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739051.951161:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665519872:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739051.951190:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.951208:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.951222:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665519872 00000100:00100000:3.0:1787739051.951226:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665519872 00000100:00100000:3.0:1787739051.951255:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2469 00000100:00100000:3.0:1787739051.951261:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7112c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:6523:x1874579665519872:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739051.951271:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739051.951284:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ab42f lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab428 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.951299:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ab42f lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab428 expref: 7 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.951310:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ab42f lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab428 expref: 7 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.951316:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x99a77dc0 (ffffa0730608b600) 00010000:00010000:3.0:1787739051.951319:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739051.951321:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa200, rc 0) 00000100:00100000:3.0:1787739051.951353:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7112c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665519872:12345-0@lo:101:mdt00_000.0 Request processed in 92us (162us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.951359:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2469 00000100:00100000:3.0:1787739051.951388:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665519872:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739051.951398:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab428 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab42f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.951407:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab428 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab42f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739051.951483:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:3.0:1787739051.951515:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665520000:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739051.951541:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.951557:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.951572:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665520000 00000100:00100000:3.0:1787739051.951575:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665520000 00000100:00100000:3.0:1787739051.951599:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2470 00000100:00100000:3.0:1787739051.951606:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665520000:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739051.951661:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665520000:12345-0@lo:1000:mdt00_000.0 Request processed in 56us (119us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.951667:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2470 00000100:00100000:3.0:1787739051.951693:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665520000:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739051.951727:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665520128:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739051.951882:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.951901:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355ed780 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.951915:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665520128 00000100:00100000:3.0:1787739051.951920:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665520128 00000100:00100000:3.0:1787739051.951945:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2471 00000100:00100000:3.0:1787739051.951951:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f711e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665520128:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739051.952026:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f711e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665520128:12345-0@lo:1000:mdt00_000.0 Request processed in 75us (144us total) trans 0 rc 1/1 00000100:00100000:3.0:1787739051.952033:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2471 00000100:00100000:3.0:1787739051.952062:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665520128:0@lo:1000:mdt00_000.0 00010000:00010000:3.0:1787739051.952114:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ab436 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.952122:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ab436 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.952130:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ab436 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739051.952142:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665520256:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739051.952164:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.952180:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739051.952192:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665520256 00000100:00100000:3.0:1787739051.952195:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665520256 00000100:00100000:3.0:1787739051.952218:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2472 00000100:00100000:3.0:1787739051.952224:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665520256:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739051.952232:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739051.952244:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab43d lrc: 2/0,0 mode: --/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab436 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.952261:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab43d lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab436 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.952272:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab43d lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab436 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.952279:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15a:0x0].0x0 (ffffa0730608bf00) 00010000:00010000:3.0:1787739051.952282:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739051.952284:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab400, rc 0) 00000100:00100000:3.0:1787739051.952314:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665520256:12345-0@lo:101:mdt00_000.0 Request processed in 91us (149us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739051.952319:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2472 00000100:00100000:3.0:1787739051.952346:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665520256:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739051.952357:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ab436 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab43d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.952366:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ab436 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab43d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.952385:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab444 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.952394:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab444 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.952402:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab444 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:1.0:1787739051.952537:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 331 committed batchid 329 00000020:00080000:1.0:1787739051.952586:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 329 00010000:00010000:3.0:1787739051.952624:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab44b lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.952637:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab405 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a29 expref: 10 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.952645:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab44b lrc: 4/0,1 mode: --/EX res: [0x240000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.952665:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab405 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a29 expref: 10 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.952679:0:6523:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab405 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a29 expref: 10 pid: 7944 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739051.952690:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665520384:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739051.952785:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739051.967344:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739051.967460:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665520384:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739051.967514:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab44b lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739051.967615:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652408704 00000100:00100000:0.0:1787739051.967627:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652408704 00010000:00010000:0.0:1787739051.967672:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab405 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a29 expref: 11 pid: 7944 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739051.967714:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1362 00000100:00100000:0.0:1787739051.967726:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41de00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2408:x1874579652408704:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739051.967741:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739051.967748:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab405 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a29 expref: 11 pid: 7944 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.967761:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab405 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a29 expref: 11 pid: 7944 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.967795:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab405 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a29 expref: 11 pid: 7944 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.967810:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb2:0x0].0x0 (ffffa07403eeb300) 00010000:00010000:0.0:1787739051.967816:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab44b lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.967826:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab44b lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.967840:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab44b lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739051.967880:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739051.967963:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab44b lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.967973:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab44b lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.968931:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1ab452 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.968948:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1ab452 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739051.968964:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1ab452 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739051.968994:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665520512:0@lo:101:mdt00_000.0 00000100:00100000:0.0:1787739051.970025:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41de00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652408704:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 2294us (2696us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739051.970108:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1362 00000100:00100000:3.0:1787739051.978527:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739051.978608:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739051.978610:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665520512 00000100:00100000:2.0:1787739051.978628:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665520512 00000100:00100000:2.0:1787739051.978702:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2473 00000100:00100000:2.0:1787739051.978712:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665520512:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739051.978732:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739051.978762:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab459 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab452 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.978794:0:17834:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab413 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a30 expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.978805:0:17834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab459 lrc: 4/0,0 mode: --/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab452 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.978849:0:17834:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab413 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a30 expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.978863:0:17834:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab413 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a30 expref: 13 pid: 6524 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739051.978875:0:17834:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f77c00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:17834:1874579665520640:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739051.978960:0:17834:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739051.985525:0:17834:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f77c00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:17834:1874579665520640:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739051.985563:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab459 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab452 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.985575:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb2:0x0].0x0 (ffffa07403eebf00) 00010000:00010000:2.0:1787739051.985579:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739051.985583:0:17834:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab459 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab452 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.985594:0:17834:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab459 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab452 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739051.985602:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633da00, rc 0) 00000100:00100000:2.0:1787739051.985678:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665520512:12345-0@lo:101:mdt00_000.0 Request processed in 6968us (7163us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739051.985687:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2473 00000100:00100000:1.0:1787739051.985850:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665520512:0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739051.985881:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1ab452 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab459 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739051.985892:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1ab452 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab459 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739052.010520:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739052.010668:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652408832 00000100:00100000:1.0:1787739052.010681:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652408832 00010000:00010000:0.0:1787739052.010853:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab413 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a30 expref: 14 pid: 6524 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739052.010968:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1363 00000100:00100000:0.0:1787739052.010981:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f77c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652408832:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739052.011002:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739052.011010:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab413 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a30 expref: 14 pid: 6524 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.011025:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab413 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a30 expref: 14 pid: 6524 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.011131:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab413 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a30 expref: 14 pid: 6524 timeout: 738 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.011147:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb2:0x0].0x0 (ffffa07403eebf00) 00010000:00010000:0.0:1787739052.011155:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab459 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab452 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.011172:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab459 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab452 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.011182:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab459 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab452 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.011224:0:6509:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab459 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab452 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739052.011242:0:6509:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d30c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665520768:0@lo:105: 00000100:00100000:0.0:1787739052.011303:0:6509:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739052.011340:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665520768 00000100:00100000:2.0:1787739052.011345:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665520768 00000100:00100000:0.0:1787739052.011351:0:6509:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4db40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739052.011433:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1039 00000100:00100000:2.0:1787739052.011442:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d0b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665520768:12345-0@lo:105: 00010000:00010000:2.0:1787739052.011455:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1ab452 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab459 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.011466:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1ab452 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab459 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.011477:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1ab452 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab459 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.011484:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1ab452 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab459 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.011490:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1ab452 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab459 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.011499:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1ab452 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ab459 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.011520:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069abc00) 00000100:00100000:2.0:1787739052.011564:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d0b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665520768:12345-0@lo:105: Request processed in 124us (259us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739052.011570:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1039 00000100:00100000:2.0:1787739052.011671:0:6509:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d30c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665520768:0@lo:105: 00010000:00010000:2.0:1787739052.011690:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739052.011739:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f77c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:2409:x1874579652408832:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 761us (1234us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739052.011747:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1363 00010000:00010000:2.0:1787739052.012255:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1ab452 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab459 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.012268:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1ab452 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab459 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739052.013579:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665520896:0@lo:403:mdt00_000.0 00000100:00100000:2.0:1787739052.013671:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739052.013713:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739052.013733:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665520896 00000100:00100000:3.0:1787739052.013742:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665520896 00000100:00100000:3.0:1787739052.013808:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2474 00000100:00100000:3.0:1787739052.013819:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f76d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665520896:12345-0@lo:403:mdt00_000.0 00000100:00100000:3.0:1787739052.016384:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f76d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665520896:12345-0@lo:403:mdt00_000.0 Request processed in 2549us (2695us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739052.016404:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2474 00000100:00100000:2.0:1787739052.016496:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665520896:0@lo:403:mdt00_000.0 00000040:00080000:3.0:1787739052.017695:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].326, 664 off497184 00000040:00080000:3.0:1787739052.017968:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].326, 744 off542960 00000004:00080000:3.0:1787739052.017978:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730610e3c0 with version 620 00000004:00080000:1.0:1787739052.018139:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 620 rpc_version 620 00000020:00100000:1.0:1787739052.018190:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739052.018201:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743c41c780 x1874579665521152/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2064/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739052.018227:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c780 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665521152:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739052.018306:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739052.018363:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739052.019146:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665521152 00000100:00100000:3.0:1787739052.019161:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665521152 00000100:00100000:3.0:1787739052.019239:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2475 00000100:00100000:3.0:1787739052.019251:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665521152:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739052.019992:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665521152:12345-0@lo:1000:osp_up0-1.0 Request processed in 740us (1681us total) trans 4294983294 rc 0/0 00000100:00100000:3.0:1787739052.020003:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2475 00000100:00100000:1.0:1787739052.020178:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c780 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665521152:0@lo:1000:osp_up0-1.0 00000004:00200000:3.0:1787739052.020259:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x15a:0x0]: returning size 4096 00010000:00010000:3.0:1787739052.020287:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab44b lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.020296:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab44b lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.020302:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab44b lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.020308:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739052.020312:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab44b lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.020319:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab44b lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.020334:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb2:0x0].0x0 (ffffa07403eeb300) 00010000:00010000:3.0:1787739052.020336:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739052.020340:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab44b lrc: 2/0,0 mode: --/EX res: [0x240000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.020343:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab44b lrc: 0/0,0 mode: --/EX res: [0x240000400:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.020355:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1ab452 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab459 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.020360:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1ab452 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab459 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.020366:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1ab452 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab459 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.020400:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739052.020443:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xb2:0x0] 00010000:00010000:2.0:1787739052.020443:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739052.020452:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab444 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.020452:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1ab452 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab459 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.020459:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab444 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.020466:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab444 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.020466:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739052.020471:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739052.020471:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1ab452 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab459 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.020475:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab444 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739052.020480:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1ab452 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab459 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.020481:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab444 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.020490:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15a:0x0].0x0 (ffffa07403eebb00) 00010000:00010000:3.0:1787739052.020493:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739052.020496:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab444 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.020501:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab444 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.020509:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ab436 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab43d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.020516:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ab436 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab43d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.020517:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1ab452 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab459 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.020524:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ab436 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab43d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.020525:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739052.020545:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739052.020555:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739052.020557:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1ab452 lrc: 1/0,0 mode: --/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab459 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.020560:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1ab41a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab421 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.020565:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abc00/0x7ee4453e4a1ab452 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab459 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.020567:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1ab41a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab421 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.020574:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1ab41a lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab421 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.020583:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787739052.020585:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1787739052.020586:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ab436 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab43d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739052.020589:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739052.020592:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739052.020593:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab428 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab42f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.020596:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ab436 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab43d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.020602:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab428 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab42f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739052.020603:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ab436 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab43d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.020623:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ab436 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab43d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.020629:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739052.020634:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ab436 lrc: 1/0,0 mode: --/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab43d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.020641:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ab436 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab43d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.020651:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739052.020655:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1ab41a lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab421 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.020660:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aaa00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739052.020663:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1ab41a lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab421 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739052.020669:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1ab41a lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab421 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739052.020681:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17406:x1874579652408576:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 71472us (71625us total) trans 4294968285 rc 0/0 00010000:00010000:2.0:1787739052.020684:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1ab41a lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab421 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.020689:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:3.0:1787739052.020690:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23139 00010000:00010000:2.0:1787739052.020705:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1ab41a lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab421 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.020716:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1ab41a lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab421 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.020726:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739052.020728:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739052.020729:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739052.020745:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665521408:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739052.020963:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787739052.020978:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739052.020987:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739052.021065:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665521280:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739052.021203:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665521536:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739052.024624:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739052.024727:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665521408 00000100:00100000:1.0:1787739052.024939:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665521408 00000100:00100000:0.0:1787739052.025076:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665521280 00000100:00100000:0.0:1787739052.025081:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665521280 00000100:00100000:1.0:1787739052.025864:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1159 00000100:00100000:1.0:1787739052.025880:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ade00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3653:x1874579665521408:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739052.025906:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739052.025947:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab43d lrc: 0/0,0 mode: --/EX res: [0x240000402:0x15a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab436 expref: 11 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.025977:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15a:0x0].0x0 (ffffa0730608bf00) 00010000:00010000:1.0:1787739052.025991:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739052.026011:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739052.026225:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ade00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+10:3653:x1874579665521408:12345-0@lo:103:ldlm_bl.0 Request processed in 349us (5260us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739052.026233:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1159 00000100:00100000:0.0:1787739052.027548:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665521536 00000100:00100000:0.0:1787739052.027564:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665521536 00000100:00100000:0.0:1787739052.027622:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1160 00000100:00100000:0.0:1787739052.027632:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156eda40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+10:3652:x1874579665521280:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739052.027649:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739052.027675:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab459 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab452 expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.027698:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb2:0x0].0x0 (ffffa07403eebf00) 00010000:00010000:0.0:1787739052.027708:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739052.027791:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156eda40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+9:3652:x1874579665521280:12345-0@lo:103:ldlm_bl.0 Request processed in 161us (3180us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739052.027799:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1160 00000100:00100000:0.0:1787739052.027817:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1161 00000100:00100000:0.0:1787739052.027822:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3650:x1874579665521536:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739052.027830:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739052.027841:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab421 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab41a expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.027852:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0741590d200) 00010000:00010000:0.0:1787739052.027859:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739052.027888:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+7:3650:x1874579665521536:12345-0@lo:103:ldlm_bl.0 Request processed in 66us (1878us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739052.027894:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1161 00000100:00100000:1.0:1787739052.028116:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665521280:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739052.028158:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665521408:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739052.029191:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665521536:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739052.038778:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739052.038943:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652408960 00000100:00100000:3.0:1787739052.038957:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652408960 00000100:00100000:3.0:1787739052.039073:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23140 00000100:00100000:3.0:1787739052.039086:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17407:x1874579652408960:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739052.039120:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739052.039151:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab460 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094a37 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.039243:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab467 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.039260:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab467 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.039276:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab467 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.039290:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab46e lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.039297:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab46e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.039304:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab46e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.039577:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab467 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.039587:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab467 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.039594:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab467 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.039607:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab467 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.039614:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739052.039620:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab467 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.039629:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab467 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.039646:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730608b500) 00010000:00010000:3.0:1787739052.039650:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739052.039655:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab467 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.039662:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab467 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.039680:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab46e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.039686:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab46e lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.039692:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab46e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.039698:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739052.039701:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab46e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.039713:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab46e lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.039721:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xdf2a7902 (ffffa0730608b000) 00010000:00010000:3.0:1787739052.039723:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739052.039726:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab46e lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.039732:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab46e lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.039778:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab460 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a37 expref: 14 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.039792:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730608b500) 00010000:00010000:3.0:1787739052.039795:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739052.039798:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab460 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a37 expref: 14 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.039810:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abe00, rc 0) 00000100:00100000:3.0:1787739052.040036:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17407:x1874579652408960:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 951us (1269us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739052.040046:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23140 00000100:00100000:3.0:1787739052.045909:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739052.046036:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652409088 00000100:00100000:1.0:1787739052.046050:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652409088 00000100:00100000:1.0:1787739052.046134:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23141 00000100:00100000:1.0:1787739052.046147:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f712d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17407:x1874579652409088:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739052.046179:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739052.046216:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab475 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094a3e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.046320:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab47c lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.046347:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab47c lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.046362:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab47c lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.046384:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab483 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.046391:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab483 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.046398:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab483 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.046506:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab47c lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.046515:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab47c lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.046522:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab47c lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.046529:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab47c lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.046536:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739052.046540:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab47c lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.046548:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab47c lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.046562:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae300) 00010000:00010000:1.0:1787739052.046565:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739052.046570:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab47c lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.046578:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab47c lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.046593:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab483 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.046599:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab483 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.046606:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab483 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.046611:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739052.046614:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab483 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.046620:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab483 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.046627:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xdf2a7902 (ffffa07305eae000) 00010000:00010000:1.0:1787739052.046630:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739052.046632:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab483 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.046637:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab483 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.046672:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab475 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a3e expref: 14 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.046688:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae300) 00010000:00010000:1.0:1787739052.046691:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739052.046695:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab475 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a3e expref: 14 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.046705:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1ca00, rc 0) 00000100:00100000:1.0:1787739052.046774:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f712d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17407:x1874579652409088:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 629us (881us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739052.046782:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23141 00000100:00100000:0.0:1787739052.052655:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739052.052884:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652409216 00000100:00100000:3.0:1787739052.052901:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652409216 00000100:00100000:3.0:1787739052.052977:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23142 00000100:00100000:3.0:1787739052.052989:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17407:x1874579652409216:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739052.053015:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739052.053044:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab48a lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094a45 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.053134:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab491 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.053150:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab491 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.053164:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab491 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739052.053281:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:3.0:1787739052.053288:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739052.053642:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab48a lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a45 expref: 15 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.053666:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab491 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a45 expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.053675:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730608b500) 00010000:00010000:3.0:1787739052.053679:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739052.053682:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aac00, rc 0) 00000100:00100000:3.0:1787739052.053767:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17407:x1874579652409216:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 780us (1127us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739052.053776:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23142 00000100:00100000:2.0:1787739052.070116:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739052.072398:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652409472 00000100:00100000:3.0:1787739052.072439:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652409472 00000100:00100000:3.0:1787739052.072574:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23143 00000100:00100000:3.0:1787739052.072587:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17407:x1874579652409472:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:3.0:1787739052.072672:0:7944:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739052.072708:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab491 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a45 expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.072729:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730608b500) 00010000:00010000:3.0:1787739052.072740:0:7944:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739052.072828:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab498 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.072845:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab498 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.072856:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab498 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.073039:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab49f lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.073050:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab49f lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.073057:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab49f lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787739052.078319:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].339, 2168 off522208 00000040:00080000:3.0:1787739052.078454:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].339, 2368 off565936 00000004:00080000:3.0:1787739052.078464:0:7944:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07435539d80 with version 647 00000004:00080000:2.0:1787739052.078502:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 647 rpc_version 647 00000020:00100000:2.0:1787739052.078554:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739052.078567:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074356aed00 x1874579665521792/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 4232/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739052.078721:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356aed00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665521792:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739052.078959:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739052.079029:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610eb40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739052.079142:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665521792 00000100:00100000:3.0:1787739052.079152:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665521792 00000100:00100000:3.0:1787739052.079221:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2476 00000100:00100000:3.0:1787739052.079233:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07436d8c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665521792:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739052.080661:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07436d8c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665521792:12345-0@lo:1000:osp_up1-0.0 Request processed in 1422us (1694us total) trans 4294968286 rc 0/0 00000100:00100000:3.0:1787739052.080678:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2476 00000100:00100000:2.0:1787739052.086562:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356aed00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665521792:0@lo:1000:osp_up1-0.0 00010000:00010000:3.0:1787739052.086857:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab4a6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x14f2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.087031:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab4a6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14f2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.087046:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab4a6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14f2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.087074:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab4ad lrc: 3/0,1 mode: --/PW res: [0x200000400:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.087081:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab4ad lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.087089:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab4ad lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.087146:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ab4b4 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.087154:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ab4b4 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.087165:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ab4b4 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739052.087192:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f7130c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665521920:0@lo:101:mdt00_003.0 00000100:00100000:3.0:1787739052.087280:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739052.087357:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739052.087550:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665521920 00000100:00100000:2.0:1787739052.087561:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665521920 00000100:00100000:3.0:1787739052.087650:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2477 00000100:00100000:3.0:1787739052.087662:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7112c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665521920:12345-0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739052.087690:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739052.087713:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab4bb lrc: 2/0,0 mode: --/PW res: [0x240000401:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab4b4 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.087742:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab4bb lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab4b4 expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.087760:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab4bb lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab4b4 expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.087770:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb5:0x0].0x0 (ffffa0730608b400) 00010000:00010000:3.0:1787739052.087773:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739052.087778:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa600, rc 0) 00000100:00100000:3.0:1787739052.088165:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7112c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665521920:12345-0@lo:101:mdt00_003.0 Request processed in 504us (885us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739052.088175:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2477 00000100:00100000:2.0:1787739052.091102:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f7130c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665521920:0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739052.091148:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ab4b4 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab4bb expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.091165:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ab4b4 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab4bb expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739052.091198:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000082bf55d0 reply state 00000000d5e34afc transno 4294983295 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab4ad lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.091214:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab4ad lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.091223:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb5:0x0].0x0 (ffffa0730608b100) 00010000:00010000:2.0:1787739052.091227:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739052.091233:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787739052.091241:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ab4b4 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab4bb expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.091250:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ab4b4 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab4bb expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739052.091264:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xb5:0x0] 00000004:00010000:2.0:1787739052.091272:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000082bf55d0 reply state 00000000d5e34afc transno 4294983295 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab4a6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14f2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.091280:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab4a6 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x14f2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.091286:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f2:0x0].0x0 (ffffa0730608bd00) 00010000:00010000:2.0:1787739052.091289:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739052.091292:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:2.0:1787739052.091299:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14f2:0x0]: returning size 4096 00010000:00010000:2.0:1787739052.091313:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab498 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.091320:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab498 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.091325:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab498 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.091334:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab498 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.091340:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739052.091346:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab498 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.091354:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab498 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.091364:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730608b000) 00010000:00010000:2.0:1787739052.091366:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739052.091370:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab498 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.091377:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab498 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739052.091398:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000082bf55d0 reply state 00000000d5e34afc transno 4294983295 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab49f lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.091407:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab49f lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.091432:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xdf2a7902 (ffffa0730608b800) 00010000:00010000:2.0:1787739052.091435:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739052.091437:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 2 00000100:00100000:2.0:1787739052.091558:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17407:x1874579652409472:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 18970us (21463us total) trans 4294983295 rc 0/0 00000100:00100000:2.0:1787739052.091569:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23143 00000100:00100000:3.0:1787739052.136298:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739052.137695:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652409728 00000100:00100000:0.0:1787739052.137713:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652409728 00000100:00100000:0.0:1787739052.137915:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23144 00000100:00100000:0.0:1787739052.137930:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17408:x1874579652409728:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:0.0:1787739052.138237:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17408:x1874579652409728:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 310us (1950us total) trans 0 rc -17/-17 00000100:00100000:0.0:1787739052.138245:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23144 00000100:00100000:2.0:1787739052.188378:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739052.188506:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652409856 00000100:00100000:3.0:1787739052.188521:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652409856 00000100:00100000:3.0:1787739052.188609:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23145 00000100:00100000:3.0:1787739052.188623:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356af840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17409:x1874579652409856:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739052.188655:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739052.188702:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab4c2 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094a4c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.188814:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab4c9 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.188834:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab4c9 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.188849:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab4c9 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.188861:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab4d0 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.188870:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab4d0 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.188878:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab4d0 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.188990:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab4d7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14f2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.188999:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab4d7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14f2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.189007:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab4d7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14f2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739052.189205:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14f2:0x0]: returning size 4096 00000004:00200000:3.0:1787739052.189211:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:3.0:1787739052.189237:0:6523:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab4d7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14f2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.189302:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab4c9 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.189310:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab4c9 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.189318:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab4c9 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.189325:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab4c9 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.189332:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739052.189336:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab4c9 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.189345:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab4c9 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.189360:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730608b500) 00010000:00010000:3.0:1787739052.189363:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739052.189368:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab4c9 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.189375:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab4c9 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.189388:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab4d0 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.189400:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab4d0 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.189407:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab4d0 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.189448:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aae00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739052.189452:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab4d0 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.189459:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab4d0 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.189467:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xdf2a7902 (ffffa0730608b800) 00010000:00010000:3.0:1787739052.189469:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739052.189471:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab4d0 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.189475:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab4d0 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.189499:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab4c2 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a4c expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.189512:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab4d7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14f2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a4c expref: 16 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.189517:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f2:0x0].0x0 (ffffa0730608bd00) 00010000:00010000:3.0:1787739052.189519:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787739052.189521:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abe00, rc 0) 00000100:00100000:3.0:1787739052.189594:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356af840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17409:x1874579652409856:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 974us (1227us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739052.189603:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23145 00000100:00100000:2.0:1787739052.193784:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739052.196473:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652409984 00000100:00100000:1.0:1787739052.196492:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652409984 00000100:00100000:1.0:1787739052.196574:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23146 00000100:00100000:1.0:1787739052.196588:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17409:x1874579652409984:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739052.196616:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739052.196644:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab4de lrc: 2/0,0 mode: --/CR res: [0x200000400:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094a53 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.196723:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab4e5 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.196741:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab4e5 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.196756:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab4e5 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739052.196823:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xb5:0x0]: returning size 4096 00000004:00200000:1.0:1787739052.196831:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739052.196882:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab4de lrc: 0/0,0 mode: --/CR res: [0x200000400:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a53 expref: 18 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.196901:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab4e5 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a53 expref: 17 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.196912:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb5:0x0].0x0 (ffffa0730608b100) 00010000:00010000:1.0:1787739052.196915:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739052.196919:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d800, rc 0) 00000100:00100000:1.0:1787739052.197005:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17409:x1874579652409984:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 419us (3234us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739052.197013:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23146 00000100:00100000:2.0:1787739052.207583:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739052.207701:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652410112 00000100:00100000:1.0:1787739052.207716:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652410112 00000100:00100000:1.0:1787739052.207875:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23147 00000100:00100000:1.0:1787739052.207888:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17409:x1874579652410112:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739052.207920:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739052.207946:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab4ec lrc: 2/0,0 mode: --/CR res: [0x240000401:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094a5a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.208028:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab4f3 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.208046:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab4bb lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab4b4 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.208057:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab4f3 lrc: 4/1,0 mode: --/PR res: [0x240000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.208113:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab4bb lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ab4b4 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739052.208133:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665522048:0@lo:104: 00000100:00100000:1.0:1787739052.208197:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739052.208259:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739052.208282:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665522048 00000100:00100000:1.0:1787739052.208287:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665522048 00000100:00100000:1.0:1787739052.208318:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1040 00000100:00100000:1.0:1787739052.208325:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6525:x1874579665522048:12345-0@lo:104: 00010000:00010000:1.0:1787739052.208339:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ab4b4 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab4bb expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.208404:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787739052.208431:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6525:x1874579665522048:12345-0@lo:104: Request processed in 86us (213us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739052.208439:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1040 00010000:00010000:1.0:1787739052.208464:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739052.208471:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ab4b4 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab4bb expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.208481:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aba00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739052.208487:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ab4b4 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xb5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ab4bb expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739052.208498:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ab4b4 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xb5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ab4bb expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.208532:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ab4b4 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xb5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab4bb expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.208540:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739052.208564:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ab4b4 lrc: 1/0,0 mode: --/PW res: [0x240000401:0xb5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab4bb expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.208572:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ab4b4 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xb5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab4bb expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.208594:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739052.208596:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739052.208598:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739052.208611:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665522176:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739052.208649:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739052.208695:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665522176 00000100:00100000:1.0:1787739052.208699:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665522176 00010000:00010000:1.0:1787739052.208717:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab4bb lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ab4b4 expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739052.208741:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1364 00000100:00100000:1.0:1787739052.208747:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba0850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+10:3651:x1874579665522176:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739052.208758:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787739052.209373:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665522048:0@lo:104: 00010000:00010000:1.0:1787739052.209398:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab4f3 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.211358:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab428 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab42f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739052.211382:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab428 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab42f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.211475:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab428 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab42f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.211483:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739052.211522:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab800/0x7ee4453e4a1ab428 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab42f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787739052.211543:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968286 is committed 00000001:00080000:1.0:1787739052.211558:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968285 is committed 00000020:00080000:1.0:1787739052.211569:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 331 batchid updated 00000020:00080000:1.0:1787739052.211581:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa0730610ec00 refcount 2 committed 0 result 0 batchid 332 00000020:00080000:1.0:1787739052.211584:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2f00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730610ecc0 00000020:00080000:1.0:1787739052.211588:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].325 00000020:00080000:1.0:1787739052.211590:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2400 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa074184eea00 00000020:00080000:1.0:1787739052.211593:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].325 00000020:00080000:1.0:1787739052.211606:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 330 batchid updated 00000020:00080000:1.0:1787739052.211620:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 332 committed batchid 331 00000004:00080000:1.0:1787739052.211746:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07305fcc540 with version 621 00000020:00080000:1.0:1787739052.211758:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 330 cancel update log [0x200000401:0x3:0x0].323: rc = 0 00000020:00080000:1.0:1787739052.211818:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 330 cancel update log [0x240000400:0x3:0x0].323: rc = 0 00000004:00080000:1.0:1787739052.211842:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07305fccf00 with version 622 00000020:00080000:1.0:1787739052.211847:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 331 cancel update log [0x200000401:0x3:0x0].324: rc = 0 00000020:00080000:1.0:1787739052.211868:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 331 cancel update log [0x240000400:0x3:0x0].324: rc = 0 00000020:00080000:1.0:1787739052.211878:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 331 00000004:00080000:1.0:1787739052.211891:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 621 rpc_version 621 00000020:00100000:1.0:1787739052.211913:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739052.211925:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074156ecf00 x1874579665522560/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739052.211944:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665522560:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739052.212028:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739052.212062:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc000 going to sleep for 11 seconds 00000100:00100000:3.0:1787739052.212071:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787739052.212089:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665522304:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:3.0:1787739052.212134:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787739052.212208:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab3b8 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x15a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.212219:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab3b8 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x15a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.212226:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab3b8 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x15a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.212233:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739052.212238:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab3b8 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x15a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.212245:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab3b8 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x15a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.212260:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15a:0x0].0x0 (ffffa07403eebb00) 00010000:00010000:2.0:1787739052.212263:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739052.212269:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab3b8 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x15a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.212281:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab3b8 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x15a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.212300:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab3bf lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.212307:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab3bf lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.212313:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab3bf lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.212318:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633cc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739052.212321:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab3bf lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.212327:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab3bf lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.212336:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb2:0x0].0x0 (ffffa07403eeb300) 00010000:00010000:2.0:1787739052.212338:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739052.212341:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab3bf lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.212347:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab3bf lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739052.212376:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665522560 00000100:00100000:2.0:1787739052.212384:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665522560 00000100:00100000:2.0:1787739052.212468:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2478 00000100:00100000:2.0:1787739052.212478:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665522560:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739052.212642:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665522560:12345-0@lo:1000:osp_up0-1.0 Request processed in 164us (612us total) trans 4294983296 rc 0/0 00000100:00100000:2.0:1787739052.212651:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2478 00000100:00100000:3.0:1787739052.212702:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665522560:0@lo:1000:osp_up0-1.0 00000004:00080000:3.0:1787739052.212717:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 622 rpc_version 622 00000100:00100000:1.0:1787739052.212728:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665522304 00000020:00100000:3.0:1787739052.212733:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739052.212734:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665522304 00000100:00100000:3.0:1787739052.212743:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743f710f00 x1874579665522816/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739052.212760:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710f00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665522816:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739052.212844:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1162 00000100:00100000:1.0:1787739052.212852:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f711e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+8:3653:x1874579665522304:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:1.0:1787739052.212866:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739052.212886:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ab42f lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x99a77dc0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab428 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.212901:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x99a77dc0 (ffffa0730608b600) 00010000:00010000:1.0:1787739052.212908:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739052.212946:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f711e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+7:3653:x1874579665522304:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 95us (810us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739052.212954:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739052.212958:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1162 00000100:00100000:1.0:1787739052.212994:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665522304:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:3.0:1787739052.213076:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054780 going to sleep for 11 seconds 00000100:00100000:2.0:1787739052.213139:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665522816 00000100:00100000:2.0:1787739052.213144:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665522816 00000100:00100000:2.0:1787739052.213378:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2479 00000100:00100000:2.0:1787739052.213387:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665522816:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739052.213532:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665522816:12345-0@lo:1000:osp_up0-1.0 Request processed in 146us (580us total) trans 4294983297 rc 0/0 00000100:00100000:2.0:1787739052.213540:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2479 00000100:00100000:3.0:1787739052.214581:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710f00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665522816:0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1787739052.214839:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab4bb lrc: 0/0,0 mode: --/PW res: [0x240000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ab4b4 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.214860:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb5:0x0].0x0 (ffffa0730608b400) 00010000:00010000:0.0:1787739052.214865:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab4f3 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.214882:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab4f3 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.214890:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab4f3 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.214931:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739052.214998:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba0850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665522176:12345-0@lo:103:ldlm_bl.0 Request processed in 6250us (6348us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739052.215008:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1364 00000100:00100000:3.0:1787739052.215206:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665522176:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739052.215279:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab4f3 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.215289:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab4f3 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739052.215365:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xb5:0x0]: returning size 4096 00000004:00200000:2.0:1787739052.215372:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739052.215466:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab4ec lrc: 0/0,0 mode: --/CR res: [0x240000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a5a expref: 9 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.215485:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab4f3 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a5a expref: 8 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.215493:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb5:0x0].0x0 (ffffa0730608b400) 00010000:00010000:2.0:1787739052.215496:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739052.215498:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c200, rc 0) 00000100:00100000:2.0:1787739052.215553:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17409:x1874579652410112:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 7669us (7987us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739052.215562:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23147 00000100:00100000:3.0:1787739052.221385:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739052.221531:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652410368 00000100:00100000:0.0:1787739052.221544:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652410368 00000100:00100000:0.0:1787739052.221615:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23148 00000100:00100000:0.0:1787739052.221626:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f711a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17409:x1874579652410368:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:0.0:1787739052.221702:0:6523:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739052.221738:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab4d7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14f2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a4c expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.221889:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f2:0x0].0x0 (ffffa0730608bd00) 00010000:00010000:0.0:1787739052.221896:0:6523:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739052.221935:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab4fa lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.221951:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab4fa lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.221963:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab4fa lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.221981:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab501 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.221991:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab501 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.221998:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab501 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.222109:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab508 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14f2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.222119:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab508 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14f2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.222127:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab508 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14f2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.222148:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab50f lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.222160:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab4e5 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a53 expref: 16 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.222169:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab50f lrc: 4/0,1 mode: --/EX res: [0x200000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.222210:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab4e5 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a53 expref: 16 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.222227:0:6523:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab4e5 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a53 expref: 16 pid: 7944 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739052.222247:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d12c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665522944:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739052.222373:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739052.230788:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739052.231003:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d12c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665522944:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739052.231042:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab50f lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739052.231155:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652410496 00000100:00100000:2.0:1787739052.231164:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652410496 00010000:00010000:2.0:1787739052.231212:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab4e5 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a53 expref: 17 pid: 7944 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739052.231259:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1365 00000100:00100000:2.0:1787739052.231270:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7121c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652410496:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739052.231290:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739052.231298:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab4e5 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a53 expref: 17 pid: 7944 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.231312:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab4e5 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a53 expref: 17 pid: 7944 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.231342:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab4e5 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a53 expref: 17 pid: 7944 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.231362:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb5:0x0].0x0 (ffffa0730608b100) 00010000:00010000:2.0:1787739052.231368:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab50f lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.231380:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab50f lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.231388:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab50f lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.231606:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739052.231676:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7121c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:2409:x1874579652410496:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 406us (901us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739052.231686:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1365 00010000:00010000:0.0:1787739052.233797:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab50f lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.233820:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab50f lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.233977:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab516 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.233988:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab516 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.234002:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab516 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739052.234034:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d12c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665523072:0@lo:101:mdt00_000.0 00000100:00100000:0.0:1787739052.234143:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739052.234191:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739052.236311:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665523072 00000100:00100000:3.0:1787739052.236331:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665523072 00000100:00100000:3.0:1787739052.236402:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2480 00000100:00100000:3.0:1787739052.236433:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d3840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+5:6523:x1874579665523072:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739052.236462:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739052.236491:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab51d lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab516 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.236539:0:8448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab4f3 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a5a expref: 7 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.236551:0:8448:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab51d lrc: 4/0,0 mode: --/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab516 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.236592:0:8448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab4f3 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a5a expref: 7 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.236608:0:8448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab4f3 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a5a expref: 7 pid: 6525 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739052.236624:0:8448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f712580 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8448:1874579665523200:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739052.236738:0:8448:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054780 going to sleep for 11 seconds 00000100:00100000:0.0:1787739052.240342:0:8448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f712580 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8448:1874579665523200:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739052.240393:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab51d lrc: 3/0,0 mode: --/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab516 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.240432:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb5:0x0].0x0 (ffffa0730608b400) 00010000:00010000:0.0:1787739052.240437:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739052.240444:0:8448:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab51d lrc: 3/0,0 mode: --/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab516 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.240456:0:8448:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab51d lrc: 3/0,0 mode: --/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab516 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.240464:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aae00, rc 0) 00000100:00100000:0.0:1787739052.241197:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d3840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6523:x1874579665523072:12345-0@lo:101:mdt00_000.0 Request processed in 4678us (6948us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739052.241211:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2480 00000100:00100000:3.0:1787739052.245711:0:6523:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968286 gen 2 00000004:00080000:3.0:1787739052.245730:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968281 committed transno 4294968286 00000100:00100000:2.0:1787739052.246128:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739052.246179:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652410624 00000100:00100000:2.0:1787739052.246187:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652410624 00010000:00010000:2.0:1787739052.246237:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab4f3 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a5a expref: 8 pid: 6525 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739052.246277:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1366 00000100:00100000:2.0:1787739052.246289:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2408:x1874579652410624:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739052.246307:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739052.246316:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab4f3 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a5a expref: 8 pid: 6525 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.246331:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab4f3 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a5a expref: 8 pid: 6525 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.246359:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab4f3 lrc: 0/0,0 mode: --/PR res: [0x240000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a5a expref: 8 pid: 6525 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.246376:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb5:0x0].0x0 (ffffa0730608b400) 00010000:00010000:2.0:1787739052.246384:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab51d lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab516 expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.246402:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab51d lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab516 expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.246431:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab51d lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab516 expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.246471:0:10340:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab51d lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab516 expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739052.246488:0:10340:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ade00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665523328:0@lo:105: 00000100:00100000:2.0:1787739052.246552:0:10340:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739052.246594:0:10340:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739052.246614:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665523328 00000100:00100000:2.0:1787739052.246617:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665523328 00000100:00100000:2.0:1787739052.246647:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1041 00000100:00100000:2.0:1787739052.246654:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665523328:12345-0@lo:105: 00010000:00010000:2.0:1787739052.246668:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab516 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab51d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.246680:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab516 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab51d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.246688:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab516 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab51d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.246695:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab516 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab51d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.246703:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab516 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab51d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.246710:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab516 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ab51d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.246723:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609a400) 00000020:00080000:3.0:1787739052.247324:0:6523:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07410404a80 refcount 2 committed 0 result 0 batchid 341 00000020:00080000:3.0:1787739052.247334:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07302ea6900 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07410404000 00000020:00080000:3.0:1787739052.247338:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].338 00000020:00080000:3.0:1787739052.247341:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07302ea6700 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa074357c0000 00000020:00080000:3.0:1787739052.247344:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].338 00000004:00080000:3.0:1787739052.247380:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968283 committed transno 4294968286 00000004:00080000:3.0:1787739052.247392:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968284 committed transno 4294968286 00000004:00080000:3.0:1787739052.247440:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968286 committed transno 4294968286 00000100:00100000:3.0:1787739052.247461:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d12c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665523072:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739052.247484:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab516 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab51d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739052.248600:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665523328:12345-0@lo:105: Request processed in 1938us (2038us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739052.248615:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1041 00000100:00100000:2.0:1787739052.248696:0:10340:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ade00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665523328:0@lo:105: 00010000:00010000:2.0:1787739052.248714:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739052.249948:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652410624:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 3651us (3818us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739052.249963:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1366 00000100:00100000:3.0:1787739052.250692:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665523456:0@lo:403:mdt00_000.0 00000020:00080000:2.0:1787739052.251731:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 341 committed batchid 340 00080000:02000000:2.0:1787739052.251786:0:6555:0:(osd_handler.c:2218:osd_trans_start()) *** cfs_fail_loc=19a, val=0*** 00000020:00080000:2.0:1787739052.251803:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 341 committed batchid 340 00000100:00100000:3.0:1787739052.252162:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739052.252240:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054540 going to sleep for 11 seconds 00000100:00100000:2.0:1787739052.254270:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665523456 00000100:00100000:2.0:1787739052.254291:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665523456 00000100:00100000:2.0:1787739052.254370:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2481 00000100:00100000:2.0:1787739052.254382:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f712580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6523:x1874579665523456:12345-0@lo:403:mdt00_000.0 00000100:00100000:2.0:1787739052.262700:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f712580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6523:x1874579665523456:12345-0@lo:403:mdt00_000.0 Request processed in 8308us (10527us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739052.262721:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2481 00000100:00100000:0.0:1787739052.262920:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665523456:0@lo:403:mdt00_000.0 00000040:00080000:0.0:1787739052.269539:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].340, 664 off522872 00000040:00080000:0.0:1787739052.269785:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].340, 744 off566680 00000004:00080000:0.0:1787739052.269792:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07410b4d000 with version 648 00000004:00080000:2.0:1787739052.270955:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 648 rpc_version 648 00000020:00100000:2.0:1787739052.271010:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739052.271024:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306be7480 x1874579665523712/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1448/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739052.271054:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be7480 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665523712:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739052.271158:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739052.271201:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739052.271228:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665523712 00000100:00100000:2.0:1787739052.271236:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665523712 00000100:00100000:2.0:1787739052.271294:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2482 00000100:00100000:2.0:1787739052.271305:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665523712:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739052.271629:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665523712:12345-0@lo:1000:osp_up1-0.0 Request processed in 325us (470us total) trans 4294968287 rc 0/0 00000100:00100000:2.0:1787739052.271638:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2482 00000100:00100000:2.0:1787739052.271706:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be7480 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665523712:0@lo:1000:osp_up1-0.0 00000004:00200000:0.0:1787739052.271940:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14f2:0x0]: returning size 4096 00010000:00010000:0.0:1787739052.271971:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab50f lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.271982:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab50f lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.271990:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab50f lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.271997:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739052.272003:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab50f lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.272012:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab50f lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.272031:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb5:0x0].0x0 (ffffa0730608b100) 00010000:00010000:0.0:1787739052.272035:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739052.272041:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab50f lrc: 2/0,0 mode: --/EX res: [0x200000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.272047:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab50f lrc: 0/0,0 mode: --/EX res: [0x200000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.272063:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab516 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab51d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.272072:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab516 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab51d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.272078:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab516 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab51d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.272114:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739052.272126:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xb5:0x0] 00010000:00010000:0.0:1787739052.272135:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab508 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14f2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.272141:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab508 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x14f2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.272147:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab508 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14f2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.272152:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739052.272155:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab508 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14f2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.272161:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab508 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x14f2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.272170:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f2:0x0].0x0 (ffffa0730608bd00) 00010000:00010000:0.0:1787739052.272173:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739052.272176:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab508 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14f2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.272182:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab508 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14f2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.272199:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab4fa lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.272206:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab4fa lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.272213:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab4fa lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.272218:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab4fa lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.272224:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609be00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739052.272226:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab4fa lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.272237:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab4fa lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.272244:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0741590d200) 00010000:00010000:0.0:1787739052.272246:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739052.272250:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab4fa lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.272255:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab4fa lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739052.272274:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000004c723c87 reply state 00000000979024a4 transno 4294983298 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab501 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.272288:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab501 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.272296:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xdf2a7902 (ffffa0730608b800) 00010000:00010000:0.0:1787739052.272298:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739052.272301:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787739052.272387:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f711a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17409:x1874579652410368:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 50760us (51017us total) trans 4294983298 rc 0/0 00000100:00100000:0.0:1787739052.272396:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23148 00010000:00010000:2.0:1787739052.272902:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739052.272910:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab516 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab51d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.272919:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739052.272925:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab516 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab51d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739052.272934:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab516 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab51d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.272969:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab516 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab51d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.272977:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739052.273006:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab516 lrc: 1/0,0 mode: --/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab51d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.273013:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab516 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab51d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.273033:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739052.273035:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739052.273037:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739052.273047:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787739052.273063:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665523840:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739052.273128:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739052.273536:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665523840 00000100:00100000:0.0:1787739052.273545:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665523840 00000100:00100000:0.0:1787739052.273610:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1163 00000100:00100000:0.0:1787739052.273620:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be43c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+7:3652:x1874579665523840:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739052.273636:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739052.273658:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab51d lrc: 0/0,0 mode: --/EX res: [0x240000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab516 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.273673:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb5:0x0].0x0 (ffffa0730608b400) 00010000:00010000:0.0:1787739052.273682:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739052.273713:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be43c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+6:3652:x1874579665523840:12345-0@lo:103:ldlm_bl.0 Request processed in 96us (585us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739052.273720:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1163 00000100:00100000:2.0:1787739052.273919:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665523840:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739052.321220:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739052.321312:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652410752 00000100:00100000:1.0:1787739052.321330:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652410752 00000100:00100000:1.0:1787739052.321405:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23149 00000100:00100000:1.0:1787739052.321450:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17410:x1874579652410752:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739052.321482:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739052.321514:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab524 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094a61 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.321603:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab52b lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.321620:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab52b lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.321632:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab52b lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.321646:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab532 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.321652:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab532 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.321659:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab532 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.321712:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab52b lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.321720:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab52b lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.321726:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab52b lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.321739:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab52b lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.321745:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739052.321750:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab52b lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.321758:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab52b lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.321942:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235800) 00010000:00010000:1.0:1787739052.321945:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739052.321950:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab52b lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.321957:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab52b lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.321973:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab532 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.321980:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab532 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.321986:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab532 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.321992:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739052.321995:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab532 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.322004:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab532 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.322011:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xa7fab2a2 (ffffa07433235200) 00010000:00010000:1.0:1787739052.322014:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739052.322017:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab532 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.322023:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab532 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.322057:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab524 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a61 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.322068:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235800) 00010000:00010000:1.0:1787739052.322071:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739052.322075:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab524 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a61 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.322086:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c600, rc 0) 00000100:00100000:1.0:1787739052.322525:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17410:x1874579652410752:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 1077us (1315us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739052.322535:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23149 00000100:00100000:3.0:1787739052.343470:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739052.343606:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652410880 00000100:00100000:2.0:1787739052.343619:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652410880 00000100:00100000:2.0:1787739052.343691:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23150 00000100:00100000:2.0:1787739052.343703:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17410:x1874579652410880:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739052.343729:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739052.343759:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab539 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094a68 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.343852:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab540 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.343879:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab540 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.343893:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab540 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.343908:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab547 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.343916:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab547 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.343923:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab547 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.343975:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab540 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.343982:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab540 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.343989:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab540 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.343995:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab540 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.344002:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633cc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739052.344007:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab540 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.344015:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab540 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.344031:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403eeb300) 00010000:00010000:2.0:1787739052.344035:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739052.344041:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab540 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.344049:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab540 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.344063:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab547 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.344069:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab547 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.344076:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab547 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.344081:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739052.344084:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab547 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.344103:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab547 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.344112:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xa7fab2a2 (ffffa07403eebb00) 00010000:00010000:2.0:1787739052.344115:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739052.344118:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab547 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.344124:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab547 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.344158:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab539 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a68 expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.344175:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403eeb300) 00010000:00010000:2.0:1787739052.344177:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739052.344181:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ab539 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a68 expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.344191:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c600, rc 0) 00000100:00100000:2.0:1787739052.345048:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17410:x1874579652410880:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 1340us (1585us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739052.345061:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23150 00010000:00010000:1.0:1787739052.355570:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739052.377914:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739052.378328:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652411008 00000100:00100000:0.0:1787739052.378342:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652411008 00000100:00100000:0.0:1787739052.378435:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23151 00000100:00100000:0.0:1787739052.378448:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17410:x1874579652411008:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739052.378475:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739052.378505:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab54e lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094a6f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.378582:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab555 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.378601:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab555 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.378615:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab555 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739052.378679:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:0.0:1787739052.378686:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739052.378729:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab54e lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a6f expref: 19 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.378748:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab555 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a6f expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.378758:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0741590d200) 00010000:00010000:0.0:1787739052.378761:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739052.378765:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609ac00, rc 0) 00000100:00100000:0.0:1787739052.378846:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17410:x1874579652411008:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 402us (956us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739052.378854:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23151 00000100:00100000:3.0:1787739052.409528:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739052.409651:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652411264 00000100:00100000:0.0:1787739052.409665:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652411264 00000100:00100000:0.0:1787739052.409741:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23152 00000100:00100000:0.0:1787739052.409754:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7130c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17410:x1874579652411264:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:0.0:1787739052.409942:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665523968:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739052.410015:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739052.410077:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404000 going to sleep for 11 seconds 00000100:00100000:0.0:1787739052.410114:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665523968 00000100:00100000:0.0:1787739052.410118:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665523968 00000100:00100000:0.0:1787739052.410151:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2483 00000100:00100000:0.0:1787739052.410157:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d21c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665523968:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739052.410239:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d21c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665523968:12345-0@lo:1000:mdt00_004.0 Request processed in 81us (223us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739052.410246:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2483 00000100:00100000:0.0:1787739052.410295:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665523968:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739052.410352:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665524096:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739052.410379:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739052.410396:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739052.410440:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665524096 00000100:00100000:0.0:1787739052.410444:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665524096 00000100:00100000:0.0:1787739052.410472:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2484 00000100:00100000:0.0:1787739052.410478:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665524096:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739052.410567:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665524096:12345-0@lo:1000:mdt00_004.0 Request processed in 89us (187us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739052.410574:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2484 00000100:00100000:0.0:1787739052.410602:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665524096:0@lo:1000:mdt00_004.0 00010000:00010000:0.0:1787739052.410664:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ab55c lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.410677:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ab55c lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.410689:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ab55c lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739052.410702:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665524224:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739052.410728:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739052.410748:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404000 going to sleep for 11 seconds 00000100:00100000:0.0:1787739052.410760:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665524224 00000100:00100000:0.0:1787739052.410763:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665524224 00000100:00100000:0.0:1787739052.410786:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2485 00000100:00100000:0.0:1787739052.410792:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d1e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665524224:12345-0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739052.410802:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739052.410821:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab563 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab55c expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.410851:0:17834:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab555 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a6f expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.410864:0:17834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab563 lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab55c expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.410892:0:17834:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab555 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a6f expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.410908:0:17834:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab555 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a6f expref: 17 pid: 6523 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739052.410919:0:17834:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d12c0 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:17834:1874579665524352:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739052.410982:0:17834:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739052.419013:0:17834:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d12c0 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:17834:1874579665524352:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739052.419068:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab563 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab55c expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.419102:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0741590d200) 00010000:00010000:1.0:1787739052.419106:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739052.419110:0:17834:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab563 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab55c expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.419122:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a800, rc 0) 00000100:00100000:1.0:1787739052.419193:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d1e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665524224:12345-0@lo:101:mdt00_004.0 Request processed in 8400us (8463us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739052.419203:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2485 00000100:00100000:0.0:1787739052.419288:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665524224:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739052.419310:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ab55c lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab563 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.419322:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ab55c lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab563 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739052.428068:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739052.434971:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652411392 00000100:00100000:2.0:1787739052.434991:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652411392 00010000:00010000:2.0:1787739052.435053:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab555 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a6f expref: 18 pid: 6523 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739052.435116:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1367 00000100:00100000:2.0:1787739052.435127:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2409:x1874579652411392:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739052.435145:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739052.435153:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab555 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a6f expref: 18 pid: 6523 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.435169:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab555 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a6f expref: 18 pid: 6523 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.435198:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab555 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a6f expref: 18 pid: 6523 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.435218:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0741590d200) 00010000:00010000:2.0:1787739052.435226:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab563 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab55c expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.435238:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab563 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab55c expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.435248:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab563 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab55c expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.435287:0:6509:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab563 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab55c expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739052.435304:0:6509:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f2d00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665524480:0@lo:105: 00000100:00100000:2.0:1787739052.435387:0:6509:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739052.435462:0:6509:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610eb40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739052.437195:0:6507:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:2.0:1787739052.437226:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665524480 00000100:00100000:2.0:1787739052.437236:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665524480 00000100:00100000:2.0:1787739052.437307:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1042 00000100:00100000:2.0:1787739052.437319:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07436dfec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665524480:12345-0@lo:105: 00010000:00010000:2.0:1787739052.437339:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ab55c lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab563 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.437353:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ab55c lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab563 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.437370:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ab55c lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab563 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.437380:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ab55c lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab563 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.437390:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ab55c lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab563 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.437400:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ab55c lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ab563 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.437458:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609be00) 00000100:00100000:2.0:1787739052.437544:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07436dfec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665524480:12345-0@lo:105: Request processed in 228us (2156us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739052.437553:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1042 00000100:00100000:2.0:1787739052.437643:0:6509:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f2d00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665524480:0@lo:105: 00010000:00010000:2.0:1787739052.437663:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739052.437719:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652411392:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 2592us (9658us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739052.437729:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1367 00010000:00010000:0.0:1787739052.438037:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ab55c lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab563 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.438049:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ab55c lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab563 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739052.438076:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739052.438137:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ab56a lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.438146:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ab56a lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.438157:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ab56a lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739052.438178:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665524608:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739052.438225:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739052.438256:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404000 going to sleep for 11 seconds 00000100:00100000:0.0:1787739052.438492:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665524608 00000100:00100000:0.0:1787739052.438499:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665524608 00000100:00100000:0.0:1787739052.438537:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2486 00000100:00100000:0.0:1787739052.438545:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d2580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665524608:12345-0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739052.438565:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739052.438584:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab571 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab56a expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.438610:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab571 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab56a expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.438625:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab571 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab56a expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.438634:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xa7fab2a2 (ffffa0741590de00) 00010000:00010000:0.0:1787739052.438637:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739052.438640:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609bc00, rc 0) 00000100:00100000:0.0:1787739052.438673:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d2580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665524608:12345-0@lo:101:mdt00_004.0 Request processed in 129us (449us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739052.438681:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2486 00000100:00100000:0.0:1787739052.441570:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665524608:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739052.441610:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ab56a lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab571 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.441628:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ab56a lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab571 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739052.441660:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:0.0:1787739052.441727:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665524736:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739052.441947:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739052.441984:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739052.442019:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665524736 00000100:00100000:0.0:1787739052.442028:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665524736 00000100:00100000:0.0:1787739052.442104:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2487 00000100:00100000:0.0:1787739052.442115:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d0b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665524736:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739052.442212:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d0b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665524736:12345-0@lo:1000:mdt00_004.0 Request processed in 98us (265us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739052.442220:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2487 00000100:00100000:0.0:1787739052.442261:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665524736:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739052.442309:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665524864:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739052.442333:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739052.442348:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739052.442360:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665524864 00000100:00100000:0.0:1787739052.442363:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665524864 00000100:00100000:0.0:1787739052.442385:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2488 00000100:00100000:0.0:1787739052.442390:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d12c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665524864:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739052.442489:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d12c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665524864:12345-0@lo:1000:mdt00_004.0 Request processed in 98us (154us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739052.442496:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2488 00000100:00100000:0.0:1787739052.442523:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665524864:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739052.442602:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665524992:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739052.442625:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739052.442638:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739052.442649:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665524992 00000100:00100000:0.0:1787739052.442651:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665524992 00000100:00100000:0.0:1787739052.442672:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2489 00000100:00100000:0.0:1787739052.442677:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665524992:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739052.442723:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665524992:12345-0@lo:1000:mdt00_004.0 Request processed in 45us (97us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739052.442729:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2489 00000100:00100000:0.0:1787739052.445289:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665524992:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739052.445409:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665525120:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739052.445508:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739052.445554:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739052.448140:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665525120 00000100:00100000:2.0:1787739052.448162:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665525120 00000100:00100000:2.0:1787739052.448233:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2490 00000100:00100000:2.0:1787739052.448244:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665525120:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739052.448407:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665525120:12345-0@lo:1000:mdt00_004.0 Request processed in 164us (2899us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739052.448435:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2490 00000100:00100000:0.0:1787739052.449449:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665525120:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739052.449565:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665525248:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739052.449631:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739052.449666:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739052.449692:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665525248 00000100:00100000:0.0:1787739052.449700:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665525248 00000100:00100000:0.0:1787739052.449747:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2491 00000100:00100000:0.0:1787739052.449755:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562f840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665525248:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739052.452129:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562f840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665525248:12345-0@lo:1000:mdt00_004.0 Request processed in 2362us (2486us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739052.452146:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2491 00000100:00100000:0.0:1787739052.452235:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665525248:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739052.452335:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665525376:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739052.452377:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739052.452406:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739052.452448:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665525376 00000100:00100000:0.0:1787739052.452456:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665525376 00000100:00100000:0.0:1787739052.452502:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2492 00000100:00100000:0.0:1787739052.452512:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665525376:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739052.452564:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665525376:12345-0@lo:1000:mdt00_004.0 Request processed in 53us (187us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739052.452571:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2492 00000100:00100000:0.0:1787739052.452597:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665525376:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739052.452626:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665525504:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739052.452649:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739052.452667:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739052.452680:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665525504 00000100:00100000:0.0:1787739052.452684:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665525504 00000100:00100000:0.0:1787739052.452702:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2493 00000100:00100000:0.0:1787739052.452707:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d0780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665525504:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739052.453492:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d0780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665525504:12345-0@lo:1000:mdt00_004.0 Request processed in 779us (837us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739052.453502:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2493 00000100:00100000:0.0:1787739052.453727:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665525504:0@lo:1000:mdt00_004.0 00000040:00080000:0.0:1787739052.460772:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].327, 2168 off499352 00000040:00080000:0.0:1787739052.461139:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].327, 2368 off545328 00000004:00080000:0.0:1787739052.461147:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07410404000 with version 623 00000004:00080000:0.0:1787739052.462568:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 623 rpc_version 623 00000020:00100000:0.0:1787739052.462635:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739052.462661:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743562e1c0 x1874579665525760/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739052.462692:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562e1c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665525760:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739052.462796:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739052.462837:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739052.462871:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665525760 00000100:00100000:0.0:1787739052.462879:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665525760 00000100:00100000:0.0:1787739052.462943:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2494 00000100:00100000:0.0:1787739052.462957:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665525760:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739052.463926:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:7467:x1874579665525760:12345-0@lo:1000:osp_up0-1.0 Request processed in 969us (1126us total) trans 4294983299 rc 0/0 00000100:00100000:0.0:1787739052.463937:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2494 00000100:00100000:0.0:1787739052.464038:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562e1c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665525760:0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1787739052.464193:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab578 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x15b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.464214:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab578 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x15b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.464230:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab578 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x15b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.464255:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ab57f lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.464264:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ab57f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.464271:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ab57f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.464311:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab586 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.464319:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab586 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.464332:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab586 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739052.464350:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d1a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665525888:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739052.464391:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739052.464441:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730638b180 going to sleep for 11 seconds 00000100:00100000:0.0:1787739052.464465:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665525888 00000100:00100000:0.0:1787739052.464472:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665525888 00000100:00100000:0.0:1787739052.464509:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2495 00000100:00100000:0.0:1787739052.464517:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d1680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665525888:12345-0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739052.464537:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739052.464556:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab58d lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab586 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.464578:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab58d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab586 expref: 12 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.464591:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab58d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab586 expref: 12 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.464602:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb3:0x0].0x0 (ffffa0741590d100) 00010000:00010000:0.0:1787739052.464605:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739052.464610:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a400, rc 0) 00000100:00100000:0.0:1787739052.464642:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d1680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:13793:x1874579665525888:12345-0@lo:101:mdt00_004.0 Request processed in 126us (251us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739052.464649:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2495 00000100:00100000:0.0:1787739052.464681:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d1a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665525888:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739052.464695:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab586 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab58d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.464706:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab586 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab58d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739052.464727:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000042c05826 reply state 000000008f47660f transno 4294968288 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ab57f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.464740:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ab57f lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.464748:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb3:0x0].0x0 (ffffa0741590d400) 00010000:00010000:0.0:1787739052.464750:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739052.464753:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739052.464776:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab586 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab58d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.464789:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab586 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab58d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739052.464800:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xb3:0x0] 00000004:00010000:0.0:1787739052.464807:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000042c05826 reply state 000000008f47660f transno 4294968288 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab578 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x15b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.464816:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab578 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x15b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.464822:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15b:0x0].0x0 (ffffa0741590d700) 00010000:00010000:0.0:1787739052.464824:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739052.464826:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787739052.464833:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x15b:0x0]: returning size 4096 00000004:00080000:0.0:1787739052.464841:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739052.464849:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ab55c lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab563 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.464856:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ab55c lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab563 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.464863:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ab55c lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab563 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.464881:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739052.464884:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739052.464888:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ab56a lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab571 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.464900:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ab56a lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab571 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739052.464989:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7130c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17410:x1874579652411264:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 55236us (55482us total) trans 4294968288 rc 0/0 00000100:00100000:0.0:1787739052.464997:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23152 00010000:00010000:0.0:1787739052.465024:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739052.465032:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ab55c lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab563 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.465040:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609be00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739052.465046:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ab55c lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab563 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739052.465055:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ab55c lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab563 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.465082:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ab55c lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab563 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.465100:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739052.465129:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ab55c lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab563 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.465138:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ab55c lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab563 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.465160:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739052.465162:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739052.465163:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739052.465174:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665526016:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739052.465210:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739052.467409:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665526016 00000100:00100000:2.0:1787739052.467510:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665526016 00000100:00100000:2.0:1787739052.467588:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1164 00000100:00100000:2.0:1787739052.467600:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+12:3650:x1874579665526016:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739052.467621:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739052.467661:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab563 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab55c expref: 12 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.467682:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0741590d200) 00010000:00010000:2.0:1787739052.467695:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739052.467783:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+11:3650:x1874579665526016:12345-0@lo:103:ldlm_bl.0 Request processed in 185us (2572us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739052.467791:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1164 00000100:00100000:1.0:1787739052.467875:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665526016:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739052.522158:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739052.523057:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652411648 00000100:00100000:1.0:1787739052.523071:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652411648 00000100:00100000:1.0:1787739052.523148:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23153 00000100:00100000:1.0:1787739052.523160:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17420:x1874579652411648:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:1.0:1787739052.523383:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17420:x1874579652411648:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 225us (1239us total) trans 0 rc -17/-17 00000100:00100000:1.0:1787739052.523391:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23153 00000100:00100000:3.0:1787739052.553109:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739052.553810:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652411776 00000100:00100000:1.0:1787739052.553826:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652411776 00000100:00100000:1.0:1787739052.553910:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23154 00000100:00100000:1.0:1787739052.553922:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17421:x1874579652411776:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739052.553954:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739052.553987:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab594 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094a76 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.554083:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab59b lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.554110:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab59b lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.554125:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab59b lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.554139:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5a2 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.554151:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab571 lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab56a expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.554160:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5a2 lrc: 4/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.554204:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab571 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ab56a expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739052.554222:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665526144:0@lo:104: 00000100:00100000:1.0:1787739052.554300:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739052.554918:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fccb40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739052.564738:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665526144 00000100:00100000:2.0:1787739052.564769:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665526144 00000100:00100000:1.0:1787739052.565217:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1043 00000100:00100000:1.0:1787739052.565230:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665526144:12345-0@lo:104: 00010000:00010000:1.0:1787739052.565256:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ab56a lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab571 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739052.565473:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665526144:0@lo:104: 00010000:00010000:1.0:1787739052.565503:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5a2 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.565533:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787739052.565539:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665526144:12345-0@lo:104: Request processed in 312us (11239us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739052.565547:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1043 00010000:00010000:1.0:1787739052.565572:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739052.565580:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ab56a lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab571 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.565590:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609aa00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739052.565596:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ab56a lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ab571 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739052.565607:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ab56a lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ab571 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.565655:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ab56a lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab571 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.565662:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739052.565692:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ab56a lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab571 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.565700:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ab56a lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab571 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.565726:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739052.565728:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739052.565730:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739052.565761:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665526272:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739052.566170:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739052.566233:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665526272 00000100:00100000:1.0:1787739052.566240:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665526272 00010000:00010000:1.0:1787739052.566267:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab571 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ab56a expref: 11 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739052.566296:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1368 00000100:00100000:1.0:1787739052.566304:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba2850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3651:x1874579665526272:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739052.566318:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:1.0:1787739052.579385:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983299 is committed 00000001:00080000:1.0:1787739052.579468:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983298 is committed 00000020:00080000:1.0:1787739052.579479:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 341 batchid updated 00000020:00080000:1.0:1787739052.579498:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07435539240 refcount 2 committed 0 result 0 batchid 342 00000020:00080000:1.0:1787739052.579504:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c695f00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07435539d80 00000020:00080000:1.0:1787739052.579509:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].339 00000020:00080000:1.0:1787739052.579512:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c695800 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073069bfe00 00000020:00080000:1.0:1787739052.579515:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].339 00010000:00010000:1.0:1787739052.579605:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab571 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ab56a expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.579626:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xa7fab2a2 (ffffa0741590de00) 00010000:00010000:1.0:1787739052.579633:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5a2 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.579645:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5a2 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.579654:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5a2 lrc: 4/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.579682:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739052.579807:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba2850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+6:3651:x1874579665526272:12345-0@lo:103:ldlm_bl.0 Request processed in 13497us (13630us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739052.579818:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1368 00000100:00100000:1.0:1787739052.579924:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665526272:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739052.579955:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5a2 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.579963:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5a2 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739052.580114:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665526400:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739052.580158:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739052.580186:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739052.580210:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665526400 00000100:00100000:1.0:1787739052.580218:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665526400 00000100:00100000:1.0:1787739052.580267:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2496 00000100:00100000:1.0:1787739052.580276:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ec780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6524:x1874579665526400:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739052.580370:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ec780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6524:x1874579665526400:12345-0@lo:1000:mdt00_001.0 Request processed in 95us (211us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739052.580381:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2496 00000100:00100000:1.0:1787739052.580438:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665526400:0@lo:1000:mdt00_001.0 00010000:00010000:1.0:1787739052.580487:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab5a9 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x15b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.580501:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab5a9 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.580511:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab5a9 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739052.580537:0:6524:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab5a9 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.580553:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab59b lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.580559:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab59b lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.580565:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab59b lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.580572:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab59b lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.580578:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739052.580583:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab59b lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.580591:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab59b lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.580603:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235800) 00010000:00010000:1.0:1787739052.580606:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739052.580610:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab59b lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.580616:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab59b lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.580628:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5a2 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.580635:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5a2 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.580641:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5a2 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.580646:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739052.580649:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5a2 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.580654:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5a2 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.580661:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xa7fab2a2 (ffffa0741590de00) 00010000:00010000:1.0:1787739052.580663:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739052.580666:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5a2 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.580671:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5a2 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.580722:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab594 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a76 expref: 17 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.580741:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab5a9 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x15b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a76 expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.580748:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15b:0x0].0x0 (ffffa07433235200) 00010000:00010000:1.0:1787739052.580750:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739052.580753:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c200, rc 0) 00000100:00100000:1.0:1787739052.580811:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17421:x1874579652411776:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 26891us (27729us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739052.580819:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23154 00000020:00080000:2.0:1787739052.585315:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 342 committed batchid 341 00000004:00080000:2.0:1787739052.585553:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730610eb40 with version 649 00000020:00080000:2.0:1787739052.585585:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 341 cancel update log [0x240000401:0x3:0x0].338: rc = 0 00000020:00080000:2.0:1787739052.585636:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 341 cancel update log [0x200000400:0x3:0x0].338: rc = 0 00000020:00080000:2.0:1787739052.585647:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 341 00000004:00080000:1.0:1787739052.586330:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 649 rpc_version 649 00000020:00100000:1.0:1787739052.586395:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739052.586426:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074156ef0c0 x1874579665526656/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739052.586456:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665526656:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739052.586548:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739052.586606:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fccc00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739052.586765:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665526656 00000100:00100000:2.0:1787739052.586776:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665526656 00000100:00100000:2.0:1787739052.586863:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2497 00000100:00100000:2.0:1787739052.586875:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156efc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665526656:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739052.587064:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156efc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665526656:12345-0@lo:1000:osp_up1-0.0 Request processed in 190us (514us total) trans 4294968289 rc 0/0 00000100:00100000:2.0:1787739052.587073:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2497 00000100:00100000:1.0:1787739052.587772:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665526656:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787739052.588063:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab49f lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588076:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab49f lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588084:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab49f lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588102:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739052.588108:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab49f lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588117:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab49f lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588137:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xdf2a7902 (ffffa0730608b800) 00010000:00010000:0.0:1787739052.588140:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739052.588145:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab49f lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588152:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab49f lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588171:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab4a6 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x14f2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588178:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab4a6 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x14f2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588183:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab4a6 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x14f2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588188:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739052.588191:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab4a6 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x14f2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588197:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab4a6 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x14f2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588204:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f2:0x0].0x0 (ffffa0730608bd00) 00010000:00010000:0.0:1787739052.588207:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739052.588210:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab4a6 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x14f2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588216:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab4a6 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x14f2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588235:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab4ad lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588241:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab4ad lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588246:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab4ad lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588252:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739052.588254:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab4ad lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588259:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab4ad lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588267:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb5:0x0].0x0 (ffffa0730608b100) 00010000:00010000:0.0:1787739052.588269:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739052.588272:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab4ad lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588278:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab4ad lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588296:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab501 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588303:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab501 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588310:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab501 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588315:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739052.588317:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab501 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588323:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab501 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588480:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xdf2a7902 (ffffa0730608b800) 00010000:00010000:0.0:1787739052.588483:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739052.588487:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab501 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.588493:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab501 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xdf2a7902 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739052.592764:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739052.592895:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652411904 00000100:00100000:0.0:1787739052.592909:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652411904 00000100:00100000:0.0:1787739052.592989:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23155 00000100:00100000:0.0:1787739052.593001:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17421:x1874579652411904:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739052.593035:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739052.593062:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab5b0 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x15b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094a7d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.593155:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab5b7 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x15b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.593172:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab5b7 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.593186:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab5b7 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739052.593314:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x15b:0x0]: returning size 4096 00000004:00200000:0.0:1787739052.593321:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739052.593369:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab5b0 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x15b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a7d expref: 11 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.593398:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab5b7 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x15b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a7d expref: 10 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.593408:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15b:0x0].0x0 (ffffa0741590d700) 00010000:00010000:0.0:1787739052.593440:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739052.593445:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b600, rc 0) 00000100:00100000:0.0:1787739052.593525:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17421:x1874579652411904:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 527us (776us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739052.593534:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23155 00000100:00100000:1.0:1787739052.606528:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739052.606630:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652412032 00000100:00100000:0.0:1787739052.606644:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652412032 00000100:00100000:0.0:1787739052.606716:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23156 00000100:00100000:0.0:1787739052.606728:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156eda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17421:x1874579652412032:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739052.606756:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739052.606784:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ab5be lrc: 2/0,0 mode: --/CR res: [0x240000400:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094a84 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.606864:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab5c5 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.606883:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab5c5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.606897:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab5c5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739052.606958:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xb3:0x0]: returning size 4096 00000004:00200000:0.0:1787739052.606966:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739052.607013:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ab5be lrc: 0/0,0 mode: --/CR res: [0x240000400:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a84 expref: 12 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.607036:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab5c5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a84 expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.607046:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb3:0x0].0x0 (ffffa0741590d400) 00010000:00010000:0.0:1787739052.607050:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739052.607054:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a200, rc 0) 00000100:00100000:0.0:1787739052.607145:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156eda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17421:x1874579652412032:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 417us (633us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739052.607153:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23156 00000100:00100000:2.0:1787739052.620477:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739052.621853:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652412160 00000100:00100000:1.0:1787739052.621878:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652412160 00000100:00100000:1.0:1787739052.621965:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23157 00000100:00100000:1.0:1787739052.621979:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17421:x1874579652412160:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739052.622018:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739052.622046:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab5cc lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094a8b expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.622137:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5d3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.622158:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab58d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab586 expref: 5 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.622168:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5d3 lrc: 4/1,0 mode: --/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.622210:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab58d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ab586 expref: 5 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739052.622229:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f711e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665526784:0@lo:104: 00000100:00100000:1.0:1787739052.622301:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739052.622356:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fccc00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739052.622400:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665526784 00000100:00100000:0.0:1787739052.622406:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665526784 00000100:00100000:0.0:1787739052.622464:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1044 00000100:00100000:0.0:1787739052.622472:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7121c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665526784:12345-0@lo:104: 00010000:00010000:0.0:1787739052.622488:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab586 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab58d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.622608:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787739052.622615:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7121c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665526784:12345-0@lo:104: Request processed in 144us (312us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739052.622622:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1044 00000100:00100000:2.0:1787739052.622694:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f711e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665526784:0@lo:104: 00010000:00010000:2.0:1787739052.622717:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5d3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.623243:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739052.623252:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab586 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab58d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.623262:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739052.623269:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab586 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ab58d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739052.623279:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab586 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ab58d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.623319:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab586 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab58d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.623481:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1787739052.623531:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665526912:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739052.623587:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787739052.623636:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab586 lrc: 1/0,0 mode: --/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab58d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.623645:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ab586 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab58d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.623677:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739052.623679:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739052.623681:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739052.623693:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665526912 00000100:00100000:1.0:1787739052.623699:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665526912 00010000:00010000:1.0:1787739052.623726:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab58d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ab586 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739052.624516:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1369 00000100:00100000:1.0:1787739052.624529:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+6:3652:x1874579665526912:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739052.624547:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00080000:2.0:1787739052.629784:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 342 batchid updated 00000020:00080000:3.0:1787739052.629882:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 342 00000004:00080000:3.0:1787739052.629966:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa073070540c0 with version 650 00000020:00080000:3.0:1787739052.629994:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 342 cancel update log [0x240000401:0x3:0x0].339: rc = 0 00000020:00080000:3.0:1787739052.630117:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 342 cancel update log [0x200000400:0x3:0x0].339: rc = 0 00000020:00080000:3.0:1787739052.630127:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 342 00000004:00080000:2.0:1787739052.630210:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 650 rpc_version 650 00000020:00100000:2.0:1787739052.630256:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739052.630269:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074080f0f00 x1874579665527168/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739052.630302:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f0f00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665527168:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739052.630407:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739052.630475:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739052.630539:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665527168 00000100:00100000:3.0:1787739052.630548:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665527168 00000100:00100000:3.0:1787739052.630615:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2498 00000100:00100000:3.0:1787739052.630626:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665527168:12345-0@lo:1000:osp_up1-0.0 00010000:00010000:1.0:1787739052.630762:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab58d lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ab586 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.630796:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb3:0x0].0x0 (ffffa0741590d100) 00010000:00010000:1.0:1787739052.630803:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5d3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.630815:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5d3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739052.630816:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665527168:12345-0@lo:1000:osp_up1-0.0 Request processed in 191us (409us total) trans 4294968290 rc 0/0 00000100:00100000:3.0:1787739052.630824:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2498 00010000:00010000:1.0:1787739052.630825:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5d3 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739052.630882:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f0f00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665527168:0@lo:1000:osp_up1-0.0 00010000:00010000:1.0:1787739052.631149:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739052.631206:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+5:3652:x1874579665526912:12345-0@lo:103:ldlm_bl.0 Request processed in 6680us (7618us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739052.631216:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1369 00010000:00010000:2.0:1787739052.631230:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5d3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.631240:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5d3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739052.631308:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb3:0x0]: returning size 4096 00000004:00200000:2.0:1787739052.631316:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739052.631385:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab5cc lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a8b expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.631405:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5d3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a8b expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.631449:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb3:0x0].0x0 (ffffa0741590d100) 00010000:00010000:2.0:1787739052.631453:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739052.631456:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c600, rc 0) 00000100:00100000:2.0:1787739052.631512:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17421:x1874579652412160:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 9538us (11050us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739052.631521:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23157 00000100:00100000:3.0:1787739052.631705:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f3c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665526912:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739052.643231:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739052.643284:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739052.643321:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652412544 00000100:00100000:1.0:1787739052.643332:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652412544 00000100:00100000:1.0:1787739052.643403:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23158 00000100:00100000:1.0:1787739052.643437:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f711e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17421:x1874579652412544:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:1.0:1787739052.643517:0:7944:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739052.643554:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab5b7 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x15b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a7d expref: 11 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.643577:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15b:0x0].0x0 (ffffa0741590d700) 00010000:00010000:1.0:1787739052.643584:0:7944:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739052.643668:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ab5da lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.643679:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ab5da lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.643692:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ab5da lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739052.643720:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f77c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665527296:0@lo:101:mdt00_003.0 00000100:00100000:1.0:1787739052.643784:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739052.643820:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739052.643864:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665527296 00000100:00100000:1.0:1787739052.643868:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665527296 00000100:00100000:1.0:1787739052.643900:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2499 00000100:00100000:1.0:1787739052.643906:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f76d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+5:7944:x1874579665527296:12345-0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739052.643918:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739052.643936:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab5e1 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab5da expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.643962:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab5e1 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab5da expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.645109:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab5e1 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab5da expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.645127:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235900) 00010000:00010000:1.0:1787739052.645130:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739052.645132:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1de00, rc 0) 00000100:00100000:1.0:1787739052.645209:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f76d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:7944:x1874579665527296:12345-0@lo:101:mdt00_003.0 Request processed in 1301us (1422us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739052.645221:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2499 00000100:00100000:1.0:1787739052.645302:0:7944:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983299 gen 2 00000004:00080000:1.0:1787739052.645310:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983294 committed transno 4294983299 00000020:00080000:1.0:1787739052.645317:0:7944:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa0730610ed80 refcount 2 committed 0 result 0 batchid 333 00000020:00080000:1.0:1787739052.645322:0:7944:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2a00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730610e3c0 00000020:00080000:1.0:1787739052.645326:0:7944:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].326 00000020:00080000:1.0:1787739052.645329:0:7944:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2900 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa074184ef200 00000020:00080000:1.0:1787739052.645331:0:7944:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].326 00000004:00080000:1.0:1787739052.645360:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983296 committed transno 4294983299 00000004:00080000:1.0:1787739052.645376:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983297 committed transno 4294983299 00000004:00080000:1.0:1787739052.645388:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983299 committed transno 4294983299 00000100:00100000:1.0:1787739052.645429:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f77c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665527296:0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739052.645452:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ab5da lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab5e1 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.645465:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ab5da lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab5e1 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739052.645483:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739052.645519:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1ce00/0x7ee4453e4a1ab5e8 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.645527:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1ce00/0x7ee4453e4a1ab5e8 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.645537:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1ce00/0x7ee4453e4a1ab5e8 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739052.645558:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f77c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665527424:0@lo:101:mdt00_003.0 00000100:00100000:3.0:1787739052.645987:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652412288 00000100:00100000:3.0:1787739052.645995:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652412288 00000100:00100000:3.0:1787739052.646059:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1165 00000100:00100000:3.0:1787739052.646069:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2409:x1874579652412288:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:3.0:1787739052.646085:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739052.646122:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab5a9 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x15b:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a76 expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.646137:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15b:0x0].0x0 (ffffa07433235200) 00010000:00010000:3.0:1787739052.646147:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739052.646207:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652412288:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 139us (2989us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739052.646215:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1165 00000020:00080000:2.0:1787739052.646761:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 333 committed batchid 331 00000020:00080000:2.0:1787739052.647495:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 331 00000100:00100000:1.0:1787739052.649188:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739052.649276:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739052.649810:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665527424 00000100:00100000:2.0:1787739052.649825:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665527424 00000100:00100000:2.0:1787739052.649895:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2500 00000100:00100000:2.0:1787739052.649907:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f752c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:7944:x1874579665527424:12345-0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739052.649932:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739052.649966:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab5ef lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab5e8 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.650008:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab5ef lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab5e8 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.650028:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab5ef lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab5e8 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.650037:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xa7fab2a2 (ffffa07303c78c00) 00010000:00010000:2.0:1787739052.650040:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739052.650044:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d000, rc 0) 00000100:00100000:2.0:1787739052.650144:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f752c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665527424:12345-0@lo:101:mdt00_003.0 Request processed in 239us (961us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739052.650151:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2500 00000100:00100000:0.0:1787739052.651514:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f77c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665527424:0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739052.651551:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1ce00/0x7ee4453e4a1ab5e8 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab5ef expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.651566:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1ce00/0x7ee4453e4a1ab5e8 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab5ef expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739052.651593:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:0.0:1787739052.651668:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073040003c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665527552:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739052.651746:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739052.651783:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730638ba80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739052.654060:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665527552 00000100:00100000:1.0:1787739052.654080:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665527552 00000100:00100000:1.0:1787739052.654177:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2501 00000100:00100000:1.0:1787739052.654189:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304000b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665527552:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739052.654383:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304000b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665527552:12345-0@lo:1000:mdt00_003.0 Request processed in 194us (2636us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739052.654391:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2501 00000100:00100000:2.0:1787739052.654479:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073040003c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665527552:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739052.654573:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665527680:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739052.654624:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739052.654657:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739052.654675:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665527680 00000100:00100000:2.0:1787739052.654678:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665527680 00000100:00100000:2.0:1787739052.654721:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2502 00000100:00100000:2.0:1787739052.654728:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be43c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665527680:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739052.654822:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be43c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665527680:12345-0@lo:1000:mdt00_003.0 Request processed in 94us (198us total) trans 0 rc 1/1 00000100:00100000:2.0:1787739052.654829:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2502 00000100:00100000:2.0:1787739052.654854:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665527680:0@lo:1000:mdt00_003.0 00010000:00010000:2.0:1787739052.654921:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1ab5f6 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.654933:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1ab5f6 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.654944:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1ab5f6 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739052.654955:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665527808:0@lo:101:mdt00_003.0 00000100:00100000:2.0:1787739052.654980:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739052.654995:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739052.655179:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665527808 00000100:00100000:0.0:1787739052.655184:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665527808 00000100:00100000:0.0:1787739052.655223:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2503 00000100:00100000:0.0:1787739052.655231:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665527808:12345-0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739052.655243:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739052.655266:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ab5fd lrc: 2/0,0 mode: --/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab5f6 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.655294:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ab5fd lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab5f6 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.655311:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ab5fd lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab5f6 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.655322:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15b:0x0].0x0 (ffffa0741590d500) 00010000:00010000:0.0:1787739052.655325:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739052.655328:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a600, rc 0) 00000100:00100000:0.0:1787739052.655361:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665527808:12345-0@lo:101:mdt00_003.0 Request processed in 132us (380us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739052.655378:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2503 00000100:00100000:2.0:1787739052.655731:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665527808:0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739052.655748:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1ab5f6 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab5fd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.655760:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1ab5f6 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab5fd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.655798:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab604 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.655808:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab604 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.655817:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab604 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.655838:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab60b lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.655853:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab5c5 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a84 expref: 10 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.655862:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab60b lrc: 4/0,1 mode: --/EX res: [0x240000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.655891:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab5c5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a84 expref: 10 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.655909:0:7944:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab5c5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a84 expref: 10 pid: 6523 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739052.655921:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665527936:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739052.656021:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739052.677488:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665527936:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739052.677532:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab60b lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739052.678192:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739052.678272:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652412672 00000100:00100000:1.0:1787739052.678283:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652412672 00010000:00010000:1.0:1787739052.678340:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab5c5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a84 expref: 11 pid: 6523 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739052.678393:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1370 00000100:00100000:1.0:1787739052.678405:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2408:x1874579652412672:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739052.678451:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739052.678460:0:6510:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab5c5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a84 expref: 11 pid: 6523 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.678472:0:6510:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab5c5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a84 expref: 11 pid: 6523 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.678500:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab5c5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a84 expref: 11 pid: 6523 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.678524:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb3:0x0].0x0 (ffffa0741590d400) 00010000:00010000:1.0:1787739052.678529:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab60b lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.678540:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab60b lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.678550:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab60b lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.678581:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739052.678648:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652412672:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 244us (461us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739052.678657:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1370 00010000:00010000:3.0:1787739052.679046:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab60b lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.679059:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab60b lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.679130:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab612 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.679139:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab612 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.679150:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab612 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739052.679176:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356aed00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665528064:0@lo:101:mdt00_003.0 00000100:00100000:3.0:1787739052.679238:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739052.679299:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665528064 00000100:00100000:3.0:1787739052.679305:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665528064 00000100:00100000:3.0:1787739052.679347:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2504 00000100:00100000:3.0:1787739052.679355:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665528064:12345-0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739052.679387:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739052.679405:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab619 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab612 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.679460:0:17834:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5d3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a8b expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.679470:0:17834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab619 lrc: 4/0,0 mode: --/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab612 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.679499:0:17834:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5d3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a8b expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.679514:0:17834:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5d3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a8b expref: 13 pid: 6524 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739052.679525:0:17834:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c780 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:17834:1874579665528192:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739052.679588:0:17834:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739052.679672:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739052.698084:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739052.700095:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652412800 00000100:00100000:1.0:1787739052.700116:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652412800 00010000:00010000:1.0:1787739052.700184:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5d3 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a8b expref: 14 pid: 6524 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739052.700238:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1371 00000100:00100000:1.0:1787739052.700249:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c30c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652412800:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739052.700286:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739052.700296:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5d3 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a8b expref: 14 pid: 6524 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.700312:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5d3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a8b expref: 14 pid: 6524 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.700352:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb3:0x0].0x0 (ffffa0741590d100) 00010000:00010000:1.0:1787739052.700361:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab619 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab612 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.700384:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab619 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab612 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.700392:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab619 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab612 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.700458:0:6509:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab619 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab612 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739052.700476:0:6509:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665528320:0@lo:105: 00000100:00100000:1.0:1787739052.700561:0:6509:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739052.700622:0:6509:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fccf00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739052.700656:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665528320 00000100:00100000:1.0:1787739052.700660:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665528320 00000100:00100000:1.0:1787739052.700698:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1045 00000100:00100000:1.0:1787739052.700705:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f77c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665528320:12345-0@lo:105: 00010000:00010000:1.0:1787739052.700718:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab612 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab619 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.700729:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab612 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab619 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.700737:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab612 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab619 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.700758:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab612 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab619 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.700764:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab612 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab619 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.700773:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab612 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ab619 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.700789:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069aac00) 00000100:00100000:1.0:1787739052.700829:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f77c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665528320:12345-0@lo:105: Request processed in 125us (268us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739052.700836:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1045 00000100:00100000:1.0:1787739052.700887:0:6509:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665528320:0@lo:105: 00010000:00010000:1.0:1787739052.700904:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739052.700949:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c30c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652412800:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 702us (2894us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739052.700956:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1371 00010000:00010000:3.0:1787739052.702823:0:17834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab5d3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a8b expref: 13 pid: 6524 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739052.702857:0:17834:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c780 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:17834:1874579665528192:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739052.702893:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab619 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab612 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.702906:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb3:0x0].0x0 (ffffa0741590d100) 00010000:00010000:3.0:1787739052.702909:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739052.702912:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab600, rc 0) 00000100:00100000:3.0:1787739052.703018:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665528064:12345-0@lo:101:mdt00_003.0 Request processed in 23663us (23779us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739052.703028:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2504 00000100:00100000:2.0:1787739052.704375:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356aed00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665528064:0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739052.704409:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab612 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab619 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739052.705201:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c0780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665528448:0@lo:403:mdt00_003.0 00000100:00100000:2.0:1787739052.705309:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739052.705358:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739052.706323:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665528448 00000100:00100000:1.0:1787739052.706337:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665528448 00000100:00100000:1.0:1787739052.706438:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2505 00000100:00100000:1.0:1787739052.706450:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c03c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665528448:12345-0@lo:403:mdt00_003.0 00000100:00100000:1.0:1787739052.720666:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c03c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665528448:12345-0@lo:403:mdt00_003.0 Request processed in 14202us (15341us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739052.720687:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2505 00000100:00100000:2.0:1787739052.732998:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c0780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665528448:0@lo:403:mdt00_003.0 00000040:00080000:2.0:1787739052.734133:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].328, 952 off500304 00000040:00080000:2.0:1787739052.734361:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].328, 1032 off546360 00000004:00080000:2.0:1787739052.734368:0:7944:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730610e3c0 with version 624 00000004:00080000:2.0:1787739052.734437:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 624 rpc_version 624 00000020:00100000:2.0:1787739052.734476:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739052.734487:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073043c0780 x1874579665528704/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739052.734514:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c0780 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665528704:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739052.734601:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739052.734645:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739052.735666:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665528704 00000100:00100000:0.0:1787739052.735681:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665528704 00000100:00100000:0.0:1787739052.735752:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2506 00000100:00100000:0.0:1787739052.735763:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c21c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665528704:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739052.736333:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c21c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665528704:12345-0@lo:1000:osp_up0-1.0 Request processed in 569us (1729us total) trans 4294983300 rc 0/0 00000100:00100000:0.0:1787739052.736344:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2506 00000100:00100000:2.0:1787739052.738758:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c0780 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665528704:0@lo:1000:osp_up0-1.0 00000004:00200000:2.0:1787739052.738825:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x15b:0x0]: returning size 4096 00010000:00010000:2.0:1787739052.738857:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab60b lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.738874:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab60b lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.738882:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab60b lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.738890:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739052.738895:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab60b lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.738903:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab60b lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.738923:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb3:0x0].0x0 (ffffa0741590d400) 00010000:00010000:2.0:1787739052.738926:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739052.738931:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab60b lrc: 2/0,0 mode: --/EX res: [0x240000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.738938:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab60b lrc: 0/0,0 mode: --/EX res: [0x240000400:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.738956:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab612 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab619 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.738965:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab612 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab619 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.738972:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab612 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab619 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.739006:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739052.739017:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xb3:0x0] 00010000:00010000:2.0:1787739052.739026:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab604 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.739033:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab604 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.739039:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab604 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.739044:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739052.739047:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab604 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.739054:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab604 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.739062:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15b:0x0].0x0 (ffffa0741590d700) 00010000:00010000:2.0:1787739052.739065:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739052.739068:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab604 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.739086:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab604 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.739105:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1ab5f6 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab5fd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.739112:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1ab5f6 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab5fd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.739120:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1ab5f6 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab5fd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.739139:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:2.0:1787739052.739153:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739052.739163:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1787739052.739170:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab612 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab619 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.739177:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aac00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739052.739182:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab612 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab619 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739052.739191:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab612 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab619 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.739237:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab612 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab619 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.739245:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739052.739283:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab612 lrc: 1/0,0 mode: --/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab619 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.739291:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ab612 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab619 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.739310:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739052.739313:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1ab5f6 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab5fd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.739321:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739052.739325:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1ab5f6 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab5fd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739052.739332:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1ab5f6 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab5fd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.739350:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1ab5f6 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab5fd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.739357:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739052.739473:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739052.740949:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356aed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665528832:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739052.741049:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787739052.741163:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ab5da lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab5e1 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.741177:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ab5da lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab5e1 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.741186:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ab5da lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab5e1 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.741209:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739052.741219:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739052.741229:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739052.741235:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ab5da lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab5e1 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.741244:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1cc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739052.741250:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ab5da lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab5e1 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739052.741276:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ab5da lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab5e1 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.741319:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ab5da lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab5e1 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.741328:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739052.741523:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1ab5f6 lrc: 1/0,0 mode: --/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab5fd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.741534:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1ab5f6 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab5fd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.741558:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739052.741560:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739052.741562:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739052.741579:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665528960:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739052.741618:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739052.741700:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665528832 00000100:00100000:0.0:1787739052.741709:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665528832 00000100:00100000:0.0:1787739052.741763:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665528960 00000100:00100000:0.0:1787739052.741765:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665528960 00000100:00100000:0.0:1787739052.741789:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1166 00000100:00100000:0.0:1787739052.741798:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3653:x1874579665528832:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739052.741813:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739052.741835:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab619 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab612 expref: 11 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.741851:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb3:0x0].0x0 (ffffa0741590d100) 00010000:00010000:0.0:1787739052.741863:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739052.741904:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+10:3653:x1874579665528832:12345-0@lo:103:ldlm_bl.0 Request processed in 107us (856us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739052.741912:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1166 00000100:00100000:0.0:1787739052.741929:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1167 00000100:00100000:0.0:1787739052.741935:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+9:3650:x1874579665528960:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739052.741942:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739052.741953:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ab5fd lrc: 0/0,0 mode: --/EX res: [0x240000402:0x15b:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab5f6 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.741963:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15b:0x0].0x0 (ffffa0741590d500) 00010000:00010000:0.0:1787739052.741969:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739052.748408:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ab5da lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab5e1 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.748445:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ab5da lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab5e1 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.748477:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739052.748479:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739052.748482:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739052.748567:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356aed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665528832:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739052.748611:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665529088:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739052.748693:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739052.748762:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665529088 00000100:00100000:3.0:1787739052.748770:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665529088 00000100:00100000:3.0:1787739052.748830:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1168 00000100:00100000:3.0:1787739052.748841:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba0050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+9:3651:x1874579665529088:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739052.748861:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739052.748891:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab5e1 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab5da expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.748914:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235900) 00010000:00010000:3.0:1787739052.748925:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739052.748969:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba0050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+8:3651:x1874579665529088:12345-0@lo:103:ldlm_bl.0 Request processed in 129us (275us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739052.748977:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1168 00000100:00100000:3.0:1787739052.749011:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665529088:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739052.749037:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1ce00/0x7ee4453e4a1ab5e8 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab5ef expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.749047:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1ce00/0x7ee4453e4a1ab5e8 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab5ef expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739052.749170:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f711e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17421:x1874579652412544:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 105754us (105881us total) trans 4294968291 rc 0/0 00000100:00100000:3.0:1787739052.749177:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23158 00000100:00100000:0.0:1787739052.758519:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+7:3650:x1874579665528960:12345-0@lo:103:ldlm_bl.0 Request processed in 16568us (16884us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739052.758540:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1167 00000100:00100000:2.0:1787739052.759564:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665528960:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739052.818479:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739052.818669:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652412928 00000100:00100000:0.0:1787739052.818682:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652412928 00000100:00100000:0.0:1787739052.818766:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23159 00000100:00100000:0.0:1787739052.818778:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17422:x1874579652412928:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739052.818812:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739052.818839:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ab620 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094a92 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.818929:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ab627 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.818946:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ab627 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.818960:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ab627 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.818977:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab62e lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.818985:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab62e lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.818996:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab62e lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.819056:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ab627 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.819065:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ab627 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.819073:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ab627 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.819080:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ab627 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.819099:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609aa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739052.819105:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ab627 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.819114:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ab627 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.819127:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b900) 00010000:00010000:0.0:1787739052.819131:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739052.819137:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ab627 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.819144:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ab627 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.819161:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab62e lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.819167:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab62e lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.819174:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab62e lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.819178:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739052.819181:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab62e lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.819187:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab62e lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.819194:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x82d7929c (ffffa0741590d500) 00010000:00010000:0.0:1787739052.819197:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739052.819199:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab62e lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.819206:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ab62e lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.819244:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ab620 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a92 expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.819255:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b900) 00010000:00010000:0.0:1787739052.819258:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739052.819261:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ab620 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a92 expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.819270:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a600, rc 0) 00000100:00100000:0.0:1787739052.819354:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17422:x1874579652412928:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 577us (890us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739052.819363:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23159 00000100:00100000:3.0:1787739052.837552:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739052.837661:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652413056 00000100:00100000:0.0:1787739052.837676:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652413056 00000100:00100000:0.0:1787739052.837755:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23160 00000100:00100000:0.0:1787739052.837768:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17422:x1874579652413056:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739052.837797:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739052.837839:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab635 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094a99 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.837922:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab63c lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.837940:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab63c lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.837952:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab63c lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739052.838016:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:0.0:1787739052.838022:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739052.838066:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab635 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a99 expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.838109:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab63c lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a99 expref: 14 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.838119:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0741590d100) 00010000:00010000:0.0:1787739052.838124:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739052.838128:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a000, rc 0) 00000100:00100000:0.0:1787739052.838205:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17422:x1874579652413056:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 440us (667us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739052.838213:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23160 00000100:00100000:0.0:1787739052.850327:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739052.850598:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652413312 00000100:00100000:3.0:1787739052.850612:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652413312 00000100:00100000:3.0:1787739052.850719:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23161 00000100:00100000:3.0:1787739052.850730:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073040003c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17422:x1874579652413312:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:3.0:1787739052.850870:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665529216:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739052.850937:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739052.851024:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665529216 00000100:00100000:3.0:1787739052.851027:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739052.851036:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665529216 00000100:00100000:2.0:1787739052.851123:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2507 00000100:00100000:2.0:1787739052.851133:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665529216:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739052.851272:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665529216:12345-0@lo:1000:mdt00_003.0 Request processed in 139us (334us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739052.851280:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2507 00000100:00100000:3.0:1787739052.852099:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665529216:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739052.852164:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665529344:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739052.852206:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739052.852250:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739052.853672:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665529344 00000100:00100000:2.0:1787739052.853689:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665529344 00000100:00100000:2.0:1787739052.853766:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2508 00000100:00100000:2.0:1787739052.853776:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665529344:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739052.853972:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665529344:12345-0@lo:1000:mdt00_003.0 Request processed in 196us (1763us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739052.853981:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2508 00000100:00100000:2.0:1787739052.854037:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665529344:0@lo:1000:mdt00_003.0 00010000:00010000:2.0:1787739052.854148:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1ab643 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.854161:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1ab643 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.854174:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1ab643 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739052.854199:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb6940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665529472:0@lo:101:mdt00_003.0 00000100:00100000:2.0:1787739052.854242:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739052.854558:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739052.855436:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665529472 00000100:00100000:3.0:1787739052.855449:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665529472 00000100:00100000:3.0:1787739052.855504:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2509 00000100:00100000:3.0:1787739052.855514:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb70c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665529472:12345-0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739052.855531:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739052.855550:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab64a lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab643 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.855579:0:8447:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab1d5 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949b9 expref: 9 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.855590:0:8447:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab64a lrc: 4/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab643 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.855623:0:8447:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab1d5 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949b9 expref: 9 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.855637:0:8447:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab1d5 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949b9 expref: 9 pid: 6524 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739052.855650:0:8447:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8447:1874579665529600:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739052.855725:0:8447:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739052.866464:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739052.867250:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652413440 00000100:00100000:3.0:1787739052.867267:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652413440 00010000:00010000:3.0:1787739052.867314:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab1d5 lrc: 5/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949b9 expref: 10 pid: 6524 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739052.867377:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1372 00000100:00100000:3.0:1787739052.867388:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652413440:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739052.867429:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739052.867443:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab1d5 lrc: 5/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949b9 expref: 10 pid: 6524 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.867459:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab1d5 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949b9 expref: 10 pid: 6524 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.867485:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b900) 00010000:00010000:3.0:1787739052.867493:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab64a lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab643 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.867505:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab64a lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab643 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.867515:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab64a lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab643 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.867551:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab64a lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab643 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739052.867568:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c780 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665529728:0@lo:105: 00000100:00100000:3.0:1787739052.867633:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739052.867682:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054780 going to sleep for 11 seconds 00010000:00010000:3.0:1787739052.867722:0:8447:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab1d5 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0949b9 expref: 10 pid: 6524 timeout: 739 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739052.867735:0:8447:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8447:1874579665529600:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739052.867774:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab64a lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab643 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.867783:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b900) 00010000:00010000:3.0:1787739052.867785:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739052.867788:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab400, rc 0) 00000100:00100000:3.0:1787739052.867844:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb70c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665529472:12345-0@lo:101:mdt00_003.0 Request processed in 12332us (13601us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739052.867852:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2509 00000100:00100000:0.0:1787739052.868707:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb6940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665529472:0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739052.868727:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1ab643 lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab64a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.868748:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1ab643 lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab64a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739052.869012:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665529728 00000100:00100000:1.0:1787739052.869019:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665529728 00000100:00100000:1.0:1787739052.869068:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1046 00000100:00100000:1.0:1787739052.869077:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356aed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665529728:12345-0@lo:105: 00010000:00010000:1.0:1787739052.869100:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1ab643 lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab64a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.869110:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1ab643 lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab64a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.869119:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1ab643 lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab64a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.869127:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1ab643 lrc: 5/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab64a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.869136:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1ab643 lrc: 6/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab64a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.869144:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1ab643 lrc: 6/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ab64a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739052.869163:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633d400) 00000100:00100000:1.0:1787739052.869736:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356aed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665529728:12345-0@lo:105: Request processed in 658us (2100us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739052.869746:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1046 00000100:00100000:2.0:1787739052.869930:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c780 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665529728:0@lo:105: 00010000:00010000:2.0:1787739052.869947:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739052.870008:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:2408:x1874579652413440:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 2619us (3559us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739052.870020:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1372 00010000:00010000:0.0:1787739052.870456:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1ab643 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab64a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.870468:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1ab643 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab64a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739052.870490:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739052.870531:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab651 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.870539:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab651 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.870549:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab651 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739052.870569:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d03c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665529856:0@lo:101:mdt00_003.0 00000100:00100000:0.0:1787739052.870610:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739052.870641:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730638b240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739052.870941:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665529856 00000100:00100000:2.0:1787739052.870947:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665529856 00000100:00100000:2.0:1787739052.871016:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2510 00000100:00100000:2.0:1787739052.871024:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665529856:12345-0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739052.871038:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739052.871055:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab658 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab651 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.871079:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab658 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab651 expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.871101:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab658 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab651 expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.871109:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x82d7929c (ffffa07303c78f00) 00010000:00010000:2.0:1787739052.871112:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739052.871114:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633da00, rc 0) 00000100:00100000:2.0:1787739052.871154:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665529856:12345-0@lo:101:mdt00_003.0 Request processed in 131us (544us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739052.871161:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2510 00000100:00100000:0.0:1787739052.871393:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d03c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665529856:0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739052.871491:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab651 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab658 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.871503:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab651 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab658 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739052.871519:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:0.0:1787739052.871551:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d03c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665529984:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739052.871579:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739052.871592:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730638b000 going to sleep for 11 seconds 00000100:00100000:1.0:1787739052.871650:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665529984 00000100:00100000:1.0:1787739052.871654:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665529984 00000100:00100000:1.0:1787739052.871688:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2511 00000100:00100000:1.0:1787739052.871694:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c30c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665529984:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739052.871796:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c30c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665529984:12345-0@lo:1000:mdt00_003.0 Request processed in 101us (215us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739052.871803:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2511 00000100:00100000:1.0:1787739052.871831:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d03c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665529984:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739052.871869:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d03c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665530112:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739052.871917:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739052.871934:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc240 going to sleep for 11 seconds 00000100:00100000:1.0:1787739052.871945:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665530112 00000100:00100000:1.0:1787739052.871948:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665530112 00000100:00100000:1.0:1787739052.871986:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2512 00000100:00100000:1.0:1787739052.871992:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665530112:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739052.872062:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665530112:12345-0@lo:1000:mdt00_003.0 Request processed in 69us (165us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739052.872069:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2512 00000100:00100000:1.0:1787739052.872105:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d03c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665530112:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739052.872172:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d03c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665530240:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739052.874907:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739052.874959:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739052.875334:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665530240 00000100:00100000:2.0:1787739052.875345:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665530240 00000100:00100000:2.0:1787739052.875398:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2513 00000100:00100000:2.0:1787739052.875434:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665530240:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739052.875556:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665530240:12345-0@lo:1000:mdt00_003.0 Request processed in 149us (654us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739052.875564:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2513 00000100:00100000:3.0:1787739052.876029:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d03c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665530240:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739052.876621:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d03c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665530368:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739052.876663:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739052.876697:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054780 going to sleep for 11 seconds 00000100:00100000:2.0:1787739052.876751:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665530368 00000100:00100000:2.0:1787739052.876755:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665530368 00000100:00100000:2.0:1787739052.876792:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2514 00000100:00100000:2.0:1787739052.876798:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665530368:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739052.876859:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665530368:12345-0@lo:1000:mdt00_003.0 Request processed in 60us (195us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739052.876865:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2514 00000100:00100000:3.0:1787739052.877104:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d03c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665530368:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739052.877145:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665530496:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739052.877166:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739052.877190:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054780 going to sleep for 11 seconds 00000100:00100000:1.0:1787739052.877292:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665530496 00000100:00100000:1.0:1787739052.877297:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665530496 00000100:00100000:1.0:1787739052.877327:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2515 00000100:00100000:1.0:1787739052.877334:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665530496:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739052.877654:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665530496:12345-0@lo:1000:mdt00_003.0 Request processed in 319us (485us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739052.877662:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2515 00000100:00100000:1.0:1787739052.877689:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665530496:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739052.877726:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710f00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665530624:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739052.877821:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739052.877838:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739052.877848:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665530624 00000100:00100000:1.0:1787739052.877851:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665530624 00000100:00100000:1.0:1787739052.877867:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2516 00000100:00100000:1.0:1787739052.877872:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665530624:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739052.877915:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665530624:12345-0@lo:1000:mdt00_003.0 Request processed in 42us (94us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739052.877921:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2516 00000100:00100000:1.0:1787739052.877941:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710f00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665530624:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739052.878139:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710f00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665530752:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739052.878178:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739052.878511:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739052.878953:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665530752 00000100:00100000:2.0:1787739052.878958:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665530752 00000100:00100000:2.0:1787739052.879170:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2517 00000100:00100000:2.0:1787739052.879179:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ecf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665530752:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739052.879651:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ecf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665530752:12345-0@lo:1000:mdt00_003.0 Request processed in 472us (1472us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739052.879658:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2517 00000100:00100000:2.0:1787739052.880188:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710f00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665530752:0@lo:1000:mdt00_003.0 00000040:00080000:3.0:1787739052.887361:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].342, 2168 off526456 00000040:00080000:2.0:1787739052.888501:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].341, 2368 off569048 00000004:00080000:2.0:1787739052.888509:0:7944:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730610ee40 with version 651 00000004:00080000:3.0:1787739052.888753:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 651 rpc_version 651 00000020:00100000:3.0:1787739052.888912:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739052.888924:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07305656940 x1874579665531008/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739052.888948:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305656940 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665531008:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739052.889039:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739052.889082:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054780 going to sleep for 11 seconds 00000100:00100000:2.0:1787739052.890751:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665531008 00000100:00100000:2.0:1787739052.890772:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665531008 00000100:00100000:2.0:1787739052.890835:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2518 00000100:00100000:2.0:1787739052.890846:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665531008:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739052.893312:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:7740:x1874579665531008:12345-0@lo:1000:osp_up1-0.0 Request processed in 2462us (4266us total) trans 4294968292 rc 0/0 00000100:00100000:2.0:1787739052.893325:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2518 00000100:00100000:3.0:1787739052.899942:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305656940 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665531008:0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1787739052.900401:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab65f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x14f5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.900508:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab65f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14f5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.900520:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab65f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14f5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.900541:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab666 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.900550:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab666 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.900556:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab666 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.901179:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ab66d lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.901188:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ab66d lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.901200:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ab66d lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739052.901223:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb7840 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665531136:0@lo:101:mdt00_003.0 00000100:00100000:2.0:1787739052.901289:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739052.901373:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e840 going to sleep for 11 seconds 00000100:00100000:3.0:1787739052.901562:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665531136 00000100:00100000:3.0:1787739052.901569:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665531136 00000100:00100000:2.0:1787739052.902245:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2519 00000100:00100000:2.0:1787739052.902257:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb6940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:7944:x1874579665531136:12345-0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739052.902282:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739052.902299:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1ab674 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab66d expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.902324:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1ab674 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab66d expref: 12 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.902337:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1ab674 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab66d expref: 12 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739052.902345:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb6:0x0].0x0 (ffffa07303c78600) 00010000:00010000:2.0:1787739052.902347:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739052.902350:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c200, rc 0) 00000100:00100000:2.0:1787739052.902404:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb6940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:7944:x1874579665531136:12345-0@lo:101:mdt00_003.0 Request processed in 150us (1115us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739052.902437:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2519 00000100:00100000:3.0:1787739052.902510:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb7840 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665531136:0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739052.902526:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ab66d lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab674 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.902535:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ab66d lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab674 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739052.902558:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c3cf8569 reply state 000000001c053b32 transno 4294983301 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab666 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.902571:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab666 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.902577:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb6:0x0].0x0 (ffffa07303c78000) 00010000:00010000:3.0:1787739052.902580:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739052.902582:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787739052.902599:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ab66d lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab674 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.902607:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ab66d lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab674 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739052.902624:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xb6:0x0] 00000004:00010000:3.0:1787739052.902631:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c3cf8569 reply state 000000001c053b32 transno 4294983301 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab65f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14f5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.902639:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab65f lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x14f5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.902644:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f5:0x0].0x0 (ffffa07303c78400) 00010000:00010000:3.0:1787739052.902647:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739052.902649:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:3.0:1787739052.902654:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14f5:0x0]: returning size 4096 00000004:00080000:3.0:1787739052.902664:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739052.902672:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1ab643 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab64a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.902678:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1ab643 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab64a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.902684:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1ab643 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab64a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.902710:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739052.902714:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739052.902718:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab651 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab658 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739052.902725:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab651 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab658 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739052.903065:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073040003c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17422:x1874579652413312:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 52326us (52741us total) trans 4294983301 rc 0/0 00000100:00100000:3.0:1787739052.903073:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23161 00010000:00010000:0.0:1787739052.903496:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739052.903504:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1ab643 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab64a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.903512:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739052.903517:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1ab643 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab64a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739052.903525:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1ab643 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab64a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.903558:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1ab643 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab64a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.903565:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739052.903600:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1ab643 lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab64a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.903608:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1ab643 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab64a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.903629:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739052.903630:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739052.903632:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739052.903639:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787739052.903651:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a31e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665531264:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739052.903691:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739052.903736:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665531264 00000100:00100000:0.0:1787739052.903746:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665531264 00000100:00100000:0.0:1787739052.904014:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1169 00000100:00100000:0.0:1787739052.904022:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+12:3653:x1874579665531264:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739052.904033:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739052.904050:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab64a lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab643 expref: 12 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739052.904063:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b900) 00010000:00010000:0.0:1787739052.904070:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739052.904111:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+11:3653:x1874579665531264:12345-0@lo:103:ldlm_bl.0 Request processed in 91us (420us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739052.904118:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1169 00000100:00100000:0.0:1787739052.904151:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a31e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665531264:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739052.944618:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739052.944880:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652413696 00000100:00100000:3.0:1787739052.944890:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652413696 00000100:00100000:3.0:1787739052.944952:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23162 00000100:00100000:3.0:1787739052.944962:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17423:x1874579652413696:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:3.0:1787739052.945141:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17423:x1874579652413696:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 180us (530us total) trans 0 rc -17/-17 00000100:00100000:3.0:1787739052.945150:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23162 00000100:00100000:3.0:1787739053.013217:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739053.013256:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652413824 00000100:00100000:3.0:1787739053.013265:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652413824 00000100:00100000:3.0:1787739053.013323:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23163 00000100:00100000:3.0:1787739053.013343:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17424:x1874579652413824:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739053.013363:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00080000:2.0:1787739053.014686:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:3.0:1787739053.015247:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab67b lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094aa0 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.015320:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab682 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.015335:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab682 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.015345:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab682 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.015357:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab689 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.015377:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab658 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab651 expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.015386:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab689 lrc: 4/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.015452:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab658 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ab651 expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739053.015467:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383da40 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665531392:0@lo:104: 00000100:00100000:3.0:1787739053.015527:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.015582:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739053.016287:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665531392 00000100:00100000:1.0:1787739053.016295:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665531392 00000100:00100000:1.0:1787739053.016362:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1047 00000100:00100000:1.0:1787739053.016380:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6525:x1874579665531392:12345-0@lo:104: 00010000:00010000:1.0:1787739053.016394:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab651 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab658 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.016486:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787739053.016493:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6525:x1874579665531392:12345-0@lo:104: Request processed in 124us (966us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.016500:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1047 00010000:00010000:1.0:1787739053.016520:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739053.016526:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab651 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab658 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.016535:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739053.016540:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab651 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ab658 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739053.016548:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab651 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ab658 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.016576:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab651 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab658 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.016583:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739053.016609:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab651 lrc: 1/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab658 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.016616:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab651 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab658 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.016635:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739053.016636:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739053.016638:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739053.016694:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304260f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665531520:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739053.016725:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.020787:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383da40 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665531392:0@lo:104: 00010000:00010000:3.0:1787739053.020816:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab689 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739053.021106:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665531520 00000100:00100000:1.0:1787739053.021114:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665531520 00010000:00010000:1.0:1787739053.021155:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab658 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ab651 expref: 11 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739053.021197:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1373 00000100:00100000:1.0:1787739053.021206:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+11:3653:x1874579665531520:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739053.021220:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739053.025569:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1ce00/0x7ee4453e4a1ab5e8 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab5ef expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739053.025660:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1ce00/0x7ee4453e4a1ab5e8 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab5ef expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.025707:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1ce00/0x7ee4453e4a1ab5e8 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab5ef expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.025719:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1787739053.025787:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665531648:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:2.0:1787739053.025874:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.026015:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665531648 00000100:00100000:1.0:1787739053.026023:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665531648 00000100:00100000:1.0:1787739053.026075:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1170 00000100:00100000:1.0:1787739053.026083:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+7:3650:x1874579665531648:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:1.0:1787739053.026106:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739053.026126:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab5ef lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab5e8 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.026143:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xa7fab2a2 (ffffa07303c78c00) 00010000:00010000:1.0:1787739053.026155:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739053.026263:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1ce00/0x7ee4453e4a1ab5e8 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xa7fab2a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab5ef expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787739053.026278:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968292 is committed 00000001:00080000:3.0:1787739053.026293:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968291 is committed 00000020:00080000:3.0:1787739053.026299:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 333 batchid updated 00000020:00080000:3.0:1787739053.026307:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07410404300 refcount 2 committed 0 result 0 batchid 334 00000020:00080000:3.0:1787739053.026310:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07302ea6300 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07410404000 00000020:00080000:3.0:1787739053.026313:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].327 00000020:00080000:3.0:1787739053.026315:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07302ea6800 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa074357c1600 00000020:00080000:3.0:1787739053.026318:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].327 00000020:00080000:3.0:1787739053.026340:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 332 batchid updated 00010000:00010000:3.0:1787739053.026369:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab578 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x15b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.026377:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab578 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x15b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.026384:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab578 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x15b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.026389:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ba00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739053.026393:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab578 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x15b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.026399:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab578 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x15b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.026432:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15b:0x0].0x0 (ffffa0741590d700) 00010000:00010000:3.0:1787739053.026435:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739053.026439:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab578 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x15b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.026445:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab578 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x15b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.026458:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ab57f lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.026463:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ab57f lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.026468:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ab57f lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.026472:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739053.026475:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ab57f lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.026479:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ab57f lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.026486:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb3:0x0].0x0 (ffffa0741590d400) 00010000:00010000:3.0:1787739053.026487:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739053.026490:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ab57f lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.026495:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ab57f lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xb3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1787739053.026515:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 334 committed batchid 333 00000004:00080000:3.0:1787739053.026630:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07435539540 with version 625 00000020:00080000:3.0:1787739053.026640:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 332 cancel update log [0x200000401:0x3:0x0].325: rc = 0 00000020:00080000:3.0:1787739053.026676:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 332 cancel update log [0x240000400:0x3:0x0].325: rc = 0 00000004:00080000:3.0:1787739053.026694:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07435539a80 with version 626 00000020:00080000:3.0:1787739053.026697:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 333 cancel update log [0x200000401:0x3:0x0].326: rc = 0 00000020:00080000:3.0:1787739053.026714:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 333 cancel update log [0x240000400:0x3:0x0].326: rc = 0 00000020:00080000:3.0:1787739053.026719:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 333 00010000:00010000:3.0:1787739053.028725:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab658 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ab651 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.029702:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x82d7929c (ffffa07303c78f00) 00010000:00010000:3.0:1787739053.029709:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab689 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.029720:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab689 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.029728:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab689 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.030037:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739053.030139:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+6:3653:x1874579665531520:12345-0@lo:103:ldlm_bl.0 Request processed in 8931us (13410us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.030150:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1373 00000100:00100000:3.0:1787739053.030218:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304260f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665531520:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739053.030257:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab689 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.030267:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab689 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739053.030393:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383ed00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665531776:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739053.033043:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.033098:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054780 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.033121:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665531776 00000100:00100000:3.0:1787739053.033128:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665531776 00000100:00100000:3.0:1787739053.033193:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2520 00000100:00100000:3.0:1787739053.033202:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f7840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6525:x1874579665531776:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739053.033290:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f7840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6525:x1874579665531776:12345-0@lo:1000:mdt00_002.0 Request processed in 90us (251us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.033299:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2520 00000100:00100000:3.0:1787739053.033343:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383ed00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665531776:0@lo:1000:mdt00_002.0 00010000:00010000:3.0:1787739053.033439:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab690 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14f5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.033455:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab690 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.033466:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab690 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739053.033487:0:6525:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab690 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.033502:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab682 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.033513:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab682 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.033518:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab682 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.033525:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab682 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.033531:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aaa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739053.033535:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab682 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.033542:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab682 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.033554:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b300) 00010000:00010000:3.0:1787739053.033556:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739053.033560:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab682 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.033566:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab682 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.033576:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab689 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.033582:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab689 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.033587:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab689 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.033592:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739053.033595:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab689 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.033600:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab689 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.033607:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x82d7929c (ffffa07303c78f00) 00010000:00010000:3.0:1787739053.033609:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739053.033611:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab689 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.033617:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab689 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.033651:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab67b lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aa0 expref: 8 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.033666:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab690 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aa0 expref: 7 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.033674:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f5:0x0].0x0 (ffffa074331c2500) 00010000:00010000:3.0:1787739053.033676:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739053.033678:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abc00, rc 0) 00000100:00100000:3.0:1787739053.033721:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17424:x1874579652413824:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 20380us (20514us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.033729:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23163 00000004:00080000:2.0:1787739053.037586:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 625 rpc_version 625 00000020:00100000:2.0:1787739053.037627:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739053.037638:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07409fb7840 x1874579665532032/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739053.037664:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb7840 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665532032:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739053.037731:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.037784:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ed80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.038631:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665532032 00000100:00100000:0.0:1787739053.038639:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665532032 00000100:00100000:0.0:1787739053.038693:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2521 00000100:00100000:0.0:1787739053.038703:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665532032:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739053.039372:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665532032:12345-0@lo:1000:osp_up0-1.0 Request processed in 671us (1639us total) trans 4294983302 rc 0/0 00000100:00100000:0.0:1787739053.039381:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2521 00000100:00100000:3.0:1787739053.041121:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb7840 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665532032:0@lo:1000:osp_up0-1.0 00000004:00080000:3.0:1787739053.041139:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 626 rpc_version 626 00000020:00100000:3.0:1787739053.041170:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739053.041178:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074074f5680 x1874579665532288/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739053.041198:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f5680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665532288:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739053.041250:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.041290:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.041443:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665532288 00000100:00100000:0.0:1787739053.041449:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665532288 00000100:00100000:0.0:1787739053.041495:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2522 00000100:00100000:0.0:1787739053.041503:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f6d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665532288:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739053.041633:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f6d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665532288:12345-0@lo:1000:osp_up0-1.0 Request processed in 130us (381us total) trans 4294983303 rc 0/0 00000100:00100000:0.0:1787739053.041641:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2522 00000100:00100000:3.0:1787739053.041962:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f5680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665532288:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739053.042072:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+8:3650:x1874579665531648:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 15988us (16197us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.042081:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1170 00000100:00100000:2.0:1787739053.044352:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383da40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665531648:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787739053.055075:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739053.055223:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652413952 00000100:00100000:3.0:1787739053.055251:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652413952 00000100:00100000:3.0:1787739053.055356:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23164 00000100:00100000:3.0:1787739053.055375:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304260f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17424:x1874579652413952:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739053.055399:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739053.055476:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab697 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x14f5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094aa7 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.055598:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab69e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14f5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.055613:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab69e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14f5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.055626:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab69e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14f5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739053.055756:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14f5:0x0]: returning size 4096 00000004:00200000:3.0:1787739053.055762:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739053.055799:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab697 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x14f5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aa7 expref: 18 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.055834:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab69e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14f5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aa7 expref: 17 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.055852:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f5:0x0].0x0 (ffffa07303c78400) 00010000:00010000:3.0:1787739053.055855:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739053.055859:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab200, rc 0) 00000100:00100000:3.0:1787739053.055944:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304260f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17424:x1874579652413952:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 579us (875us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.055951:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23164 00000100:00080000:2.0:1787739053.057290:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787739053.067612:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739053.067766:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652414080 00000100:00100000:2.0:1787739053.067776:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652414080 00000100:00100000:2.0:1787739053.067837:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23165 00000100:00100000:2.0:1787739053.067848:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a31a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17424:x1874579652414080:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739053.067868:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739053.067888:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab6a5 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094aae expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.067948:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab6ac lrc: 3/1,0 mode: --/PR res: [0x200000400:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.067963:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab6ac lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.067976:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab6ac lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739053.068027:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xb6:0x0]: returning size 4096 00000004:00200000:2.0:1787739053.068032:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739053.068067:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab6a5 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aae expref: 19 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.068096:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab6ac lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aae expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.068105:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb6:0x0].0x0 (ffffa07303c78000) 00010000:00010000:2.0:1787739053.068107:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739053.068111:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d800, rc 0) 00000100:00100000:2.0:1787739053.068173:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a31a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17424:x1874579652414080:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 327us (571us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739053.068180:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23165 00000100:00100000:3.0:1787739053.077385:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739053.077491:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652414208 00000100:00100000:2.0:1787739053.077504:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652414208 00000100:00100000:2.0:1787739053.077563:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23166 00000100:00100000:2.0:1787739053.077575:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17424:x1874579652414208:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739053.077594:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739053.077614:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab6b3 lrc: 2/0,0 mode: --/CR res: [0x240000401:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094ab5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.077674:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab6ba lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.077689:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1ab674 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab66d expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.077697:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab6ba lrc: 4/1,0 mode: --/PR res: [0x240000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.077730:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1ab674 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ab66d expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739053.077758:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665532416:0@lo:104: 00000100:00100000:2.0:1787739053.077806:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.077844:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e840 going to sleep for 11 seconds 00000100:00100000:2.0:1787739053.077861:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665532416 00000100:00100000:2.0:1787739053.077865:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665532416 00000100:00100000:2.0:1787739053.077937:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1048 00000100:00100000:2.0:1787739053.077946:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564e580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6525:x1874579665532416:12345-0@lo:104: 00010000:00010000:2.0:1787739053.077958:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ab66d lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab674 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.078009:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739053.078015:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564e580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6525:x1874579665532416:12345-0@lo:104: Request processed in 71us (209us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739053.078021:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1048 00010000:00010000:2.0:1787739053.078045:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739053.078051:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ab66d lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab674 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.078059:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633de00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739053.078065:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ab66d lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xb6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ab674 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739053.078073:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ab66d lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xb6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ab674 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.078107:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ab66d lrc: 2/0,0 mode: --/PW res: [0x240000401:0xb6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab674 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.078114:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739053.078132:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ab66d lrc: 1/0,0 mode: --/PW res: [0x240000401:0xb6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab674 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.078139:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ab66d lrc: 0/0,0 mode: --/PW res: [0x240000401:0xb6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab674 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.078164:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739053.078166:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739053.078167:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739053.078190:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665532416:0@lo:104: 00010000:00010000:2.0:1787739053.078208:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab6ba lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739053.078224:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787739053.078235:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665532544:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739053.078262:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.078636:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665532544 00000100:00100000:0.0:1787739053.078641:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665532544 00010000:00010000:0.0:1787739053.078664:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1ab674 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ab66d expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.078741:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1374 00000100:00100000:0.0:1787739053.078749:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+6:3650:x1874579665532544:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739053.078760:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00080000:3.0:1787739053.083001:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 334 batchid updated 00000020:00080000:0.0:1787739053.083070:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 334 00000004:00080000:0.0:1787739053.083137:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730638b3c0 with version 627 00000020:00080000:0.0:1787739053.083155:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 334 cancel update log [0x200000401:0x3:0x0].327: rc = 0 00000020:00080000:0.0:1787739053.083232:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 334 cancel update log [0x240000400:0x3:0x0].327: rc = 0 00000020:00080000:0.0:1787739053.083240:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 334 00010000:00010000:0.0:1787739053.083300:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1ab674 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ab66d expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.083317:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb6:0x0].0x0 (ffffa07303c78600) 00010000:00010000:0.0:1787739053.083323:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab6ba lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.083332:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab6ba lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.083340:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab6ba lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.083378:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739053.083483:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+5:3650:x1874579665532544:12345-0@lo:103:ldlm_bl.0 Request processed in 4733us (5217us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739053.083492:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1374 00010000:00010000:1.0:1787739053.085265:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab6ba lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.085280:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab6ba lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739053.085342:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xb6:0x0]: returning size 4096 00000004:00200000:1.0:1787739053.085348:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739053.085395:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab6b3 lrc: 0/0,0 mode: --/CR res: [0x240000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ab5 expref: 9 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.085451:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab6ba lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ab5 expref: 8 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.085462:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb6:0x0].0x0 (ffffa07303c78600) 00010000:00010000:1.0:1787739053.085466:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739053.085468:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633da00, rc 0) 00000100:00100000:1.0:1787739053.085548:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17424:x1874579652414208:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 7974us (8179us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.085557:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23166 00000004:00080000:3.0:1787739053.085832:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 627 rpc_version 627 00000020:00100000:3.0:1787739053.085865:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739053.085873:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0730383da40 x1874579665532800/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739053.085893:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383da40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665532800:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739053.085944:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.085989:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28840 going to sleep for 11 seconds 00000100:00100000:2.0:1787739053.086319:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665532544:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739053.087001:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665532800 00000100:00100000:1.0:1787739053.087009:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665532800 00000100:00100000:1.0:1787739053.087060:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2523 00000100:00100000:1.0:1787739053.087069:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665532800:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739053.087322:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665532800:12345-0@lo:1000:osp_up0-1.0 Request processed in 252us (1375us total) trans 4294983304 rc 0/0 00000100:00100000:1.0:1787739053.087330:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2523 00000100:00100000:2.0:1787739053.091564:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383da40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665532800:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739053.094699:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739053.094812:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652414592 00000100:00100000:3.0:1787739053.094821:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652414592 00000100:00100000:3.0:1787739053.094879:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23167 00000100:00100000:3.0:1787739053.094888:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17424:x1874579652414592:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:3.0:1787739053.094942:0:13793:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739053.094968:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab69e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14f5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aa7 expref: 18 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.094990:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f5:0x0].0x0 (ffffa07303c78400) 00010000:00010000:3.0:1787739053.094994:0:13793:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739053.095040:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab6c1 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.095049:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab6c1 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.095060:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab6c1 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739053.095080:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fe940 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665532928:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787739053.095136:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.095173:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28600 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.095227:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665532928 00000100:00100000:0.0:1787739053.095232:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665532928 00000100:00100000:0.0:1787739053.095269:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2524 00000100:00100000:0.0:1787739053.095276:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fe1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+5:13793:x1874579665532928:12345-0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739053.095286:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739053.095301:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab6c8 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab6c1 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.095323:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab6c8 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab6c1 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.095342:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab6c8 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab6c1 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.095349:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0741590df00) 00010000:00010000:0.0:1787739053.095352:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739053.095355:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b600, rc 0) 00000100:00100000:0.0:1787739053.095395:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fe1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:13793:x1874579665532928:12345-0@lo:101:mdt00_004.0 Request processed in 119us (258us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739053.095402:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2524 00000100:00100000:0.0:1787739053.095468:0:13793:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968292 gen 2 00000004:00080000:0.0:1787739053.095473:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968287 committed transno 4294968292 00000020:00080000:0.0:1787739053.095479:0:13793:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07410b4d0c0 refcount 2 committed 0 result 0 batchid 343 00000020:00080000:0.0:1787739053.095483:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07302ea6400 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07410b4d000 00000020:00080000:0.0:1787739053.095486:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].340 00000020:00080000:0.0:1787739053.095487:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07302ea6d00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa074357c1a00 00000020:00080000:0.0:1787739053.095490:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].340 00000004:00080000:0.0:1787739053.095516:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968289 committed transno 4294968292 00000004:00080000:0.0:1787739053.095526:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968290 committed transno 4294968292 00000004:00080000:0.0:1787739053.095536:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968292 committed transno 4294968292 00000100:00100000:0.0:1787739053.095556:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fe940 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665532928:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739053.095570:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab6c1 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab6c8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.095579:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab6c1 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab6c8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739053.095598:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739053.095618:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab6cf lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.095625:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab6cf lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.095632:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab6cf lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.095644:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665533056:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739053.095670:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.095687:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730638b240 going to sleep for 11 seconds 00000020:00080000:0.0:1787739053.095698:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 343 committed batchid 342 00000020:00080000:0.0:1787739053.095759:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 342 00000100:00100000:0.0:1787739053.095770:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665533056 00000100:00100000:0.0:1787739053.095774:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665533056 00000100:00100000:0.0:1787739053.095801:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2525 00000100:00100000:0.0:1787739053.095807:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305656940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:13793:x1874579665533056:12345-0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739053.095815:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739053.095827:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab6d6 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab6cf expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.095842:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab6d6 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab6cf expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.095852:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab6d6 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab6cf expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.095863:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x82d7929c (ffffa0741590d400) 00010000:00010000:0.0:1787739053.095866:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739053.095868:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b800, rc 0) 00000100:00100000:0.0:1787739053.095898:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305656940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665533056:12345-0@lo:101:mdt00_004.0 Request processed in 91us (227us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739053.095904:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2525 00000100:00100000:0.0:1787739053.095934:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665533056:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739053.095944:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab6cf lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab6d6 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.095952:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab6cf lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab6d6 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739053.095964:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:0.0:1787739053.095994:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665533184:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739053.096020:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.096034:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730638b000 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.096047:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665533184 00000100:00100000:0.0:1787739053.096050:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665533184 00000100:00100000:0.0:1787739053.096072:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2526 00000100:00100000:0.0:1787739053.096078:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665533184:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739053.096147:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665533184:12345-0@lo:1000:mdt00_004.0 Request processed in 69us (126us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739053.096154:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2526 00000100:00100000:0.0:1787739053.096181:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665533184:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739053.096216:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665533312:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739053.096240:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.096253:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730638b000 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.096264:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665533312 00000100:00100000:0.0:1787739053.096268:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665533312 00000100:00100000:0.0:1787739053.096289:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2527 00000100:00100000:0.0:1787739053.096294:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665533312:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739053.096358:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665533312:12345-0@lo:1000:mdt00_004.0 Request processed in 63us (117us total) trans 0 rc 1/1 00000100:00100000:0.0:1787739053.096365:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2527 00000100:00100000:0.0:1787739053.096390:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665533312:0@lo:1000:mdt00_004.0 00010000:00010000:0.0:1787739053.096446:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1ab6dd lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.096454:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1ab6dd lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.096462:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1ab6dd lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.096473:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665533440:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739053.096495:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.096509:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730638b240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.096520:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665533440 00000100:00100000:0.0:1787739053.096523:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665533440 00000100:00100000:0.0:1787739053.096543:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2528 00000100:00100000:0.0:1787739053.096549:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665533440:12345-0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739053.096556:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739053.096568:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab6e4 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab6dd expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.096585:0:18535:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab690 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aa0 expref: 7 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.096593:0:18535:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab6e4 lrc: 4/0,0 mode: --/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab6dd expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.096616:0:18535:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab690 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aa0 expref: 7 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.096629:0:18535:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab690 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aa0 expref: 7 pid: 6525 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.096640:0:18535:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:18535:1874579665533568:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739053.096690:0:18535:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730638b000 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.108584:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00010000:00010000:0.0:1787739053.108679:0:18535:0:(ldlm_lockd.c:741:ldlm_handle_ast_error()) ### client (nid 192.168.203.15@tcp) returned -22 from blocking AST (req@000000008e28b10e x1874579665533568) - normal race ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab690 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aa0 expref: 7 pid: 6525 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.108702:0:18535:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab690 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aa0 expref: 7 pid: 6525 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.108727:0:18535:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab690 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aa0 expref: 7 pid: 6525 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.108754:0:18535:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:18535:1874579665533568:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739053.108776:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f5:0x0].0x0 (ffffa074331c2500) 00010000:00010000:0.0:1787739053.108784:0:18535:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab6e4 lrc: 3/0,0 mode: --/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab6dd expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.108796:0:18535:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab6e4 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab6dd expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.108806:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab6e4 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab6dd expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.108843:0:18535:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab6e4 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab6dd expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.108868:0:18535:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:18535:1874579665533696:0@lo:105: 00000100:00100000:0.0:1787739053.108912:0:18535:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.108951:0:18535:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730638b000 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.109109:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652414336 00000100:00100000:3.0:1787739053.109115:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652414336 00000100:00100000:3.0:1787739053.109173:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1171 00000100:00100000:3.0:1787739053.109183:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7a580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652414336:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:3.0:1787739053.109196:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739053.109198:0:6508:0:(ldlm_lockd.c:1754:ldlm_request_cancel()) ### server-side cancel handler stale lock (cookie 7ee4453e4a1ab690) 00010000:00010000:3.0:1787739053.109201:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739053.109244:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7a580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652414336:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 62us (670us total) trans 0 rc 116/116 00000100:00100000:3.0:1787739053.109252:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1171 00000100:00100000:3.0:1787739053.109721:0:6507:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:3.0:1787739053.109747:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665533696 00000100:00100000:3.0:1787739053.109751:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665533696 00000100:00100000:3.0:1787739053.109793:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1049 00000100:00100000:3.0:1787739053.109800:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743578c450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:18535:x1874579665533696:12345-0@lo:105: 00010000:00010000:3.0:1787739053.109811:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1ab6dd lrc: 4/0,1 mode: --/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab6e4 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.109821:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1ab6dd lrc: 4/0,1 mode: --/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab6e4 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.109829:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1ab6dd lrc: 4/0,1 mode: --/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab6e4 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.109836:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1ab6dd lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab6e4 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.109843:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1ab6dd lrc: 5/0,1 mode: EX/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab6e4 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.109850:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1ab6dd lrc: 5/0,1 mode: EX/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ab6e4 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.109873:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609b200) 00000100:00100000:3.0:1787739053.109917:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743578c450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:18535:x1874579665533696:12345-0@lo:105: Request processed in 118us (1004us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.109924:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1049 00000100:00100000:3.0:1787739053.109986:0:18535:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:18535:1874579665533696:0@lo:105: 00010000:00010000:3.0:1787739053.110004:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab6e4 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab6dd expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.110018:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f5:0x0].0x0 (ffffa074331c2500) 00010000:00010000:3.0:1787739053.110021:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739053.110023:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609bc00, rc 0) 00000100:00100000:3.0:1787739053.110062:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665533440:12345-0@lo:101:mdt00_004.0 Request processed in 13513us (13566us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.110068:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2528 00000100:00100000:3.0:1787739053.110114:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665533440:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739053.110128:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1ab6dd lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab6e4 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.110159:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab6eb lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.110170:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab6eb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.110178:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab6eb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.110196:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab6f2 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.110207:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab6ac lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aae expref: 17 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.110215:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab6f2 lrc: 4/0,1 mode: --/EX res: [0x200000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.110243:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab6ac lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aae expref: 17 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.110257:0:13793:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab6ac lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aae expref: 17 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739053.110268:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fd680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665533824:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739053.110318:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.114172:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739053.114235:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fd680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665533824:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739053.114262:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab6f2 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.114513:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652414720 00000100:00100000:0.0:1787739053.114520:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652414720 00010000:00010000:0.0:1787739053.114553:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab6ac lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aae expref: 18 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.114589:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1375 00000100:00100000:0.0:1787739053.114598:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba1c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2408:x1874579652414720:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739053.114615:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739053.114622:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab6ac lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aae expref: 18 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.114633:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab6ac lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aae expref: 18 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.114654:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab6ac lrc: 0/0,0 mode: --/PR res: [0x200000400:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aae expref: 18 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.114670:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb6:0x0].0x0 (ffffa07303c78000) 00010000:00010000:0.0:1787739053.114674:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab6f2 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.114684:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab6f2 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.114691:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab6f2 lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.114713:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739053.114774:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba1c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2408:x1874579652414720:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 176us (611us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739053.114782:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1375 00010000:00010000:0.0:1787739053.114915:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab6f2 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.114923:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab6f2 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.114966:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab6f9 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.114974:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab6f9 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.114984:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab6f9 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.115002:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7a580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665533952:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739053.115039:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.115063:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730638ba80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.115078:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665533952 00000100:00100000:0.0:1787739053.115081:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665533952 00000100:00100000:0.0:1787739053.115118:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2529 00000100:00100000:0.0:1787739053.115124:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665533952:12345-0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739053.115139:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739053.115154:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab700 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab6f9 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.115174:0:8447:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab6ba lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ab5 expref: 6 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.115182:0:8447:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab700 lrc: 4/0,0 mode: --/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab6f9 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.115205:0:8447:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab6ba lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ab5 expref: 6 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.115222:0:8447:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab6ba lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ab5 expref: 6 pid: 6525 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.115232:0:8447:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8447:1874579665534080:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739053.115274:0:8447:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730638b240 going to sleep for 11 seconds 00000100:00100000:1.0:1787739053.121245:0:8447:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8447:1874579665534080:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739053.121283:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab700 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab6f9 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.121296:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb6:0x0].0x0 (ffffa07303c78600) 00010000:00010000:1.0:1787739053.121299:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739053.121303:0:8447:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab700 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab6f9 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.121317:0:8447:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab700 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab6f9 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.121325:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609be00, rc 0) 00000100:00100000:1.0:1787739053.121398:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:13793:x1874579665533952:12345-0@lo:101:mdt00_004.0 Request processed in 6273us (6358us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.121407:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2529 00000100:00100000:0.0:1787739053.121570:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7a580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665533952:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739053.121588:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab6f9 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab700 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.121598:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab6f9 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab700 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.127117:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739053.127244:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652414848 00000100:00100000:2.0:1787739053.127254:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652414848 00010000:00010000:2.0:1787739053.127294:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab6ba lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ab5 expref: 7 pid: 6525 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739053.127326:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1376 00000100:00100000:2.0:1787739053.127335:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7bc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652414848:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739053.127350:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739053.127356:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab6ba lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ab5 expref: 7 pid: 6525 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.127370:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab6ba lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ab5 expref: 7 pid: 6525 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.127397:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab6ba lrc: 0/0,0 mode: --/PR res: [0x240000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ab5 expref: 7 pid: 6525 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.127436:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb6:0x0].0x0 (ffffa07303c78600) 00010000:00010000:2.0:1787739053.127444:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab700 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab6f9 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.127456:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab700 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab6f9 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.127465:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab700 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab6f9 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.127495:0:6509:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab700 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab6f9 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739053.127508:0:6509:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564de00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001_UUID:6509:1874579665534208:0@lo:105: 00000100:00100000:2.0:1787739053.127554:0:6509:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.127591:0:6509:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e840 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.128222:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665534208 00000100:00100000:0.0:1787739053.128229:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665534208 00000100:00100000:0.0:1787739053.128264:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1050 00000100:00100000:0.0:1787739053.128272:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6509:x1874579665534208:12345-0@lo:105: 00010000:00010000:0.0:1787739053.128283:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab6f9 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab700 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.128292:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab6f9 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab700 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.128300:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab6f9 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab700 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.128312:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab6f9 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab700 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.128318:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab6f9 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab700 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.128326:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab6f9 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ab700 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.128346:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609a600) 00000100:00100000:0.0:1787739053.128406:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6509:x1874579665534208:12345-0@lo:105: Request processed in 134us (850us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739053.128433:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1050 00010000:00010000:0.0:1787739053.128457:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab6f9 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab700 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.128468:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab6f9 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab700 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.129338:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7a580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665534336:0@lo:403:mdt00_004.0 00000100:00100000:0.0:1787739053.129438:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.129464:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730638ba80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.129486:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665534336 00000100:00100000:0.0:1787739053.129491:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665534336 00000100:00100000:0.0:1787739053.129526:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2530 00000100:00100000:0.0:1787739053.129533:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:13793:x1874579665534336:12345-0@lo:403:mdt00_004.0 00000100:00100000:0.0:1787739053.131255:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:13793:x1874579665534336:12345-0@lo:403:mdt00_004.0 Request processed in 1715us (1837us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739053.131267:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2530 00000100:00100000:1.0:1787739053.132510:0:6509:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564de00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001_UUID:6509:1874579665534208:0@lo:105: 00010000:00010000:1.0:1787739053.132533:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739053.132581:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7bc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2408:x1874579652414848:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 5246us (5473us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.132590:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1376 00000100:00100000:1.0:1787739053.135032:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7a580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665534336:0@lo:403:mdt00_004.0 00000040:00080000:1.0:1787739053.138163:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].343, 664 off527120 00000040:00080000:1.0:1787739053.138362:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].342, 744 off569792 00000004:00080000:1.0:1787739053.138369:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07305fcc780 with version 652 00000004:00080000:3.0:1787739053.139332:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 652 rpc_version 652 00000020:00100000:3.0:1787739053.139376:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739053.139385:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073061fc780 x1874579665534592/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2064/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739053.139408:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fc780 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665534592:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739053.139493:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.139534:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.139829:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665534592 00000100:00100000:0.0:1787739053.139836:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665534592 00000100:00100000:0.0:1787739053.140198:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2531 00000100:00100000:0.0:1787739053.140209:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07436d8e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665534592:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739053.140671:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07436d8e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665534592:12345-0@lo:1000:osp_up1-0.0 Request processed in 464us (1177us total) trans 4294968293 rc 0/0 00000100:00100000:0.0:1787739053.140680:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2531 00000100:00100000:3.0:1787739053.141120:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fc780 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665534592:0@lo:1000:osp_up1-0.0 00000004:00200000:1.0:1787739053.145562:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14f5:0x0]: returning size 4096 00010000:00010000:1.0:1787739053.145592:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab6f2 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.145604:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab6f2 lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.145612:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab6f2 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.145618:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739053.145624:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab6f2 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.145631:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab6f2 lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.145646:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb6:0x0].0x0 (ffffa07303c78000) 00010000:00010000:1.0:1787739053.145649:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739053.145654:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab6f2 lrc: 2/0,0 mode: --/EX res: [0x200000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.145660:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab6f2 lrc: 0/0,0 mode: --/EX res: [0x200000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.145673:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab6f9 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab700 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.145681:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab6f9 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab700 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.145688:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab6f9 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab700 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.145715:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739053.145725:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xb6:0x0] 00010000:00010000:1.0:1787739053.146801:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab6eb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.146814:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab6eb lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.146820:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab6eb lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.146826:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739053.146830:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab6eb lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.146837:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab6eb lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.146850:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f5:0x0].0x0 (ffffa07303c78400) 00010000:00010000:1.0:1787739053.146853:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739053.146929:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab6eb lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.146936:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab6eb lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.146950:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1ab6dd lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab6e4 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.146958:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1ab6dd lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab6e4 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.146964:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1ab6dd lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab6e4 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.146984:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:1.0:1787739053.146998:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739053.147003:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab6c1 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab6c8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.147015:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab6c1 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab6c8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.147022:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab6c1 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab6c8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.147035:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:1.0:1787739053.147040:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739053.147045:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab6cf lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab6d6 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.147052:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab6cf lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab6d6 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739053.147535:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17424:x1874579652414592:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 52643us (52837us total) trans 4294983305 rc 0/0 00000100:00100000:1.0:1787739053.147545:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23167 00010000:00010000:1.0:1787739053.147579:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1787739053.147585:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab6f9 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab700 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.147593:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739053.147597:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab6f9 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab700 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739053.147605:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab6f9 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab700 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.147637:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab6f9 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab700 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.147644:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739053.147665:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab6f9 lrc: 1/0,0 mode: --/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab700 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.147672:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1ab6f9 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab700 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.147688:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1787739053.147692:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1ab6dd lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab6e4 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.147698:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739053.147701:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1ab6dd lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab6e4 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739053.147708:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1ab6dd lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab6e4 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.147725:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1ab6dd lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab6e4 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.147951:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739053.147958:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab6c1 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab6c8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.147964:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739053.147969:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab6c1 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab6c8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739053.147975:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab6c1 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab6c8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.147995:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab6c1 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab6c8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.148001:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739053.148007:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab6c1 lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab6c8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.148018:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ab6c1 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab6c8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.148031:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739053.148032:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739053.148034:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739053.148705:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739053.148727:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1ab6dd lrc: 1/0,0 mode: --/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab6e4 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.148751:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1ab6dd lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab6e4 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.148766:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739053.148767:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739053.148769:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739053.148784:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665534848:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739053.148841:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787739053.149150:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739053.149163:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f78000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665534720:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739053.149196:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.149237:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787739053.149244:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665534976:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739053.149266:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.149543:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665534848 00000100:00100000:2.0:1787739053.149551:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665534848 00000100:00100000:2.0:1787739053.149599:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665534720 00000100:00100000:2.0:1787739053.149603:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665534720 00000100:00100000:2.0:1787739053.149623:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665534976 00000100:00100000:2.0:1787739053.149626:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665534976 00000100:00100000:2.0:1787739053.149641:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1172 00000100:00100000:2.0:1787739053.149649:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+12:3650:x1874579665534848:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739053.149667:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739053.149688:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab6e4 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14f5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab6dd expref: 12 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.149702:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f5:0x0].0x0 (ffffa074331c2500) 00010000:00010000:2.0:1787739053.149711:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739053.149765:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+11:3650:x1874579665534848:12345-0@lo:103:ldlm_bl.0 Request processed in 116us (924us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739053.149771:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1172 00000100:00100000:2.0:1787739053.149787:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1173 00000100:00100000:2.0:1787739053.149793:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+10:3652:x1874579665534720:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739053.149800:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739053.149810:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab700 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab6f9 expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.149821:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb6:0x0].0x0 (ffffa07303c78600) 00010000:00010000:2.0:1787739053.149828:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739053.149966:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+9:3652:x1874579665534720:12345-0@lo:103:ldlm_bl.0 Request processed in 60us (656us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739053.149976:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1173 00000100:00100000:2.0:1787739053.149991:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1174 00000100:00100000:2.0:1787739053.149997:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fcf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+8:3652:x1874579665534976:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739053.150006:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739053.150017:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab6c8 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab6c1 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.150031:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0741590df00) 00010000:00010000:2.0:1787739053.150038:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739053.150062:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fcf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+7:3652:x1874579665534976:12345-0@lo:103:ldlm_bl.0 Request processed in 66us (795us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739053.150068:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1174 00000100:00100000:2.0:1787739053.150173:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7a580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665534848:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739053.151122:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665534976:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739053.151134:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f78000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665534720:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739053.183461:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739053.183658:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652414976 00000100:00100000:1.0:1787739053.183668:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652414976 00000100:00100000:1.0:1787739053.183724:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23168 00000100:00100000:1.0:1787739053.183733:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17425:x1874579652414976:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739053.183763:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739053.183783:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ab707 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094abc expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.183846:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab70e lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.183860:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab70e lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.183884:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab70e lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.184107:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab715 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.184120:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab715 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.184128:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab715 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.184172:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab70e lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.184180:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab70e lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.184186:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab70e lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.184192:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab70e lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.184198:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739053.184202:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab70e lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.184209:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab70e lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.184220:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0741590d100) 00010000:00010000:1.0:1787739053.184223:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739053.184228:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab70e lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.184233:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab70e lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.184246:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab715 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.184252:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab715 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.184257:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab715 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.184262:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739053.184265:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab715 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.184270:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab715 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.184277:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xaa220cb4 (ffffa07433235d00) 00010000:00010000:1.0:1787739053.184279:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739053.184282:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab715 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.184287:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab715 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.184318:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ab707 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094abc expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.184328:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0741590d100) 00010000:00010000:1.0:1787739053.184330:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739053.184334:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ab707 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094abc expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.184343:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1dc00, rc 0) 00000100:00100000:1.0:1787739053.184655:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17425:x1874579652414976:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 921us (1199us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739053.184663:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23168 00000100:00100000:0.0:1787739053.200937:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739053.201301:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652415104 00000100:00100000:1.0:1787739053.201311:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652415104 00000100:00100000:1.0:1787739053.201362:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23169 00000100:00100000:1.0:1787739053.201384:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17425:x1874579652415104:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739053.201405:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739053.201453:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab71c lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094ac3 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.201512:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab723 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.201525:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab723 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.201537:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab723 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739053.201581:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x5:0x0]: returning size 4096 00000004:00200000:1.0:1787739053.201587:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739053.201622:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab71c lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ac3 expref: 8 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.201640:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab723 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ac3 expref: 7 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.201650:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07433235600) 00010000:00010000:1.0:1787739053.201653:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739053.201656:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c800, rc 0) 00000100:00100000:1.0:1787739053.201711:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17425:x1874579652415104:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 329us (782us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.201718:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23169 00000100:00100000:0.0:1787739053.217599:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739053.218000:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652415360 00000100:00100000:1.0:1787739053.218010:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652415360 00000100:00100000:1.0:1787739053.218070:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23170 00000100:00100000:1.0:1787739053.218099:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17425:x1874579652415360:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:1.0:1787739053.218213:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415701e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665535104:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739053.218305:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.218352:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739053.218374:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665535104 00000100:00100000:1.0:1787739053.218377:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665535104 00000100:00100000:1.0:1787739053.218406:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2532 00000100:00100000:1.0:1787739053.218437:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415700b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665535104:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739053.218505:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415700b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665535104:12345-0@lo:1000:mdt00_004.0 Request processed in 93us (199us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.218512:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2532 00000100:00100000:1.0:1787739053.218554:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415701e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665535104:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739053.218599:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415701e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665535232:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739053.218623:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.218646:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739053.218659:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665535232 00000100:00100000:1.0:1787739053.218662:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665535232 00000100:00100000:1.0:1787739053.218685:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2533 00000100:00100000:1.0:1787739053.218692:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c03c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665535232:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739053.218769:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c03c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665535232:12345-0@lo:1000:mdt00_004.0 Request processed in 77us (145us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739053.218775:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2533 00000100:00100000:1.0:1787739053.218799:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415701e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665535232:0@lo:1000:mdt00_004.0 00010000:00010000:1.0:1787739053.218847:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab72a lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.218869:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab72a lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.218878:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab72a lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739053.221240:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665535360:0@lo:101:mdt00_004.0 00000100:00100000:1.0:1787739053.221313:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.221360:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc180 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.223872:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665535360 00000100:00100000:0.0:1787739053.223886:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665535360 00000100:00100000:0.0:1787739053.228502:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2534 00000100:00100000:0.0:1787739053.228512:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c30c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665535360:12345-0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739053.228533:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739053.228554:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab731 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab72a expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.228587:0:17834:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab63c lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a99 expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.228597:0:17834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab731 lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab72a expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.228631:0:17834:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab63c lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a99 expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.228645:0:17834:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab63c lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a99 expref: 16 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.228659:0:17834:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432ed00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:17834:1874579665535488:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739053.228875:0:17834:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730638b240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.232137:0:17834:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432ed00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:17834:1874579665535488:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739053.232172:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab731 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab72a expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.232184:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0741590d100) 00010000:00010000:0.0:1787739053.232187:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739053.232191:0:17834:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab731 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab72a expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.232200:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a200, rc 0) 00000100:00100000:0.0:1787739053.232266:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c30c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665535360:12345-0@lo:101:mdt00_004.0 Request processed in 3755us (10952us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739053.232274:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2534 00000100:00100000:3.0:1787739053.232359:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665535360:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739053.232387:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab72a lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab731 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.232397:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab72a lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab731 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739053.276976:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739053.277168:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652415488 00000100:00100000:0.0:1787739053.277178:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652415488 00010000:00010000:0.0:1787739053.277218:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab63c lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a99 expref: 17 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.277258:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1377 00000100:00100000:0.0:1787739053.277267:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2408:x1874579652415488:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739053.277281:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739053.277287:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab63c lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a99 expref: 17 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.277304:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab63c lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a99 expref: 17 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.277324:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab63c lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094a99 expref: 17 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.277339:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0741590d100) 00010000:00010000:0.0:1787739053.277345:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab731 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab72a expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.277355:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab731 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab72a expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.277363:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab731 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab72a expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.277394:0:10340:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab731 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab72a expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.277408:0:10340:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432ed00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:10340:1874579665535616:0@lo:105: 00000100:00100000:0.0:1787739053.277479:0:10340:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.277513:0:10340:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730638b240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.277619:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665535616 00000100:00100000:0.0:1787739053.277624:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665535616 00000100:00100000:0.0:1787739053.277651:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1051 00000100:00100000:0.0:1787739053.277657:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d3840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:10340:x1874579665535616:12345-0@lo:105: 00010000:00010000:0.0:1787739053.277670:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab72a lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab731 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.277679:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab72a lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab731 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.277688:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab72a lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab731 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.277695:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab72a lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab731 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.277701:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab72a lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab731 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.277708:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab72a lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ab731 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.277724:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07441a1dc00) 00000100:00100000:0.0:1787739053.277758:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d3840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:10340:x1874579665535616:12345-0@lo:105: Request processed in 101us (278us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739053.277764:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1051 00000100:00100000:0.0:1787739053.277801:0:10340:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432ed00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:10340:1874579665535616:0@lo:105: 00010000:00010000:0.0:1787739053.277816:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739053.277849:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:2408:x1874579652415488:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 583us (884us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739053.277856:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1377 00010000:00010000:1.0:1787739053.281910:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab72a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab731 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.281928:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab72a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab731 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739053.281955:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739053.282003:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1ab738 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.282015:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1ab738 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.282028:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1ab738 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739053.282051:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665535744:0@lo:101:mdt00_004.0 00000100:00100000:1.0:1787739053.282122:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.282161:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.282274:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665535744 00000100:00100000:0.0:1787739053.282283:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665535744 00000100:00100000:0.0:1787739053.282343:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2535 00000100:00100000:0.0:1787739053.282352:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415701e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665535744:12345-0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739053.282369:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739053.282387:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ab73f lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab738 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.282437:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ab73f lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab738 expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.282453:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ab73f lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab738 expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.282461:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xaa220cb4 (ffffa0741590de00) 00010000:00010000:0.0:1787739053.282464:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739053.282467:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a600, rc 0) 00000100:00100000:0.0:1787739053.282509:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415701e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665535744:12345-0@lo:101:mdt00_004.0 Request processed in 157us (386us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739053.282515:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2535 00000100:00100000:0.0:1787739053.282565:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665535744:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739053.282581:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1ab738 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab73f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.282591:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1ab738 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab73f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739053.282604:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:0.0:1787739053.282635:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d21c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665535872:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739053.282664:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.282689:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730638b840 going to sleep for 11 seconds 00000100:00100000:1.0:1787739053.282706:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665535872 00000100:00100000:1.0:1787739053.282711:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665535872 00000100:00100000:1.0:1787739053.282744:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2536 00000100:00100000:1.0:1787739053.282750:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665535872:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739053.282808:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665535872:12345-0@lo:1000:mdt00_004.0 Request processed in 58us (143us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.282816:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2536 00000100:00100000:0.0:1787739053.282864:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d21c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665535872:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739053.283015:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d21c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665536000:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739053.283039:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.283070:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.283095:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665536000 00000100:00100000:0.0:1787739053.283099:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665536000 00000100:00100000:1.0:1787739053.287322:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2537 00000100:00100000:1.0:1787739053.287339:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d1e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665536000:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739053.287903:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d1e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665536000:12345-0@lo:1000:mdt00_004.0 Request processed in 566us (4860us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739053.287912:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2537 00000100:00100000:3.0:1787739053.290077:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d21c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665536000:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739053.290215:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061ffc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665536128:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739053.290285:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.290340:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739053.290655:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665536128 00000100:00100000:2.0:1787739053.290663:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665536128 00000100:00100000:2.0:1787739053.290717:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2538 00000100:00100000:2.0:1787739053.290726:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061ff0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665536128:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739053.290827:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061ff0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665536128:12345-0@lo:1000:mdt00_004.0 Request processed in 102us (542us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739053.290835:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2538 00000100:00100000:2.0:1787739053.290869:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061ffc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665536128:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739053.290922:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564d680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665536256:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739053.290949:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.290965:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ee40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739053.290977:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665536256 00000100:00100000:2.0:1787739053.290981:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665536256 00000100:00100000:2.0:1787739053.290999:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2539 00000100:00100000:2.0:1787739053.291004:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665536256:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739053.291056:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665536256:12345-0@lo:1000:mdt00_004.0 Request processed in 51us (106us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739053.291067:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2539 00000100:00100000:2.0:1787739053.291105:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564d680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665536256:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739053.291145:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564d680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665536384:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739053.291168:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.291182:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ee40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739053.291193:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665536384 00000100:00100000:2.0:1787739053.291196:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665536384 00000100:00100000:2.0:1787739053.291213:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2540 00000100:00100000:2.0:1787739053.291218:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665536384:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739053.291284:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665536384:12345-0@lo:1000:mdt00_004.0 Request processed in 65us (114us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739053.291290:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2540 00000100:00100000:0.0:1787739053.291649:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564d680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665536384:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739053.291696:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d2580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665536512:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739053.291726:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.291756:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739053.292071:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665536512 00000100:00100000:2.0:1787739053.292077:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665536512 00000100:00100000:2.0:1787739053.292128:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2541 00000100:00100000:2.0:1787739053.292136:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d0b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665536512:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739053.292210:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d0b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665536512:12345-0@lo:1000:mdt00_004.0 Request processed in 74us (482us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739053.292216:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2541 00000100:00100000:2.0:1787739053.292246:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d2580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665536512:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739053.292285:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564f0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665536640:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739053.292312:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.292336:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ea80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.292837:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665536640 00000100:00100000:0.0:1787739053.292843:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665536640 00000100:00100000:0.0:1787739053.292960:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2542 00000100:00100000:0.0:1787739053.292969:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665536640:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739053.293037:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665536640:12345-0@lo:1000:mdt00_004.0 Request processed in 69us (724us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739053.293223:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564f0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665536640:0@lo:1000:mdt00_004.0 00000040:00080000:2.0:1787739053.295288:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].329, 2168 off502472 00000040:00080000:2.0:1787739053.296752:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].329, 2368 off548728 00000004:00080000:2.0:1787739053.296767:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730610eb40 with version 628 00000004:00080000:1.0:1787739053.296907:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 628 rpc_version 628 00000020:00100000:1.0:1787739053.296952:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739053.296962:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304000b40 x1874579665536896/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739053.296986:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304000b40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665536896:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739053.297061:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.297140:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739053.298645:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665536896 00000100:00100000:2.0:1787739053.298657:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665536896 00000100:00100000:2.0:1787739053.298752:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2543 00000100:00100000:2.0:1787739053.298762:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157ead00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:7467:x1874579665536896:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739053.299478:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157ead00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+13:7467:x1874579665536896:12345-0@lo:1000:osp_up0-1.0 Request processed in 716us (2414us total) trans 4294983306 rc 0/0 00000100:00100000:2.0:1787739053.299487:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2543 00000100:00100000:1.0:1787739053.299621:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304000b40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665536896:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739053.299833:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2542 00010000:00010000:2.0:1787739053.300849:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab746 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x15d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.300866:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab746 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x15d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.300877:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab746 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x15d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.300896:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab74d lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.300903:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab74d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.300909:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab74d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.300941:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c000/0x7ee4453e4a1ab754 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.300953:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c000/0x7ee4453e4a1ab754 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.300963:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c000/0x7ee4453e4a1ab754 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739053.300985:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564f0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665537024:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1787739053.301036:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.301070:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0efc00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.301131:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665537024 00000100:00100000:3.0:1787739053.301137:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665537024 00000100:00100000:3.0:1787739053.301185:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2544 00000100:00100000:3.0:1787739053.301194:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564d680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:13793:x1874579665537024:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739053.301213:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739053.301229:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab75b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab754 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.301250:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab75b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab754 expref: 13 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.301263:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab75b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab754 expref: 13 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.301271:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb4:0x0].0x0 (ffffa074331c2f00) 00010000:00010000:3.0:1787739053.301275:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739053.301279:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab200, rc 0) 00000100:00100000:3.0:1787739053.301327:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564d680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+13:13793:x1874579665537024:12345-0@lo:101:mdt00_004.0 Request processed in 134us (291us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.301335:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2544 00000100:00100000:2.0:1787739053.301441:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564f0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665537024:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739053.301456:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c000/0x7ee4453e4a1ab754 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab75b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.301467:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c000/0x7ee4453e4a1ab754 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab75b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739053.301488:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000091a1489 reply state 00000000e142d2d3 transno 4294968294 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab74d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.301500:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab74d lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.301507:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb4:0x0].0x0 (ffffa07303c78e00) 00010000:00010000:2.0:1787739053.301509:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739053.301512:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787739053.301518:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c000/0x7ee4453e4a1ab754 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab75b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.301525:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c000/0x7ee4453e4a1ab754 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab75b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739053.301536:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xb4:0x0] 00000004:00010000:2.0:1787739053.301739:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000091a1489 reply state 00000000e142d2d3 transno 4294968294 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab746 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x15d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.301751:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab746 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x15d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.301757:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15d:0x0].0x0 (ffffa07303c78600) 00010000:00010000:2.0:1787739053.301759:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739053.301761:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:2.0:1787739053.301767:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x15d:0x0]: returning size 4096 00000004:00080000:2.0:1787739053.301776:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739053.301786:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab72a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab731 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.301794:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab72a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab731 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.301800:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab72a lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab731 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.301820:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739053.301825:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739053.301830:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1ab738 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab73f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.301837:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1ab738 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab73f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739053.301915:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17425:x1874579652415360:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 83834us (84324us total) trans 4294968294 rc 0/0 00000100:00100000:2.0:1787739053.301922:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23170 00010000:00010000:3.0:1787739053.302010:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739053.302018:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab72a lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab731 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.302026:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1dc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739053.302031:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab72a lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab731 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739053.302040:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab72a lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab731 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.302067:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab72a lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab731 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.302074:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739053.302105:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab72a lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab731 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.302113:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab72a lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab731 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.302129:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739053.302131:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739053.302132:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739053.302280:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fde00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665537152:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739053.302316:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.302646:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665537152 00000100:00100000:3.0:1787739053.302653:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665537152 00000100:00100000:3.0:1787739053.302707:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1175 00000100:00100000:3.0:1787739053.302715:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+13:3651:x1874579665537152:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739053.302726:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739053.302742:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab731 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab72a expref: 13 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.302754:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0741590d100) 00010000:00010000:3.0:1787739053.302761:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739053.302800:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+12:3651:x1874579665537152:12345-0@lo:103:ldlm_bl.0 Request processed in 87us (484us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.302806:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1175 00000100:00100000:2.0:1787739053.302891:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fde00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665537152:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739053.324056:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739053.324192:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652415744 00000100:00100000:0.0:1787739053.324206:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652415744 00000100:00100000:0.0:1787739053.324295:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23171 00000100:00100000:0.0:1787739053.324305:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17426:x1874579652415744:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:0.0:1787739053.324489:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17426:x1874579652415744:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 184us (451us total) trans 0 rc -17/-17 00000100:00100000:0.0:1787739053.324499:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23171 00000100:00100000:3.0:1787739053.336876:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739053.336953:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652415872 00000100:00100000:0.0:1787739053.336962:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652415872 00000100:00100000:0.0:1787739053.337024:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23172 00000100:00100000:0.0:1787739053.337035:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17427:x1874579652415872:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739053.337063:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739053.337096:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab762 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094aca expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.337162:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab769 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.337175:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab769 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.337185:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab769 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.337196:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab770 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.337206:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ab73f lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab738 expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.337213:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab770 lrc: 4/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.337246:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ab73f lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ab738 expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.337266:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d12c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665537280:0@lo:104: 00000100:00100000:0.0:1787739053.337311:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.337358:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.337391:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665537280 00000100:00100000:3.0:1787739053.337396:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665537280 00000100:00100000:3.0:1787739053.337444:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1052 00000100:00100000:3.0:1787739053.337451:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d0780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665537280:12345-0@lo:104: 00010000:00010000:3.0:1787739053.337464:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1ab738 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab73f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.337533:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739053.337539:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d0780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665537280:12345-0@lo:104: Request processed in 89us (227us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.337545:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1052 00010000:00010000:1.0:1787739053.337607:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739053.337614:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1ab738 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab73f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.337623:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739053.337627:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1ab738 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ab73f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739053.337636:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1ab738 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ab73f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.337662:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1ab738 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab73f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739053.337665:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d12c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665537280:0@lo:104: 00010000:00010000:1.0:1787739053.337669:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739053.337684:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab770 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.337697:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1ab738 lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab73f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.337705:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c200/0x7ee4453e4a1ab738 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab73f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.337721:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739053.337722:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739053.337724:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739053.337741:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787739053.337752:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157e92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665537408:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739053.337786:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.338116:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665537408 00000100:00100000:2.0:1787739053.338121:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665537408 00010000:00010000:2.0:1787739053.338143:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ab73f lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ab738 expref: 12 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739053.338173:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1378 00000100:00100000:2.0:1787739053.338181:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba2450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+12:3653:x1874579665537408:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739053.338190:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739053.343466:0:6502:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab6cf lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab6d6 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739053.343486:0:6502:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab6cf lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab6d6 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.343527:0:6502:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab6cf lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab6d6 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.343534:0:6502:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739053.343569:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1ab6cf lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab6d6 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787739053.343588:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983306 is committed 00000001:00080000:1.0:1787739053.343602:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983305 is committed 00000020:00080000:1.0:1787739053.343610:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 343 batchid updated 00000020:00080000:1.0:1787739053.343638:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa0730610e300 refcount 2 committed 0 result 0 batchid 344 00000020:00080000:1.0:1787739053.343642:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2600 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730610ee40 00000020:00080000:1.0:1787739053.343646:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].341 00000020:00080000:1.0:1787739053.343649:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2b00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073069be000 00010000:00010000:0.0:1787739053.343650:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab65f lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x14f5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:1.0:1787739053.343652:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].342 00010000:00010000:0.0:1787739053.343664:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab65f lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x14f5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739053.343669:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00010000:00010000:0.0:1787739053.343672:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab65f lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x14f5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.343679:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c400 already unused, calling callback (ffffffffc18da4f0) 00000020:00080000:1.0:1787739053.343681:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 344 committed batchid 343 00010000:00010000:0.0:1787739053.343683:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab65f lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x14f5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739053.343692:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157ebc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665537536:0@lo:103:jbd2/dm-0-8.0 00010000:00010000:0.0:1787739053.343692:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab65f lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x14f5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.343703:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f5:0x0].0x0 (ffffa07303c78400) 00010000:00010000:0.0:1787739053.343707:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739053.343711:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab65f lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x14f5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.343716:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab65f lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x14f5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.343735:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab666 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.343744:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab666 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.343749:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab666 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.343753:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739053.343756:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab666 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739053.343760:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787739053.343762:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab666 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.343771:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb6:0x0].0x0 (ffffa07303c78000) 00010000:00010000:0.0:1787739053.343774:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739053.343776:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab666 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.343781:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab666 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739053.343788:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07305fcc240 with version 653 00000020:00080000:1.0:1787739053.343801:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 343 cancel update log [0x240000401:0x3:0x0].340: rc = 0 00000100:00100000:2.0:1787739053.343828:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665537536 00000100:00100000:2.0:1787739053.343835:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665537536 00000020:00080000:1.0:1787739053.343837:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 343 cancel update log [0x200000400:0x3:0x0].340: rc = 0 00000020:00080000:1.0:1787739053.343848:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 343 00000004:00080000:1.0:1787739053.343856:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 653 rpc_version 653 00000020:00100000:1.0:1787739053.343877:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739053.343884:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743562de00 x1874579665537792/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739053.343900:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562de00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665537792:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739053.343931:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.343952:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fccc00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739053.343978:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665537792 00000100:00100000:1.0:1787739053.343982:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665537792 00000100:00100000:2.0:1787739053.344007:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1176 00000100:00100000:1.0:1787739053.344015:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2545 00000100:00100000:2.0:1787739053.344020:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+8:3652:x1874579665537536:12345-0@lo:103:jbd2/dm-0-8.0 00000100:00100000:1.0:1787739053.344027:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562f840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665537792:12345-0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1787739053.344037:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739053.344055:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab6d6 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x82d7929c bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab6cf expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.344067:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x82d7929c (ffffa0741590d400) 00010000:00010000:2.0:1787739053.344075:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739053.344122:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+7:3652:x1874579665537536:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 106us (362us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739053.344130:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1176 00000100:00100000:2.0:1787739053.344163:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074157ebc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665537536:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:1.0:1787739053.344284:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562f840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665537792:12345-0@lo:1000:osp_up1-0.0 Request processed in 261us (351us total) trans 4294968295 rc 0/0 00000100:00100000:1.0:1787739053.344292:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2545 00010000:00010000:1.0:1787739053.344349:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ab73f lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ab738 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.344366:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xaa220cb4 (ffffa0741590de00) 00010000:00010000:1.0:1787739053.344371:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab770 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.344380:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab770 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.344387:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab770 lrc: 4/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.344407:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739053.344454:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba2450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+6:3653:x1874579665537408:12345-0@lo:103:ldlm_bl.0 Request processed in 6274us (6667us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.344461:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1378 00000100:00100000:2.0:1787739053.344506:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562de00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665537792:0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1787739053.344526:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab770 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.344534:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab770 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739053.344618:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564e580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665537920:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739053.344647:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.344665:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739053.344679:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665537920 00000100:00100000:2.0:1787739053.344682:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665537920 00000100:00100000:2.0:1787739053.344717:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2546 00000100:00100000:2.0:1787739053.344725:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665537920:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739053.344791:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665537920:12345-0@lo:1000:mdt00_001.0 Request processed in 67us (143us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739053.344810:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2546 00000100:00100000:0.0:1787739053.344849:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564e580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665537920:0@lo:1000:mdt00_001.0 00010000:00010000:0.0:1787739053.344888:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab777 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x15d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.344899:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab777 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.344907:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab777 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739053.344932:0:6524:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab777 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.344945:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab769 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.344951:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab769 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.344956:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab769 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.344962:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab769 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.344968:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739053.344971:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab769 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.344978:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab769 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.344986:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0741590da00) 00010000:00010000:0.0:1787739053.344989:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739053.344992:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab769 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.344998:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab769 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.345007:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab770 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.345013:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab770 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.345019:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab770 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.345023:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609be00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739053.345026:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab770 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.345031:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab770 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.345038:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xaa220cb4 (ffffa0741590de00) 00010000:00010000:0.0:1787739053.345040:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739053.345042:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab770 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.345060:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab770 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.345105:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab762 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aca expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.345122:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab777 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x15d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aca expref: 14 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.345130:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15d:0x0].0x0 (ffffa0741590d200) 00010000:00010000:0.0:1787739053.345132:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739053.345135:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609bc00, rc 0) 00000100:00100000:3.0:1787739053.345152:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074157e92c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665537408:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739053.345177:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17427:x1874579652415872:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 8144us (8321us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739053.345184:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23172 00000100:00100000:3.0:1787739053.350020:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739053.351133:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652416000 00000100:00100000:2.0:1787739053.351145:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652416000 00000100:00100000:2.0:1787739053.351337:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23173 00000100:00100000:2.0:1787739053.351349:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17427:x1874579652416000:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739053.351373:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739053.351394:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab77e lrc: 2/0,0 mode: --/CR res: [0x240000402:0x15d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094ad1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.351482:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ab785 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x15d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.351497:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ab785 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.351508:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ab785 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739053.351600:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x15d:0x0]: returning size 4096 00000004:00200000:2.0:1787739053.351605:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739053.351642:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab77e lrc: 0/0,0 mode: --/CR res: [0x240000402:0x15d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ad1 expref: 11 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.351662:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ab785 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x15d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ad1 expref: 10 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.351671:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15d:0x0].0x0 (ffffa07303c78600) 00010000:00010000:2.0:1787739053.351674:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739053.351678:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d400, rc 0) 00000100:00100000:2.0:1787739053.351808:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17427:x1874579652416000:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 460us (1809us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739053.351816:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23173 00000100:00100000:0.0:1787739053.356110:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739053.356199:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652416128 00000100:00100000:1.0:1787739053.356210:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652416128 00000100:00100000:1.0:1787739053.356276:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23174 00000100:00100000:1.0:1787739053.356287:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17427:x1874579652416128:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739053.356306:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739053.356327:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab78c lrc: 2/0,0 mode: --/CR res: [0x240000400:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094ad8 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.356387:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab793 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.356401:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab793 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.356439:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab793 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739053.356490:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xb4:0x0]: returning size 4096 00000004:00200000:1.0:1787739053.356495:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739053.356539:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab78c lrc: 0/0,0 mode: --/CR res: [0x240000400:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ad8 expref: 12 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.356558:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab793 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ad8 expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.356566:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb4:0x0].0x0 (ffffa07303c78e00) 00010000:00010000:1.0:1787739053.356569:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739053.356573:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1da00, rc 0) 00000100:00100000:1.0:1787739053.356630:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17427:x1874579652416128:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 346us (529us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.356637:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23174 00000100:00100000:3.0:1787739053.361072:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739053.361178:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652416256 00000100:00100000:0.0:1787739053.361187:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652416256 00000100:00100000:0.0:1787739053.361240:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23175 00000100:00100000:0.0:1787739053.361251:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17427:x1874579652416256:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739053.361271:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739053.361293:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab79a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094adf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.361355:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab7a1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.361369:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab75b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab754 expref: 5 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.361378:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab7a1 lrc: 4/1,0 mode: --/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.361410:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab75b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ab754 expref: 5 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.361453:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d2580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665538048:0@lo:104: 00000100:00100000:2.0:1787739053.361536:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.361579:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.362653:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665538048 00000100:00100000:0.0:1787739053.362661:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665538048 00000100:00100000:0.0:1787739053.362715:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1053 00000100:00100000:0.0:1787739053.362724:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665538048:12345-0@lo:104: 00010000:00010000:0.0:1787739053.362953:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c000/0x7ee4453e4a1ab754 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab75b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.363041:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787739053.363055:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665538048:12345-0@lo:104: Request processed in 326us (1513us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739053.363063:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d2580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665538048:0@lo:104: 00000100:00100000:0.0:1787739053.363066:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1053 00010000:00010000:2.0:1787739053.363097:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab7a1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.363199:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739053.363207:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c000/0x7ee4453e4a1ab754 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab75b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.363217:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739053.363222:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c000/0x7ee4453e4a1ab754 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ab75b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739053.363230:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c000/0x7ee4453e4a1ab754 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ab75b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.363262:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c000/0x7ee4453e4a1ab754 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab75b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.363269:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739053.363303:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c000/0x7ee4453e4a1ab754 lrc: 1/0,0 mode: --/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab75b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.363311:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c000/0x7ee4453e4a1ab754 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab75b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.363331:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739053.363333:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739053.363334:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739053.363397:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787739053.363410:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665538176:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739053.363467:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.363511:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665538176 00000100:00100000:2.0:1787739053.363515:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665538176 00010000:00010000:2.0:1787739053.363531:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab75b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ab754 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739053.363554:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1379 00000100:00100000:2.0:1787739053.363559:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d21c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+6:3651:x1874579665538176:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739053.363567:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00080000:2.0:1787739053.368359:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 344 batchid updated 00000020:00080000:1.0:1787739053.368400:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 344 00000004:00080000:1.0:1787739053.368489:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07305fccc00 with version 654 00000020:00080000:1.0:1787739053.368510:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 344 cancel update log [0x240000401:0x3:0x0].341: rc = 0 00000004:00080000:3.0:1787739053.368861:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 654 rpc_version 654 00000020:00100000:3.0:1787739053.368894:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739053.368905:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073061fde00 x1874579665538432/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739053.368928:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fde00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665538432:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739053.368995:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.369035:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28840 going to sleep for 11 seconds 00010000:00010000:2.0:1787739053.369248:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab75b lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ab754 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.369266:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb4:0x0].0x0 (ffffa074331c2f00) 00010000:00010000:2.0:1787739053.369272:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab7a1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.369282:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab7a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.369293:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab7a1 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.369316:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739053.369357:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d21c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665538176:12345-0@lo:103:ldlm_bl.0 Request processed in 5797us (5889us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739053.369366:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1379 00010000:00010000:2.0:1787739053.369399:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab7a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.369407:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab7a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739053.369467:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb4:0x0]: returning size 4096 00000004:00200000:2.0:1787739053.369473:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739053.369515:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ab79a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094adf expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00080000:1.0:1787739053.369744:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 344 cancel update log [0x200000400:0x3:0x0].342: rc = 0 00000020:00080000:1.0:1787739053.369752:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 344 00010000:00010000:2.0:1787739053.371204:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab7a1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094adf expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.371217:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb4:0x0].0x0 (ffffa074331c2f00) 00010000:00010000:2.0:1787739053.371219:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739053.371222:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609be00, rc 0) 00000100:00100000:2.0:1787739053.371286:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17427:x1874579652416256:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 10038us (10231us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739053.371294:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23175 00000100:00100000:0.0:1787739053.371518:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665538176:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739053.371629:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665538432 00000100:00100000:0.0:1787739053.371637:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665538432 00000100:00100000:0.0:1787739053.371674:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2547 00000100:00100000:0.0:1787739053.371682:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061ffc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665538432:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739053.372297:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fde00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665538432:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739053.373012:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061ffc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665538432:12345-0@lo:1000:osp_up1-0.0 Request processed in 172us (2859us total) trans 4294968296 rc 0/0 00000100:00100000:0.0:1787739053.373022:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2547 00010000:00010000:3.0:1787739053.382361:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739053.382699:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739053.382745:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739053.382756:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652416384 00000100:00100000:1.0:1787739053.382767:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652416384 00000100:00100000:0.0:1787739053.382775:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652416640 00000100:00100000:0.0:1787739053.382781:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652416640 00000100:00100000:0.0:1787739053.382828:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23176 00000100:00100000:0.0:1787739053.382841:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17427:x1874579652416640:12345-192.168.203.15@tcp:36:rmdir.0 00000100:00100000:1.0:1787739053.382844:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1177 00000100:00100000:1.0:1787739053.382858:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d2580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2409:x1874579652416384:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:1.0:1787739053.382886:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739053.382899:0:6525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739053.382911:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ab777 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x15d:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aca expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.382921:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ab785 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x15d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ad1 expref: 11 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.382933:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15d:0x0].0x0 (ffffa0741590d200) 00010000:00010000:0.0:1787739053.382939:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15d:0x0].0x0 (ffffa07303c78600) 00010000:00010000:0.0:1787739053.382945:0:6525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739053.382947:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739053.382997:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d2580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652416384:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 142us (303us total) trans 0 rc 0/0 00010000:00010000:0.0:1787739053.382998:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ab7a8 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739053.383006:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1177 00010000:00010000:0.0:1787739053.383013:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ab7a8 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.383025:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ab7a8 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.383048:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432ed00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665538560:0@lo:101:mdt00_002.0 00000100:00100000:0.0:1787739053.383221:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.383257:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739053.383431:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665538560 00000100:00100000:1.0:1787739053.383435:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665538560 00000100:00100000:1.0:1787739053.383466:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2548 00000100:00100000:1.0:1787739053.383474:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+5:6525:x1874579665538560:12345-0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787739053.383486:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739053.383503:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab7af lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab7a8 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.383532:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab7af lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab7a8 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.383554:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab7af lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab7a8 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.383563:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae800) 00010000:00010000:1.0:1787739053.383566:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739053.383569:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c000, rc 0) 00000100:00100000:1.0:1787739053.383677:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:6525:x1874579665538560:12345-0@lo:101:mdt00_002.0 Request processed in 201us (453us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.383685:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2548 00000100:00100000:3.0:1787739053.383741:0:6525:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983306 gen 2 00000004:00080000:3.0:1787739053.383748:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983300 committed transno 4294983306 00000020:00080000:3.0:1787739053.383754:0:6525:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa0730610e9c0 refcount 2 committed 0 result 0 batchid 335 00000020:00080000:3.0:1787739053.383758:0:6525:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2e00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730610e3c0 00000020:00080000:3.0:1787739053.383762:0:6525:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].328 00000020:00080000:3.0:1787739053.383764:0:6525:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2300 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa074184eea00 00000020:00080000:3.0:1787739053.383767:0:6525:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].328 00000020:00080000:0.0:1787739053.383811:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 335 committed batchid 334 00000020:00080000:0.0:1787739053.384125:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 334 00000004:00080000:3.0:1787739053.384289:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983302 committed transno 4294983306 00000004:00080000:3.0:1787739053.384301:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983303 committed transno 4294983306 00000004:00080000:3.0:1787739053.384309:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983304 committed transno 4294983306 00000004:00080000:3.0:1787739053.384318:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983306 committed transno 4294983306 00000100:00100000:3.0:1787739053.384344:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432ed00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665538560:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739053.384361:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ab7a8 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab7af expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.384373:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ab7a8 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab7af expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739053.384390:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739053.384618:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa400/0x7ee4453e4a1ab7b6 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.384627:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa400/0x7ee4453e4a1ab7b6 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.384635:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa400/0x7ee4453e4a1ab7b6 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739053.384701:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fe1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665538688:0@lo:101:mdt00_002.0 00000100:00100000:2.0:1787739053.384744:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.384772:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef300 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.384822:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665538688 00000100:00100000:3.0:1787739053.384828:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665538688 00000100:00100000:3.0:1787739053.384877:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2549 00000100:00100000:3.0:1787739053.384885:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:6525:x1874579665538688:12345-0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739053.384896:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739053.384910:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab7bd lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab7b6 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.384928:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab7bd lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab7b6 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.384940:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab7bd lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab7b6 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.384947:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xaa220cb4 (ffffa074331c2100) 00010000:00010000:3.0:1787739053.384950:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739053.384953:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aaa00, rc 0) 00000100:00100000:3.0:1787739053.384997:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6525:x1874579665538688:12345-0@lo:101:mdt00_002.0 Request processed in 113us (253us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.385003:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2549 00000100:00100000:3.0:1787739053.385031:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fe1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665538688:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739053.385042:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa400/0x7ee4453e4a1ab7b6 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab7bd expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.385050:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa400/0x7ee4453e4a1ab7b6 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab7bd expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739053.385063:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:3.0:1787739053.385105:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fe1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665538816:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739053.385131:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.385149:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739053.385195:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665538816 00000100:00100000:2.0:1787739053.385200:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665538816 00000100:00100000:2.0:1787739053.385233:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2550 00000100:00100000:2.0:1787739053.385240:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fd680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6525:x1874579665538816:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739053.385297:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fd680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6525:x1874579665538816:12345-0@lo:1000:mdt00_002.0 Request processed in 57us (165us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739053.385303:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2550 00000100:00100000:3.0:1787739053.385511:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fe1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665538816:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739053.385547:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fe1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665538944:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739053.385571:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.385595:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739053.385637:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665538944 00000100:00100000:2.0:1787739053.385641:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665538944 00000100:00100000:2.0:1787739053.385669:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2551 00000100:00100000:2.0:1787739053.385676:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fe940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6525:x1874579665538944:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739053.385742:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fe940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6525:x1874579665538944:12345-0@lo:1000:mdt00_002.0 Request processed in 66us (169us total) trans 0 rc 1/1 00000100:00100000:2.0:1787739053.385748:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2551 00000100:00100000:3.0:1787739053.385809:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fe1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665538944:0@lo:1000:mdt00_002.0 00010000:00010000:3.0:1787739053.385847:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ab7c4 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.385854:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ab7c4 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.385861:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ab7c4 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739053.385873:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fe1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665539072:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1787739053.385900:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.385926:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.385939:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665539072 00000100:00100000:3.0:1787739053.385942:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665539072 00000100:00100000:3.0:1787739053.385965:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2552 00000100:00100000:3.0:1787739053.385971:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6525:x1874579665539072:12345-0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739053.385979:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739053.385989:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab7cb lrc: 2/0,0 mode: --/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab7c4 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.386005:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab7cb lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab7c4 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.386017:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab7cb lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab7c4 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.386025:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15d:0x0].0x0 (ffffa074331c2200) 00010000:00010000:3.0:1787739053.386028:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739053.386030:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa800, rc 0) 00000100:00100000:3.0:1787739053.386061:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6525:x1874579665539072:12345-0@lo:101:mdt00_002.0 Request processed in 90us (164us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.386066:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2552 00000100:00100000:3.0:1787739053.386105:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fe1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665539072:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739053.386115:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ab7c4 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab7cb expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.386123:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ab7c4 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab7cb expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.386144:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab7d2 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.386155:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab7d2 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.386162:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab7d2 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.386177:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab7d9 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.386188:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab793 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ad8 expref: 10 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.386197:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab7d9 lrc: 4/0,1 mode: --/EX res: [0x240000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.386217:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab793 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ad8 expref: 10 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.386230:0:6525:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab793 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ad8 expref: 10 pid: 6523 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739053.386241:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fe1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665539200:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739053.386297:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.397672:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fe1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665539200:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739053.397704:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab7d9 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739053.404650:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739053.404835:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652416768 00000100:00100000:0.0:1787739053.404844:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652416768 00010000:00010000:0.0:1787739053.404889:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab793 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ad8 expref: 11 pid: 6523 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.404933:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1380 00000100:00100000:0.0:1787739053.404944:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2406:x1874579652416768:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739053.404959:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739053.404967:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab793 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ad8 expref: 11 pid: 6523 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.404986:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab793 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ad8 expref: 11 pid: 6523 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.405007:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ab793 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ad8 expref: 11 pid: 6523 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.405022:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb4:0x0].0x0 (ffffa07303c78e00) 00010000:00010000:0.0:1787739053.405027:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab7d9 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.405037:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab7d9 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.405045:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab7d9 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.405071:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739053.405138:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2406:x1874579652416768:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 196us (497us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739053.405147:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1380 00010000:00010000:3.0:1787739053.405885:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab7d9 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.405897:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab7d9 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.405945:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ab7e0 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.405953:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ab7e0 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.405962:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ab7e0 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739053.405982:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fe1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665539328:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1787739053.406035:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.406079:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.406166:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665539328 00000100:00100000:0.0:1787739053.406172:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665539328 00000100:00100000:0.0:1787739053.406213:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2553 00000100:00100000:0.0:1787739053.406221:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304000b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6525:x1874579665539328:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787739053.406237:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739053.406252:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab7e7 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab7e0 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.406281:0:8448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab7a1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094adf expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.406290:0:8448:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab7e7 lrc: 4/0,0 mode: --/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab7e0 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.406314:0:8448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab7a1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094adf expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.406327:0:8448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab7a1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094adf expref: 13 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.406338:0:8448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41e580 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8448:1874579665539456:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739053.406392:0:8448:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074104049c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739053.430983:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739053.431034:0:8448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41e580 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8448:1874579665539456:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739053.431062:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab7e7 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab7e0 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.431072:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb4:0x0].0x0 (ffffa074331c2f00) 00010000:00010000:1.0:1787739053.431075:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739053.431078:0:8448:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab7e7 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab7e0 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.431099:0:8448:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab7e7 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab7e0 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.431107:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609ba00, rc 0) 00000100:00100000:1.0:1787739053.431165:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304000b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6525:x1874579665539328:12345-0@lo:101:mdt00_002.0 Request processed in 24944us (25129us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.431177:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2553 00000100:00100000:1.0:1787739053.431217:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652416896 00000100:00100000:1.0:1787739053.431223:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652416896 00010000:00010000:1.0:1787739053.431251:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab7a1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094adf expref: 14 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739053.431276:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1381 00000100:00100000:1.0:1787739053.431285:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2408:x1874579652416896:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739053.431299:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739053.431305:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab7a1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094adf expref: 14 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.431316:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab7a1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094adf expref: 14 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.431335:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ab7a1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094adf expref: 14 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.431556:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb4:0x0].0x0 (ffffa074331c2f00) 00010000:00010000:1.0:1787739053.431561:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab7e7 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab7e0 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.431570:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab7e7 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab7e0 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.431581:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab7e7 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab7e0 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.431615:0:6508:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab7e7 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab7e0 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739053.431629:0:6508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561c3c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665539584:0@lo:105: 00000100:00100000:1.0:1787739053.431664:0:6508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.432206:0:6508:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739053.432253:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fe1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665539328:0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787739053.432269:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ab7e0 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab7e7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.432278:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ab7e0 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab7e7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.432646:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665539584 00000100:00100000:0.0:1787739053.432654:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665539584 00000100:00100000:0.0:1787739053.432691:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1054 00000100:00100000:0.0:1787739053.432701:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d0f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665539584:12345-0@lo:105: 00010000:00010000:0.0:1787739053.432712:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ab7e0 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab7e7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.432723:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ab7e0 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab7e7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.432732:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ab7e0 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab7e7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.432745:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ab7e0 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab7e7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.432752:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ab7e0 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab7e7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.432760:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ab7e0 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ab7e7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.432778:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069abe00) 00000100:00100000:0.0:1787739053.432837:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d0f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665539584:12345-0@lo:105: Request processed in 139us (1172us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739053.432844:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1054 00000100:00100000:0.0:1787739053.432877:0:6508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561c3c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665539584:0@lo:105: 00010000:00010000:0.0:1787739053.432890:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739053.432931:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:2408:x1874579652416896:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 1646us (1956us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739053.432939:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1381 00010000:00010000:0.0:1787739053.432963:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ab7e0 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab7e7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.432972:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ab7e0 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab7e7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.435519:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665539712:0@lo:403:mdt00_002.0 00000100:00100000:0.0:1787739053.437649:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.437694:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404300 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.437925:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665539712 00000100:00100000:3.0:1787739053.437944:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665539712 00000100:00100000:3.0:1787739053.438001:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2554 00000100:00100000:3.0:1787739053.438011:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6525:x1874579665539712:12345-0@lo:403:mdt00_002.0 00000100:00100000:3.0:1787739053.441051:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6525:x1874579665539712:12345-0@lo:403:mdt00_002.0 Request processed in 3033us (3400us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.441064:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2554 00000100:00100000:1.0:1787739053.441113:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665539712:0@lo:403:mdt00_002.0 00000040:00080000:1.0:1787739053.442098:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].330, 664 off503136 00000040:00080000:1.0:1787739053.442270:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].330, 744 off549472 00000004:00080000:1.0:1787739053.442277:0:6525:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07305fcc540 with version 629 00000004:00080000:2.0:1787739053.443108:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 629 rpc_version 629 00000020:00100000:2.0:1787739053.443147:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739053.443156:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074157ebc00 x1874579665539968/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1808/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739053.443180:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157ebc00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665539968:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739053.443245:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.443296:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.444266:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665539968 00000100:00100000:3.0:1787739053.444276:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665539968 00000100:00100000:3.0:1787739053.444357:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2555 00000100:00100000:3.0:1787739053.444367:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7bc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665539968:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739053.445228:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7bc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665539968:12345-0@lo:1000:osp_up0-1.0 Request processed in 860us (1980us total) trans 4294983307 rc 0/0 00000100:00100000:3.0:1787739053.445242:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2555 00000100:00100000:2.0:1787739053.454005:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074157ebc00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665539968:0@lo:1000:osp_up0-1.0 00000004:00200000:1.0:1787739053.454155:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x15d:0x0]: returning size 4096 00010000:00010000:1.0:1787739053.454181:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab7d9 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454192:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab7d9 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454200:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab7d9 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454212:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aae00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739053.454216:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab7d9 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454225:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab7d9 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454240:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb4:0x0].0x0 (ffffa07303c78e00) 00010000:00010000:1.0:1787739053.454243:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739053.454249:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab7d9 lrc: 2/0,0 mode: --/EX res: [0x240000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454256:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ab7d9 lrc: 0/0,0 mode: --/EX res: [0x240000400:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454272:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ab7e0 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab7e7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454292:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ab7e0 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab7e7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454299:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ab7e0 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab7e7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454334:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739053.454343:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xb4:0x0] 00010000:00010000:1.0:1787739053.454353:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739053.454359:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ab7e0 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab7e7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454371:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739053.454375:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ab7e0 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab7e7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739053.454383:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ab7e0 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab7e7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454433:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ab7e0 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab7e7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454440:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739053.454464:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ab7e0 lrc: 1/0,0 mode: --/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab7e7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454472:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ab7e0 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab7e7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454487:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739053.454489:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739053.454490:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739053.454503:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665540096:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739053.454563:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.454617:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665540096 00000100:00100000:1.0:1787739053.454624:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665540096 00000100:00100000:1.0:1787739053.454668:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1178 00000100:00100000:1.0:1787739053.454677:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+10:3651:x1874579665540096:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739053.454689:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739053.454706:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ab7e7 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab7e0 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454718:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb4:0x0].0x0 (ffffa074331c2f00) 00010000:00010000:1.0:1787739053.454724:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739053.454755:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+9:3651:x1874579665540096:12345-0@lo:103:ldlm_bl.0 Request processed in 79us (192us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.454762:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1178 00000100:00100000:1.0:1787739053.454796:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665540096:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739053.454819:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab7d2 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454827:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab7d2 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454833:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab7d2 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454839:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739053.454842:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab7d2 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454848:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab7d2 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454858:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15d:0x0].0x0 (ffffa07303c78600) 00010000:00010000:1.0:1787739053.454860:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739053.454863:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab7d2 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454869:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab7d2 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454880:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ab7c4 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab7cb expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454887:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ab7c4 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab7cb expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454895:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ab7c4 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab7cb expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454908:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739053.454920:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739053.454926:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739053.454931:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ab7c4 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab7cb expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454937:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739053.454941:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ab7c4 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab7cb expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739053.454948:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ab7c4 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab7cb expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454965:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ab7c4 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab7cb expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.454972:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739053.454988:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ab7c4 lrc: 1/0,0 mode: --/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab7cb expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.455000:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ab7c4 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab7cb expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.455012:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739053.455014:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739053.455016:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739053.455025:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665540224:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739053.455059:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.455100:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665540224 00000100:00100000:1.0:1787739053.455105:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665540224 00000100:00100000:1.0:1787739053.455130:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1179 00000100:00100000:1.0:1787739053.455137:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba3c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3652:x1874579665540224:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739053.455145:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739053.455156:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab7cb lrc: 0/0,0 mode: --/EX res: [0x240000402:0x15d:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab7c4 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.455166:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15d:0x0].0x0 (ffffa074331c2200) 00010000:00010000:1.0:1787739053.455173:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739053.455201:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba3c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+8:3652:x1874579665540224:12345-0@lo:103:ldlm_bl.0 Request processed in 65us (141us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.455208:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1179 00000100:00100000:1.0:1787739053.455257:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665540224:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739053.455273:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ab7a8 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab7af expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.455282:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ab7a8 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab7af expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.455289:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ab7a8 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab7af expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.455302:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739053.455306:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739053.455312:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739053.455316:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ab7a8 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab7af expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.455322:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739053.455326:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ab7a8 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab7af expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739053.455333:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ab7a8 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab7af expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.455349:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ab7a8 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab7af expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.455355:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739053.455366:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ab7a8 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab7af expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.455373:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ab7a8 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab7af expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.455386:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739053.455388:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739053.455390:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739053.455396:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787739053.455403:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665540352:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739053.455444:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.455468:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665540352 00000100:00100000:1.0:1787739053.455472:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665540352 00000100:00100000:1.0:1787739053.455494:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1180 00000100:00100000:1.0:1787739053.455501:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3652:x1874579665540352:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739053.455508:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739053.455519:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab7af lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab7a8 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.455528:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae800) 00010000:00010000:1.0:1787739053.455534:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739053.455561:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1874579665540352:12345-0@lo:103:ldlm_bl.0 Request processed in 60us (116us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.455567:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1180 00000100:00100000:1.0:1787739053.455593:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665540352:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739053.455611:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa400/0x7ee4453e4a1ab7b6 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab7bd expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.455619:0:6525:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa400/0x7ee4453e4a1ab7b6 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab7bd expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739053.455684:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17427:x1874579652416640:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 72845us (72938us total) trans 4294968297 rc 0/0 00000100:00100000:1.0:1787739053.455691:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23176 00000100:00100000:1.0:1787739053.481101:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739053.481186:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652417024 00000100:00100000:3.0:1787739053.481196:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652417024 00000100:00100000:3.0:1787739053.481265:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23177 00000100:00100000:3.0:1787739053.481278:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17428:x1874579652417024:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739053.481303:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739053.481327:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ab7ee lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094ae6 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.481394:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ab7f5 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.481409:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ab7f5 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.481442:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ab7f5 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.481460:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab7fc lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.481468:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab7fc lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.481476:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab7fc lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.481520:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ab7f5 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.481532:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ab7f5 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.481538:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ab7f5 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.481544:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ab7f5 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.481551:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739053.481555:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ab7f5 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.481563:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ab7f5 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.481576:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07433235600) 00010000:00010000:3.0:1787739053.481580:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739053.481585:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ab7f5 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.481591:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ab7f5 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.481607:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab7fc lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.481614:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab7fc lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.481620:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab7fc lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.481627:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739053.481630:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab7fc lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.481637:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab7fc lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.481647:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x2153044 (ffffa074331c2b00) 00010000:00010000:3.0:1787739053.481650:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739053.481653:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab7fc lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.481659:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab7fc lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.481698:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ab7ee lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ae6 expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.481711:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07433235600) 00010000:00010000:3.0:1787739053.481714:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739053.481718:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ab7ee lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ae6 expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.481742:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa200, rc 0) 00000100:00100000:3.0:1787739053.482060:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17428:x1874579652417024:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 782us (974us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739053.482091:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23177 00000100:00100000:3.0:1787739053.498877:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739053.504607:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652417152 00000100:00100000:2.0:1787739053.504621:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652417152 00000100:00100000:3.0:1787739053.505867:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23178 00000100:00100000:3.0:1787739053.505881:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17428:x1874579652417152:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739053.505902:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739053.505926:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ab803 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094aed expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.505984:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab80a lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.505999:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab80a lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.506010:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab80a lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739053.506056:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:3.0:1787739053.506062:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739053.506108:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ab803 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aed expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.506125:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab80a lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aed expref: 14 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.506133:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2b00) 00010000:00010000:3.0:1787739053.506136:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739053.506139:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa600, rc 0) 00000100:00100000:3.0:1787739053.506206:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17428:x1874579652417152:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 327us (7340us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.506214:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23178 00000100:00100000:1.0:1787739053.508970:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787739053.508990:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730564e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665540480:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787739053.509076:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.509149:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665540480 00000100:00100000:1.0:1787739053.509156:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665540480 00000100:00100000:1.0:1787739053.509196:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 350 00000100:00100000:1.0:1787739053.509206:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d21c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665540480:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787739053.509237:0:8441:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8646080 pending 0 free 3923529728 avail 3692007424 00000100:00100000:1.0:1787739053.509283:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d21c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665540480:12345-0@lo:13:osp-pre-0-0.0 Request processed in 80us (207us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.509290:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 350 00000100:00100000:1.0:1787739053.509333:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730564e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665540480:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1787739053.509372:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787739053.512790:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787739053.512812:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665540608:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1787739053.512881:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.512932:0:10361:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665540608 00000100:00100000:2.0:1787739053.512939:0:10361:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665540608 00000100:00100000:2.0:1787739053.512994:0:10361:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 351 00000100:00100000:2.0:1787739053.513003:0:10361:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1874579665540608:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1787739053.513025:0:10361:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8646080 pending 0 free 3923529728 avail 3692007424 00000100:00100000:2.0:1787739053.513075:0:10361:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1874579665540608:12345-0@lo:13:osp-pre-0-1.0 Request processed in 73us (193us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739053.513096:0:10361:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 351 00000100:00080000:2.0:1787739053.513118:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787739053.513154:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665540608:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1787739053.522767:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739053.522849:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652417408 00000100:00100000:3.0:1787739053.522869:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652417408 00000100:00100000:3.0:1787739053.522928:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23179 00000100:00100000:3.0:1787739053.522940:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17428:x1874579652417408:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:3.0:1787739053.523045:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c0780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665540736:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739053.523117:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.523174:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.523224:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665540736 00000100:00100000:0.0:1787739053.523229:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665540736 00000100:00100000:0.0:1787739053.523270:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2556 00000100:00100000:0.0:1787739053.523278:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6525:x1874579665540736:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739053.523381:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6525:x1874579665540736:12345-0@lo:1000:mdt00_002.0 Request processed in 104us (263us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739053.523389:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2556 00000100:00100000:2.0:1787739053.523966:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c0780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665540736:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739053.524026:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7a580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665540864:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739053.524064:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.524099:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.524324:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665540864 00000100:00100000:0.0:1787739053.524330:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665540864 00000100:00100000:0.0:1787739053.524373:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2557 00000100:00100000:0.0:1787739053.524380:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c0780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6525:x1874579665540864:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739053.524489:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c0780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6525:x1874579665540864:12345-0@lo:1000:mdt00_002.0 Request processed in 109us (423us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739053.524496:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2557 00000100:00100000:2.0:1787739053.526054:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7a580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665540864:0@lo:1000:mdt00_002.0 00010000:00010000:2.0:1787739053.526135:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d000/0x7ee4453e4a1ab811 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.526146:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d000/0x7ee4453e4a1ab811 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.526157:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d000/0x7ee4453e4a1ab811 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739053.526177:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7a580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665540992:0@lo:101:mdt00_002.0 00000100:00100000:2.0:1787739053.526223:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.526266:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.526786:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665540992 00000100:00100000:3.0:1787739053.526794:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665540992 00000100:00100000:3.0:1787739053.527934:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2558 00000100:00100000:3.0:1787739053.527949:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6525:x1874579665540992:12345-0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739053.527967:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739053.527986:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ab818 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab811 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.528014:0:8447:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab723 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ac3 expref: 9 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.528024:0:8447:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ab818 lrc: 4/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab811 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.528052:0:8447:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab723 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ac3 expref: 9 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.528067:0:8447:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab723 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ac3 expref: 9 pid: 6525 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739053.528079:0:8447:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a31e00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8447:1874579665541120:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739053.528168:0:8447:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.531747:0:8447:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a31e00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8447:1874579665541120:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739053.531783:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ab818 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab811 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.531797:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07433235600) 00010000:00010000:0.0:1787739053.531800:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739053.531804:0:8447:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ab818 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab811 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.531813:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab800, rc 0) 00000100:00100000:0.0:1787739053.532072:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6525:x1874579665540992:12345-0@lo:101:mdt00_002.0 Request processed in 4125us (5846us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739053.532090:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2558 00000100:00100000:3.0:1787739053.532187:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7a580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665540992:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739053.532204:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d000/0x7ee4453e4a1ab811 lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab818 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.532215:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d000/0x7ee4453e4a1ab811 lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab818 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.533948:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739053.534010:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652417536 00000100:00100000:1.0:1787739053.534018:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652417536 00010000:00010000:1.0:1787739053.534053:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab723 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ac3 expref: 10 pid: 6525 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739053.534101:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1382 00000100:00100000:1.0:1787739053.534111:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41e580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652417536:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739053.534124:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739053.534130:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab723 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ac3 expref: 10 pid: 6525 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.534141:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab723 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ac3 expref: 10 pid: 6525 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.534168:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab723 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ac3 expref: 10 pid: 6525 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.534182:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07433235600) 00010000:00010000:1.0:1787739053.534188:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ab818 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab811 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.534198:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ab818 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab811 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.534207:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ab818 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab811 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.534238:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ab818 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab811 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739053.534251:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d2580 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665541248:0@lo:105: 00000100:00100000:1.0:1787739053.534296:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.534337:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07406576480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.534459:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665541248 00000100:00100000:3.0:1787739053.534465:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665541248 00000100:00100000:3.0:1787739053.534499:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1055 00000100:00100000:3.0:1787739053.534508:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665541248:12345-0@lo:105: 00010000:00010000:3.0:1787739053.534519:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d000/0x7ee4453e4a1ab811 lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab818 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.534530:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d000/0x7ee4453e4a1ab811 lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab818 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.534540:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d000/0x7ee4453e4a1ab811 lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab818 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.534548:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d000/0x7ee4453e4a1ab811 lrc: 5/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab818 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.534555:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d000/0x7ee4453e4a1ab811 lrc: 6/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab818 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.534562:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d000/0x7ee4453e4a1ab811 lrc: 6/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ab818 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.534585:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633d000) 00000100:00100000:3.0:1787739053.534637:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665541248:12345-0@lo:105: Request processed in 130us (340us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.534644:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1055 00000100:00100000:3.0:1787739053.534679:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d2580 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665541248:0@lo:105: 00010000:00010000:3.0:1787739053.534694:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739053.534733:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41e580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:2408:x1874579652417536:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 622us (791us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.534740:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1382 00010000:00010000:1.0:1787739053.535663:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d000/0x7ee4453e4a1ab811 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab818 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.535685:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d000/0x7ee4453e4a1ab811 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab818 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739053.535710:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739053.535818:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ab81f lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.535827:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ab81f lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.535838:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ab81f lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739053.536000:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7a580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665541376:0@lo:101:mdt00_002.0 00000100:00100000:1.0:1787739053.536058:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.536101:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07406576480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.536378:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665541376 00000100:00100000:3.0:1787739053.536385:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665541376 00000100:00100000:3.0:1787739053.536444:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2559 00000100:00100000:3.0:1787739053.536453:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6525:x1874579665541376:12345-0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739053.536470:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739053.536486:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab826 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab81f expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.536508:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab826 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab81f expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.536521:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab826 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab81f expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.536529:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x2153044 (ffffa074331c2900) 00010000:00010000:3.0:1787739053.536532:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739053.536536:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739053.537046:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665541376:12345-0@lo:101:mdt00_002.0 Request processed in 593us (986us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.537055:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2559 00000100:00100000:1.0:1787739053.545209:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7a580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665541376:0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787739053.545238:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ab81f lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab826 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.545253:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ab81f lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab826 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739053.545278:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:1.0:1787739053.545332:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562c000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665541504:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739053.545393:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.545518:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739053.546055:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665541504 00000100:00100000:2.0:1787739053.546064:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665541504 00000100:00100000:2.0:1787739053.546133:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2560 00000100:00100000:2.0:1787739053.546143:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665541504:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739053.546235:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665541504:12345-0@lo:1000:mdt00_002.0 Request processed in 93us (842us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739053.546242:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2560 00000100:00100000:2.0:1787739053.546277:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562c000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665541504:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739053.546325:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665541632:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739053.546355:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.546374:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739053.546387:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665541632 00000100:00100000:2.0:1787739053.546390:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665541632 00000100:00100000:2.0:1787739053.546433:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2561 00000100:00100000:2.0:1787739053.546440:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665541632:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739053.546507:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665541632:12345-0@lo:1000:mdt00_002.0 Request processed in 67us (150us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739053.546514:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2561 00000100:00100000:2.0:1787739053.546553:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665541632:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739053.546633:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665541760:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739053.546658:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.546674:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739053.546699:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665541760 00000100:00100000:2.0:1787739053.546702:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665541760 00000100:00100000:2.0:1787739053.547311:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2562 00000100:00100000:2.0:1787739053.547323:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665541760:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739053.547396:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665541760:12345-0@lo:1000:mdt00_002.0 Request processed in 74us (735us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739053.547667:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665541760:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739053.547814:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665541888:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739053.547955:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.547978:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739053.548199:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665541888 00000100:00100000:1.0:1787739053.548206:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665541888 00000100:00100000:1.0:1787739053.548234:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2563 00000100:00100000:1.0:1787739053.548241:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fe1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:6525:x1874579665541888:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739053.548312:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fe1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:6525:x1874579665541888:12345-0@lo:1000:mdt00_002.0 Request processed in 71us (453us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.548319:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2563 00000100:00100000:0.0:1787739053.548386:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665541888:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739053.548466:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665542016:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739053.548498:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.548516:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739053.549094:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665542016 00000100:00100000:1.0:1787739053.549100:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665542016 00000100:00100000:1.0:1787739053.549125:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2564 00000100:00100000:1.0:1787739053.549132:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:6525:x1874579665542016:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739053.549213:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:6525:x1874579665542016:12345-0@lo:1000:mdt00_002.0 Request processed in 82us (716us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.549220:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2564 00000100:00100000:0.0:1787739053.549283:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665542016:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739053.549324:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665542144:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739053.549349:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.549398:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739053.549470:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2562 00000100:00100000:2.0:1787739053.549488:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665542144 00000100:00100000:2.0:1787739053.549492:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665542144 00000100:00100000:1.0:1787739053.549558:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2565 00000100:00100000:1.0:1787739053.549566:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305656940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665542144:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739053.549671:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305656940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665542144:12345-0@lo:1000:mdt00_002.0 Request processed in 105us (320us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.549678:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2565 00000100:00100000:0.0:1787739053.550069:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665542144:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739053.550118:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665542272:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739053.550145:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.550204:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739053.550242:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665542272 00000100:00100000:2.0:1787739053.550246:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665542272 00000100:00100000:2.0:1787739053.550268:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2566 00000100:00100000:2.0:1787739053.550275:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665542272:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739053.550348:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665542272:12345-0@lo:1000:mdt00_002.0 Request processed in 73us (201us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739053.550354:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2566 00000100:00100000:0.0:1787739053.550454:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665542272:0@lo:1000:mdt00_002.0 00000040:00080000:3.0:1787739053.552240:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].344, 2168 off529288 00000040:00080000:3.0:1787739053.552333:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].343, 2368 off572160 00000004:00080000:3.0:1787739053.552340:0:6525:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa074104046c0 with version 655 00000004:00080000:2.0:1787739053.553175:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 655 rpc_version 655 00000020:00100000:2.0:1787739053.553222:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739053.553232:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306be5a40 x1874579665542528/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739053.553255:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665542528:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739053.553328:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.553362:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.553463:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665542528 00000100:00100000:3.0:1787739053.553470:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665542528 00000100:00100000:3.0:1787739053.553954:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2567 00000100:00100000:3.0:1787739053.553974:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be43c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665542528:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739053.554642:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be43c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:7740:x1874579665542528:12345-0@lo:1000:osp_up1-0.0 Request processed in 670us (1311us total) trans 4294968298 rc 0/0 00000100:00100000:3.0:1787739053.554651:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2567 00000100:00100000:2.0:1787739053.563463:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665542528:0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1787739053.566995:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab82d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x14f7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.567020:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab82d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.567031:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab82d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.567050:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab834 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.567057:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab834 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.567063:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab834 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.567107:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab83b lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.567115:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab83b lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.567124:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab83b lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739053.567147:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665542656:0@lo:101:mdt00_002.0 00000100:00100000:2.0:1787739053.567211:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.567255:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.567463:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665542656 00000100:00100000:0.0:1787739053.567471:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665542656 00000100:00100000:0.0:1787739053.567517:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2568 00000100:00100000:0.0:1787739053.567536:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:6525:x1874579665542656:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787739053.567681:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739053.567700:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab842 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab83b expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.567722:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab842 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab83b expref: 12 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.567811:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab842 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab83b expref: 12 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.567945:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb7:0x0].0x0 (ffffa073068a9800) 00010000:00010000:0.0:1787739053.567948:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739053.567952:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b600, rc 0) 00000100:00100000:0.0:1787739053.568008:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:6525:x1874579665542656:12345-0@lo:101:mdt00_002.0 Request processed in 473us (796us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739053.568015:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2568 00000100:00100000:0.0:1787739053.568062:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665542656:0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787739053.568079:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab83b lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab842 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.568100:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab83b lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab842 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739053.568125:0:6525:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b021ee85 reply state 0000000038487c7b transno 4294983308 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab834 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.568137:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab834 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.568148:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb7:0x0].0x0 (ffffa07303c78500) 00010000:00010000:0.0:1787739053.568150:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739053.568153:0:6525:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739053.568158:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab83b lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab842 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.568167:0:6525:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab83b lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab842 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739053.568177:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xb7:0x0] 00000004:00010000:0.0:1787739053.568185:0:6525:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b021ee85 reply state 0000000038487c7b transno 4294983308 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab82d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.568194:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab82d lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x14f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.568200:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f7:0x0].0x0 (ffffa07303c78800) 00010000:00010000:0.0:1787739053.568202:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739053.568204:0:6525:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787739053.568209:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14f7:0x0]: returning size 4096 00000004:00080000:0.0:1787739053.568218:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739053.568226:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d000/0x7ee4453e4a1ab811 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab818 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.568234:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d000/0x7ee4453e4a1ab811 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab818 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.568241:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d000/0x7ee4453e4a1ab811 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab818 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.568272:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739053.568278:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739053.568282:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ab81f lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab826 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.568289:0:6525:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ab81f lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab826 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.568365:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17428:x1874579652417408:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 45426us (45605us total) trans 4294983308 rc 0/0 00000100:00100000:0.0:1787739053.568372:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23179 00010000:00010000:3.0:1787739053.568898:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739053.568906:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d000/0x7ee4453e4a1ab811 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab818 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.568916:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739053.568921:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d000/0x7ee4453e4a1ab811 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab818 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739053.568930:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d000/0x7ee4453e4a1ab811 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab818 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.568958:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d000/0x7ee4453e4a1ab811 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab818 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.568965:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739053.568993:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d000/0x7ee4453e4a1ab811 lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab818 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.569001:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d000/0x7ee4453e4a1ab811 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab818 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.569028:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739053.569030:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739053.569032:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739053.569270:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665542784:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739053.569321:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.569720:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665542784 00000100:00100000:2.0:1787739053.569727:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665542784 00000100:00100000:2.0:1787739053.569786:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1181 00000100:00100000:2.0:1787739053.569796:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f7840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+12:3652:x1874579665542784:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739053.569809:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739053.569826:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ab818 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab811 expref: 12 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.569840:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07433235600) 00010000:00010000:2.0:1787739053.569848:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739053.569884:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f7840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+11:3652:x1874579665542784:12345-0@lo:103:ldlm_bl.0 Request processed in 90us (564us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739053.569891:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1181 00000100:00100000:2.0:1787739053.569922:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665542784:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739053.607530:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739053.607722:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652417792 00000100:00100000:3.0:1787739053.607731:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652417792 00000100:00100000:3.0:1787739053.607787:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23180 00000100:00100000:3.0:1787739053.607796:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f6d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17429:x1874579652417792:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:3.0:1787739053.607939:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f6d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17429:x1874579652417792:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 145us (418us total) trans 0 rc -17/-17 00000100:00100000:3.0:1787739053.607947:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23180 00000100:00100000:0.0:1787739053.635016:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739053.635177:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652417920 00000100:00100000:1.0:1787739053.635186:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652417920 00000100:00100000:1.0:1787739053.635244:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23181 00000100:00100000:1.0:1787739053.635255:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17430:x1874579652417920:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739053.635634:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739053.635659:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab849 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094af4 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.635774:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab850 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.635788:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab850 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.635799:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab850 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.635813:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab857 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.635822:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab826 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab81f expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.635830:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab857 lrc: 4/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.635862:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab826 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ab81f expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739053.635875:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f72940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665542912:0@lo:104: 00000100:00100000:1.0:1787739053.635928:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.635980:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.636036:0:15907:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787739053.636048:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665542912 00000100:00100000:0.0:1787739053.636052:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665542912 00000100:00100000:0.0:1787739053.636166:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1056 00000100:00100000:0.0:1787739053.636175:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073041e1c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665542912:12345-0@lo:104: 00010000:00010000:0.0:1787739053.636187:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ab81f lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab826 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.636233:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787739053.636239:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073041e1c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665542912:12345-0@lo:104: Request processed in 66us (310us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739053.636248:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1056 00000100:00100000:1.0:1787739053.637016:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f72940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665542912:0@lo:104: 00010000:00010000:1.0:1787739053.637035:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab857 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.637051:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739053.637056:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ab81f lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab826 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.637064:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739053.637070:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ab81f lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ab826 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739053.637150:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ab81f lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ab826 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.637182:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ab81f lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab826 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.637188:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739053.637215:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ab81f lrc: 1/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab826 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.637222:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ab81f lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab826 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.637239:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739053.637240:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739053.637242:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739053.637249:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787739053.637259:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f72940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665543040:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739053.637296:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.637326:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665543040 00000100:00100000:1.0:1787739053.637330:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665543040 00010000:00010000:1.0:1787739053.637349:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab826 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ab81f expref: 11 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739053.637371:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1383 00000100:00100000:1.0:1787739053.637378:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f703c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+11:3652:x1874579665543040:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739053.637387:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739053.651258:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa400/0x7ee4453e4a1ab7b6 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab7bd expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739053.651278:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa400/0x7ee4453e4a1ab7b6 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab7bd expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.651319:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa400/0x7ee4453e4a1ab7b6 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab7bd expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.651326:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739053.651363:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa400/0x7ee4453e4a1ab7b6 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab7bd expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787739053.651387:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968298 is committed 00000001:00080000:3.0:1787739053.651402:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968297 is committed 00000020:00080000:3.0:1787739053.651409:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 335 batchid updated 00000020:00080000:3.0:1787739053.651446:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa0730610ea80 refcount 2 committed 0 result 0 batchid 336 00000020:00080000:3.0:1787739053.651449:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2c00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730610eb40 00000020:00080000:3.0:1787739053.651452:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].329 00000020:00080000:3.0:1787739053.651454:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2100 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304010400 00000020:00080000:3.0:1787739053.651456:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].329 00010000:00010000:3.0:1787739053.651481:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab746 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x15d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.651489:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab746 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x15d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.651496:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab746 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x15d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.651502:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739053.651506:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab746 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x15d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.651512:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab746 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x15d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.651523:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15d:0x0].0x0 (ffffa07303c78600) 00010000:00010000:3.0:1787739053.651526:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739053.651531:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab746 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x15d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.651541:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ab746 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x15d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.651558:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab74d lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.651563:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab74d lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.651568:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab74d lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.651572:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633de00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739053.651575:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab74d lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.651580:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab74d lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.651588:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb4:0x0].0x0 (ffffa07303c78e00) 00010000:00010000:3.0:1787739053.651591:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739053.651593:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab74d lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.651599:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ab74d lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.651661:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab826 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ab81f expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.651671:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x2153044 (ffffa074331c2900) 00010000:00010000:3.0:1787739053.651676:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab857 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.651685:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab857 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.651692:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab857 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.651716:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739053.651800:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f703c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+6:3652:x1874579665543040:12345-0@lo:103:ldlm_bl.0 Request processed in 14420us (14501us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.651809:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1383 00010000:00010000:3.0:1787739053.651841:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab857 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.651849:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab857 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739053.651937:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a31e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665543296:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739053.651975:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.651998:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073070549c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.652030:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f72940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665543040:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739053.652052:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665543296 00000100:00100000:3.0:1787739053.652057:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665543296 00000100:00100000:3.0:1787739053.652161:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2569 00000100:00100000:3.0:1787739053.652170:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157e92c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665543296:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739053.652236:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157e92c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665543296:12345-0@lo:1000:mdt00_003.0 Request processed in 67us (260us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.652245:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2569 00000100:00100000:3.0:1787739053.652270:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a31e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665543296:0@lo:1000:mdt00_003.0 00010000:00010000:3.0:1787739053.652303:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ab85e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14f7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.652313:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ab85e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.652321:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ab85e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739053.652339:0:7944:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ab85e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.652353:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab850 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.652358:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab850 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.652363:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab850 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.652368:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab850 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.652373:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739053.652376:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab850 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.652382:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab850 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.652391:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305eae100) 00010000:00010000:3.0:1787739053.652393:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739053.652396:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab850 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.652402:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ab850 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.652432:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab857 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.652439:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab857 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.652444:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab857 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.652449:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ca00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739053.652452:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab857 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.652457:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab857 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.652464:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x2153044 (ffffa074331c2900) 00010000:00010000:3.0:1787739053.652466:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739053.652469:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab857 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.652474:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab857 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.652509:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab849 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094af4 expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.652524:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ab85e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094af4 expref: 7 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.652531:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f7:0x0].0x0 (ffffa074331c2800) 00010000:00010000:3.0:1787739053.652533:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739053.652536:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa200, rc 0) 00000100:00100000:3.0:1787739053.652585:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17430:x1874579652417920:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 17333us (17580us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.652591:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23181 00000100:00100000:2.0:1787739053.652734:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665543168:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:2.0:1787739053.652765:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.652832:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665543168 00000100:00100000:3.0:1787739053.652836:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665543168 00000100:00100000:3.0:1787739053.652873:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1182 00000100:00100000:3.0:1787739053.652880:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+7:3650:x1874579665543168:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:3.0:1787739053.652889:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739053.652903:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab7bd lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xaa220cb4 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab7b6 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.652912:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xaa220cb4 (ffffa074331c2100) 00010000:00010000:3.0:1787739053.652918:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739053.652948:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+6:3650:x1874579665543168:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 69us (182us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.652955:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1182 00000020:00080000:0.0:1787739053.653035:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 336 committed batchid 335 00080000:02000000:0.0:1787739053.653052:0:7475:0:(osd_handler.c:2218:osd_trans_start()) *** cfs_fail_loc=19a, val=0*** 00000004:00080000:0.0:1787739053.653117:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07410404cc0 with version 630 00000020:00080000:0.0:1787739053.653132:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 335 cancel update log [0x200000401:0x3:0x0].328: rc = 0 00000020:00080000:0.0:1787739053.653187:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 335 cancel update log [0x240000400:0x3:0x0].328: rc = 0 00000020:00080000:0.0:1787739053.653194:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 336 committed batchid 335 00000100:00100000:2.0:1787739053.653458:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665543168:0@lo:103:jbd2/dm-1-8.0 00000004:00080000:3.0:1787739053.653511:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 630 rpc_version 630 00000020:00100000:3.0:1787739053.653531:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739053.653539:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303a32d00 x1874579665543552/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739053.653555:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665543552:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739053.653584:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.653606:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.653897:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665543552 00000100:00100000:0.0:1787739053.653902:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665543552 00000100:00100000:0.0:1787739053.653944:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2570 00000100:00100000:0.0:1787739053.653951:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:7467:x1874579665543552:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739053.654068:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665543552:12345-0@lo:1000:osp_up0-1.0 Request processed in 117us (482us total) trans 4294983309 rc 0/0 00000100:00100000:0.0:1787739053.654090:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2570 00000100:00100000:3.0:1787739053.654284:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665543552:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739053.657809:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739053.657872:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652418048 00000100:00100000:2.0:1787739053.657881:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652418048 00000100:00100000:2.0:1787739053.657942:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23182 00000100:00100000:2.0:1787739053.657952:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17430:x1874579652418048:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739053.657974:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739053.657995:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab865 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x14f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094afb expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.658054:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab86c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14f7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.658067:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab86c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14f7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.658092:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab86c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14f7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739053.658176:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14f7:0x0]: returning size 4096 00000004:00200000:2.0:1787739053.658181:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739053.658219:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab865 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x14f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094afb expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.658238:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab86c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14f7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094afb expref: 17 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.658247:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f7:0x0].0x0 (ffffa07303c78800) 00010000:00010000:2.0:1787739053.658250:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739053.658253:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d200, rc 0) 00000100:00100000:2.0:1787739053.658312:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17430:x1874579652418048:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 361us (510us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739053.658319:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23182 00000100:00100000:0.0:1787739053.663818:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739053.663918:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652418176 00000100:00100000:3.0:1787739053.663928:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652418176 00000100:00100000:3.0:1787739053.663984:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23183 00000100:00100000:3.0:1787739053.663995:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17430:x1874579652418176:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739053.664014:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739053.664035:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab873 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094b02 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.664107:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab87a lrc: 3/1,0 mode: --/PR res: [0x200000400:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.664122:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab87a lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.664134:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab87a lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739053.664185:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xb7:0x0]: returning size 4096 00000004:00200000:3.0:1787739053.664190:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739053.664224:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab873 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b02 expref: 19 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.664247:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab87a lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b02 expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.664255:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb7:0x0].0x0 (ffffa07303c78500) 00010000:00010000:3.0:1787739053.664258:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739053.664261:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739053.664327:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17430:x1874579652418176:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 334us (518us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.664334:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23183 00000100:00100000:3.0:1787739053.667655:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739053.667701:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652418304 00000100:00100000:3.0:1787739053.667710:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652418304 00000100:00100000:3.0:1787739053.667760:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23184 00000100:00100000:3.0:1787739053.667770:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17430:x1874579652418304:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739053.667788:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739053.667807:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab881 lrc: 2/0,0 mode: --/CR res: [0x240000401:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094b09 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.667863:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab888 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.667878:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab842 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab83b expref: 5 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.667886:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab888 lrc: 4/1,0 mode: --/PR res: [0x240000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.667922:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab842 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ab83b expref: 5 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739053.667938:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665543680:0@lo:104: 00000100:00100000:3.0:1787739053.667983:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.668023:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054300 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.670694:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665543680 00000100:00100000:3.0:1787739053.670709:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665543680 00000100:00100000:3.0:1787739053.670772:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1057 00000100:00100000:3.0:1787739053.670782:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d2580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665543680:12345-0@lo:104: 00010000:00010000:3.0:1787739053.670799:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab83b lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab842 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.670899:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739053.670906:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d2580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665543680:12345-0@lo:104: Request processed in 126us (2921us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.670913:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1057 00000100:00100000:3.0:1787739053.670961:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665543680:0@lo:104: 00010000:00010000:3.0:1787739053.670982:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab888 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.671121:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739053.671129:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab83b lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab842 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.671143:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739053.671152:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab83b lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xb7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ab842 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739053.671162:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab83b lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xb7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ab842 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.671196:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab83b lrc: 2/0,0 mode: --/PW res: [0x240000401:0xb7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab842 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.671203:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739053.671233:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab83b lrc: 1/0,0 mode: --/PW res: [0x240000401:0xb7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab842 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.671241:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ab83b lrc: 0/0,0 mode: --/PW res: [0x240000401:0xb7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab842 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.671260:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739053.671261:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739053.671263:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739053.671270:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787739053.671282:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f70000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665543808:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739053.671322:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.671370:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665543808 00000100:00100000:1.0:1787739053.671374:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665543808 00010000:00010000:1.0:1787739053.671393:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab842 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ab83b expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739053.673307:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1384 00000100:00100000:1.0:1787739053.673325:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f721c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+6:3650:x1874579665543808:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739053.673345:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00080000:3.0:1787739053.677826:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 336 batchid updated 00010000:00010000:1.0:1787739053.678024:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ab842 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ab83b expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.678042:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb7:0x0].0x0 (ffffa073068a9800) 00010000:00010000:1.0:1787739053.678048:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab888 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.678058:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab888 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.678066:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab888 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.678138:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739053.678217:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f721c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+5:3650:x1874579665543808:12345-0@lo:103:ldlm_bl.0 Request processed in 4898us (6895us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.678226:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1384 00000100:00100000:1.0:1787739053.678271:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f70000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665543808:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739053.679067:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab888 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.679078:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab888 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739053.679157:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xb7:0x0]: returning size 4096 00000004:00200000:3.0:1787739053.679163:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739053.679221:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab881 lrc: 0/0,0 mode: --/CR res: [0x240000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b09 expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.679239:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab888 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b09 expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.679254:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb7:0x0].0x0 (ffffa073068a9800) 00010000:00010000:3.0:1787739053.679257:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739053.679259:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa800, rc 0) 00000100:00100000:3.0:1787739053.679327:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17430:x1874579652418304:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 11558us (11677us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.679334:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23184 00000020:00080000:2.0:1787739053.680079:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 336 00000004:00080000:2.0:1787739053.680146:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730610ecc0 with version 631 00000020:00080000:2.0:1787739053.680167:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 336 cancel update log [0x200000401:0x3:0x0].329: rc = 0 00000020:00080000:2.0:1787739053.680240:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 336 cancel update log [0x240000400:0x3:0x0].329: rc = 0 00000020:00080000:2.0:1787739053.680247:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 336 00000004:00080000:1.0:1787739053.680291:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 631 rpc_version 631 00000020:00100000:1.0:1787739053.680321:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739053.680340:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303f70000 x1874579665544064/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739053.680362:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f70000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665544064:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739053.680512:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.680547:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739053.680591:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665544064 00000100:00100000:2.0:1787739053.680597:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665544064 00000100:00100000:2.0:1787739053.680645:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2571 00000100:00100000:2.0:1787739053.680653:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f72d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665544064:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739053.680793:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f72d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665544064:12345-0@lo:1000:osp_up0-1.0 Request processed in 140us (282us total) trans 4294983310 rc 0/0 00000100:00100000:2.0:1787739053.680801:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2571 00000100:00100000:1.0:1787739053.680925:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f70000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665544064:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739053.689107:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739053.689154:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739053.689197:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652418432 00000100:00100000:3.0:1787739053.689207:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652418432 00000100:00100000:3.0:1787739053.689262:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1183 00000100:00100000:3.0:1787739053.689273:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba3850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2407:x1874579652418432:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:3.0:1787739053.689288:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739053.689311:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ab85e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14f7:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094af4 expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.689328:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f7:0x0].0x0 (ffffa074331c2800) 00010000:00010000:3.0:1787739053.689338:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739053.689362:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652418688 00000100:00100000:1.0:1787739053.689369:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652418688 00000100:00100000:3.0:1787739053.689386:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba3850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2407:x1874579652418432:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 114us (230us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.689395:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1183 00000100:00100000:1.0:1787739053.689410:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23185 00000100:00100000:1.0:1787739053.689443:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07411b75c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17430:x1874579652418688:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:1.0:1787739053.689491:0:6523:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739053.689508:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ab86c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14f7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094afb expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.689523:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f7:0x0].0x0 (ffffa07303c78800) 00010000:00010000:1.0:1787739053.689526:0:6523:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739053.689570:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1ab88f lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.689578:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1ab88f lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.689588:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1ab88f lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739053.689607:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f703c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665544192:0@lo:101:mdt00_000.0 00000100:00100000:1.0:1787739053.689646:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.689680:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.689700:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665544192 00000100:00100000:3.0:1787739053.689706:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665544192 00000100:00100000:3.0:1787739053.689745:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2572 00000100:00100000:3.0:1787739053.689752:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f721c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+5:6523:x1874579665544192:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739053.689762:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739053.689775:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab896 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab88f expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.689797:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab896 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab88f expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.689810:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab896 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab88f expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.689818:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074331c2500) 00010000:00010000:3.0:1787739053.689822:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739053.689825:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab600, rc 0) 00000100:00100000:3.0:1787739053.689879:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f721c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6523:x1874579665544192:12345-0@lo:101:mdt00_000.0 Request processed in 127us (231us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.689885:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2572 00000100:00100000:3.0:1787739053.689919:0:6523:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968298 gen 2 00000004:00080000:3.0:1787739053.689924:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968293 committed transno 4294968298 00000020:00080000:3.0:1787739053.689930:0:6523:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07305fcce40 refcount 2 committed 0 result 0 batchid 345 00000020:00080000:3.0:1787739053.689934:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730424d800 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07305fcc780 00000020:00080000:3.0:1787739053.689938:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].342 00000020:00080000:3.0:1787739053.689941:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730424d700 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa0730558c000 00000020:00080000:3.0:1787739053.689944:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].343 00000004:00080000:3.0:1787739053.689964:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968295 committed transno 4294968298 00000004:00080000:3.0:1787739053.689974:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968296 committed transno 4294968298 00000004:00080000:3.0:1787739053.689984:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968298 committed transno 4294968298 00000100:00100000:3.0:1787739053.690002:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f703c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665544192:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739053.690019:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1ab88f lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab896 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.690029:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1ab88f lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab896 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739053.690044:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739053.690065:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab89d lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.690072:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab89d lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.690080:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab89d lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739053.690104:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665544320:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739053.690129:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.690147:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054e40 going to sleep for 11 seconds 00000020:00080000:3.0:1787739053.690158:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 345 committed batchid 344 00000020:00080000:3.0:1787739053.690226:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 344 00000100:00100000:3.0:1787739053.690238:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665544320 00000100:00100000:3.0:1787739053.690242:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665544320 00000100:00100000:3.0:1787739053.690277:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2573 00000100:00100000:3.0:1787739053.690287:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6523:x1874579665544320:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739053.690296:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739053.690309:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab8a4 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab89d expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.690326:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab8a4 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab89d expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.690335:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab8a4 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab89d expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.690343:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x2153044 (ffffa074331c2f00) 00010000:00010000:3.0:1787739053.690346:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739053.690348:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abe00, rc 0) 00000100:00100000:3.0:1787739053.690385:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665544320:12345-0@lo:101:mdt00_000.0 Request processed in 100us (255us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.690391:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2573 00000100:00100000:3.0:1787739053.690445:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665544320:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739053.690456:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab89d lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab8a4 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.690464:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab89d lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab8a4 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739053.690478:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:3.0:1787739053.690509:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665544448:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739053.690540:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.690560:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.690576:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665544448 00000100:00100000:3.0:1787739053.690579:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665544448 00000100:00100000:3.0:1787739053.690607:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2574 00000100:00100000:3.0:1787739053.690613:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fde00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665544448:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739053.690681:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fde00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665544448:12345-0@lo:1000:mdt00_000.0 Request processed in 67us (140us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.690687:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2574 00000100:00100000:3.0:1787739053.690713:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665544448:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739053.690745:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fc780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665544576:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739053.690767:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.690783:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.690793:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665544576 00000100:00100000:3.0:1787739053.690797:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665544576 00000100:00100000:3.0:1787739053.690816:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2575 00000100:00100000:3.0:1787739053.690821:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d0780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665544576:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739053.690885:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d0780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665544576:12345-0@lo:1000:mdt00_000.0 Request processed in 63us (117us total) trans 0 rc 1/1 00000100:00100000:3.0:1787739053.690892:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2575 00000100:00100000:3.0:1787739053.690916:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fc780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665544576:0@lo:1000:mdt00_000.0 00010000:00010000:3.0:1787739053.690951:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aae00/0x7ee4453e4a1ab8ab lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.690959:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aae00/0x7ee4453e4a1ab8ab lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.690967:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aae00/0x7ee4453e4a1ab8ab lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739053.690979:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fc780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665544704:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739053.691002:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.691017:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.691028:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665544704 00000100:00100000:3.0:1787739053.691031:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665544704 00000100:00100000:3.0:1787739053.691052:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2576 00000100:00100000:3.0:1787739053.691059:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665544704:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739053.691067:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739053.691077:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab8b2 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab8ab expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.691107:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab8b2 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab8ab expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.691116:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab8b2 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab8ab expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.691122:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f7:0x0].0x0 (ffffa074331c2200) 00010000:00010000:3.0:1787739053.691124:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739053.691127:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab200, rc 0) 00000100:00100000:3.0:1787739053.691162:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6523:x1874579665544704:12345-0@lo:101:mdt00_000.0 Request processed in 103us (159us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.691168:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2576 00000100:00100000:3.0:1787739053.691193:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fc780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665544704:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739053.691203:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aae00/0x7ee4453e4a1ab8ab lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab8b2 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.691217:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aae00/0x7ee4453e4a1ab8ab lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab8b2 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.691234:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab8b9 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.691243:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab8b9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.691250:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab8b9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.691264:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab8c0 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.691274:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab87a lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b02 expref: 17 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.691282:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab8c0 lrc: 4/0,1 mode: --/EX res: [0x200000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.691299:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab87a lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b02 expref: 17 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.691311:0:6523:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab87a lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b02 expref: 17 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739053.691320:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fc780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665544832:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739053.691366:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.720883:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739053.720984:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fc780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665544832:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739053.721011:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab8c0 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.731650:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652418816 00000100:00100000:0.0:1787739053.731663:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652418816 00010000:00010000:3.0:1787739053.732090:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab87a lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b02 expref: 18 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739053.732128:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1385 00000100:00100000:3.0:1787739053.732139:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2409:x1874579652418816:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739053.732155:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739053.732164:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab87a lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b02 expref: 18 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.732177:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab87a lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b02 expref: 18 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.732200:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab87a lrc: 0/0,0 mode: --/PR res: [0x200000400:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b02 expref: 18 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.732212:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb7:0x0].0x0 (ffffa07303c78500) 00010000:00010000:3.0:1787739053.732218:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab8c0 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.732228:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab8c0 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.732237:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab8c0 lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.732266:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739053.732325:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652418816:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 188us (11451us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.732333:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1385 00010000:00010000:3.0:1787739053.732498:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab8c0 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.732506:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab8c0 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.732547:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ab8c7 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.732555:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ab8c7 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.732565:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ab8c7 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739053.732583:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fc780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665544960:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739053.732632:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.732661:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.733152:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665544960 00000100:00100000:3.0:1787739053.733158:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665544960 00000100:00100000:3.0:1787739053.733251:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2577 00000100:00100000:3.0:1787739053.733260:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6523:x1874579665544960:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739053.733276:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739053.733291:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ab8ce lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab8c7 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.733317:0:18535:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab888 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b09 expref: 6 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.733326:0:18535:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ab8ce lrc: 4/0,0 mode: --/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab8c7 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.733349:0:18535:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab888 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b09 expref: 6 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.733361:0:18535:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab888 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b09 expref: 6 pid: 7944 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739053.733371:0:18535:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7840 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:18535:1874579665545088:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739053.733432:0:18535:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054540 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.746438:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739053.746473:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652418944 00000100:00100000:0.0:1787739053.746482:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652418944 00010000:00010000:0.0:1787739053.746522:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab888 lrc: 5/0,0 mode: PR/PR res: [0x240000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b09 expref: 7 pid: 7944 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.746556:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1386 00000100:00100000:0.0:1787739053.746565:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730564e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2407:x1874579652418944:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739053.746579:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739053.746585:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab888 lrc: 5/0,0 mode: PR/PR res: [0x240000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b09 expref: 7 pid: 7944 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.746597:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab888 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b09 expref: 7 pid: 7944 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.746618:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb7:0x0].0x0 (ffffa073068a9800) 00010000:00010000:0.0:1787739053.746624:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ab8ce lrc: 3/0,0 mode: --/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab8c7 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.746634:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ab8ce lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab8c7 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.746643:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ab8ce lrc: 4/0,0 mode: EX/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab8c7 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.746980:0:18535:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ab888 lrc: 0/0,0 mode: --/PR res: [0x240000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b09 expref: 7 pid: 7944 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739053.747001:0:18535:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7840 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:18535:1874579665545088:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739053.747025:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ab8ce lrc: 4/0,0 mode: EX/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab8c7 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.747033:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb7:0x0].0x0 (ffffa073068a9800) 00010000:00010000:1.0:1787739053.747035:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739053.747038:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa200, rc 0) 00000100:00100000:1.0:1787739053.747100:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6523:x1874579665544960:12345-0@lo:101:mdt00_000.0 Request processed in 13842us (14468us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.747108:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2577 00000100:00100000:1.0:1787739053.747146:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fc780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665544960:0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739053.747161:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ab8c7 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab8ce expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.747171:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ab8c7 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab8ce expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.747248:0:6508:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ab8ce lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab8c7 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.747262:0:6508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d84000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001_UUID:6508:1874579665545216:0@lo:105: 00000100:00100000:0.0:1787739053.747294:0:6508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.747328:0:6508:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730638b3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739053.747387:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665545216 00000100:00100000:1.0:1787739053.747392:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665545216 00000100:00100000:1.0:1787739053.747442:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1058 00000100:00100000:1.0:1787739053.747450:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d843c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6508:x1874579665545216:12345-0@lo:105: 00010000:00010000:1.0:1787739053.747460:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ab8c7 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab8ce expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.747469:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ab8c7 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab8ce expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.747478:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ab8c7 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab8ce expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.747486:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ab8c7 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab8ce expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.747493:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ab8c7 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab8ce expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.747501:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ab8c7 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ab8ce expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.747516:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069ab800) 00000100:00100000:1.0:1787739053.747545:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d843c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6508:x1874579665545216:12345-0@lo:105: Request processed in 97us (250us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.747551:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1058 00010000:00010000:1.0:1787739053.747569:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ab8c7 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab8ce expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.747577:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ab8c7 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab8ce expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739053.748362:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f72940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665545344:0@lo:403:mdt00_000.0 00000100:00100000:1.0:1787739053.748408:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.748456:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739053.748483:0:6508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d84000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001_UUID:6508:1874579665545216:0@lo:105: 00010000:00010000:1.0:1787739053.748496:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739053.748534:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730564e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2407:x1874579652418944:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 1969us (2120us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.748542:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1386 00000100:00100000:0.0:1787739053.748934:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665545344 00000100:00100000:0.0:1787739053.748950:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665545344 00000100:00100000:0.0:1787739053.748991:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2578 00000100:00100000:0.0:1787739053.748999:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f703c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6523:x1874579665545344:12345-0@lo:403:mdt00_000.0 00000100:00100000:1.0:1787739053.765866:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f703c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6523:x1874579665545344:12345-0@lo:403:mdt00_000.0 Request processed in 16858us (17447us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.765880:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2578 00000100:00100000:0.0:1787739053.776627:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f72940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665545344:0@lo:403:mdt00_000.0 00000040:00080000:0.0:1787739053.777760:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].345, 664 off529952 00000040:00080000:0.0:1787739053.777932:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].344, 744 off572904 00000004:00080000:0.0:1787739053.777939:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730638b0c0 with version 656 00000004:00080000:2.0:1787739053.781830:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 656 rpc_version 656 00000020:00100000:2.0:1787739053.781887:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739053.781899:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306be5a40 x1874579665545600/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1808/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739053.781924:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665545600:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739053.781993:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.782027:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739053.782054:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665545600 00000100:00100000:2.0:1787739053.782061:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665545600 00000100:00100000:2.0:1787739053.782118:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2579 00000100:00100000:2.0:1787739053.782128:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665545600:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739053.784263:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665545600:12345-0@lo:1000:osp_up1-0.0 Request processed in 2131us (2264us total) trans 4294968299 rc 0/0 00000100:00100000:2.0:1787739053.785430:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2579 00000100:00100000:2.0:1787739053.789605:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665545600:0@lo:1000:osp_up1-0.0 00000004:00200000:0.0:1787739053.790542:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14f7:0x0]: returning size 4096 00010000:00010000:0.0:1787739053.790573:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab8c0 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.790584:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab8c0 lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.790591:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab8c0 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.790597:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739053.790601:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab8c0 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.790608:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab8c0 lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.790624:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb7:0x0].0x0 (ffffa07303c78500) 00010000:00010000:0.0:1787739053.790627:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739053.790631:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab8c0 lrc: 2/0,0 mode: --/EX res: [0x200000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.790636:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ab8c0 lrc: 0/0,0 mode: --/EX res: [0x200000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.790649:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ab8c7 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab8ce expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.790663:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ab8c7 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab8ce expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.790669:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ab8c7 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab8ce expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.790701:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739053.790712:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xb7:0x0] 00010000:00010000:0.0:1787739053.790720:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab8b9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.790912:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab8b9 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.790921:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab8b9 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.790927:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739053.790929:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab8b9 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.790935:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab8b9 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.790954:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f7:0x0].0x0 (ffffa07303c78800) 00010000:00010000:0.0:1787739053.790957:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739053.790960:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab8b9 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.790966:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ab8b9 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.790979:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aae00/0x7ee4453e4a1ab8ab lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab8b2 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.790986:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aae00/0x7ee4453e4a1ab8ab lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab8b2 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.790993:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aae00/0x7ee4453e4a1ab8ab lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab8b2 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.791013:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:0.0:1787739053.791026:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739053.791030:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1ab88f lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab896 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.791036:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1ab88f lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab896 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.791042:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1ab88f lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab896 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.791056:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:0.0:1787739053.791059:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739053.791064:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab89d lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab8a4 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.791071:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab89d lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab8a4 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.791167:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07411b75c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17430:x1874579652418688:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 101725us (102067us total) trans 4294983311 rc 0/0 00000100:00100000:0.0:1787739053.791176:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23185 00010000:00010000:2.0:1787739053.793668:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1787739053.793692:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ab8c7 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab8ce expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.793705:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739053.793711:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ab8c7 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab8ce expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739053.793731:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ab8c7 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab8ce expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.793770:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ab8c7 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab8ce expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.793776:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739053.793802:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ab8c7 lrc: 1/0,0 mode: --/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab8ce expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.793809:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ab8c7 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab8ce expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.793828:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1787739053.793831:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aae00/0x7ee4453e4a1ab8ab lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab8b2 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.793837:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aae00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739053.793840:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aae00/0x7ee4453e4a1ab8ab lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab8b2 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739053.793846:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aae00/0x7ee4453e4a1ab8ab lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab8b2 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.793867:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aae00/0x7ee4453e4a1ab8ab lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab8b2 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.793873:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739053.793878:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aae00/0x7ee4453e4a1ab8ab lrc: 1/0,0 mode: --/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab8b2 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.793884:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aae00/0x7ee4453e4a1ab8ab lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab8b2 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.793894:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739053.793897:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1ab88f lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab896 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.793902:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1da00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739053.793905:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1ab88f lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab896 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739053.793911:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1ab88f lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab896 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.793923:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1ab88f lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab896 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.793929:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739053.793950:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1ab88f lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab896 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.793957:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1ab88f lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab896 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.793968:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739053.793970:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739053.793972:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739053.793980:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787739053.793997:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665545984:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739053.794047:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.794095:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665545984 00000100:00100000:2.0:1787739053.794101:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665545984 00000100:00100000:2.0:1787739053.794143:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1184 00000100:00100000:2.0:1787739053.794150:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+10:3651:x1874579665545984:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739053.794163:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739053.794179:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ab896 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab88f expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.794191:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074331c2500) 00010000:00010000:2.0:1787739053.794198:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739053.794230:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+9:3651:x1874579665545984:12345-0@lo:103:ldlm_bl.0 Request processed in 80us (182us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739053.794237:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1184 00000100:00100000:2.0:1787739053.794497:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665545984:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739053.794636:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665545728:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739053.794666:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.794702:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787739053.794708:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665545856:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739053.794742:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.794769:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665545728 00000100:00100000:0.0:1787739053.794772:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665545728 00000100:00100000:0.0:1787739053.794800:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665545856 00000100:00100000:0.0:1787739053.794803:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665545856 00000100:00100000:0.0:1787739053.794822:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1185 00000100:00100000:0.0:1787739053.794828:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+10:3652:x1874579665545728:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739053.794837:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739053.794990:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ab8ce lrc: 0/0,0 mode: --/EX res: [0x240000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab8c7 expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.795003:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb7:0x0].0x0 (ffffa073068a9800) 00010000:00010000:0.0:1787739053.795010:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739053.795040:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+9:3652:x1874579665545728:12345-0@lo:103:ldlm_bl.0 Request processed in 212us (373us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739053.795047:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1185 00000100:00100000:0.0:1787739053.795061:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1186 00000100:00100000:0.0:1787739053.795067:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d85a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+8:3652:x1874579665545856:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739053.795077:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739053.795098:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ab8b2 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14f7:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab8ab expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.795108:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f7:0x0].0x0 (ffffa074331c2200) 00010000:00010000:0.0:1787739053.795116:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739053.795142:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d85a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+7:3652:x1874579665545856:12345-0@lo:103:ldlm_bl.0 Request processed in 76us (399us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739053.795148:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1186 00000100:00100000:0.0:1787739053.795177:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665545856:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739053.795185:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665545728:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739053.795373:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739053.795379:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739053.835624:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739053.836018:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652419072 00000100:00100000:2.0:1787739053.836028:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652419072 00000100:00100000:2.0:1787739053.836101:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23186 00000100:00100000:2.0:1787739053.836112:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d861c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17440:x1874579652419072:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739053.836134:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739053.836155:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab8d5 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094b10 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.836216:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab8dc lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.836235:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab8dc lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.836247:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab8dc lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.836261:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab8e3 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.836268:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab8e3 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.836275:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab8e3 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.836317:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab8dc lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.836324:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab8dc lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.836331:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab8dc lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.836337:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab8dc lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.836344:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739053.836347:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab8dc lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.836354:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab8dc lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.836366:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2b00) 00010000:00010000:2.0:1787739053.836369:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739053.836373:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab8dc lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.836380:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab8dc lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.836391:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab8e3 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.836397:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab8e3 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.836403:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab8e3 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.836408:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739053.836428:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab8e3 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.836435:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab8e3 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.836443:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x836d1894 (ffffa07303c78300) 00010000:00010000:2.0:1787739053.836445:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739053.836448:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab8e3 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.836454:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ab8e3 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.836488:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab8d5 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b10 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.836498:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2b00) 00010000:00010000:2.0:1787739053.836501:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739053.836505:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ab8d5 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b10 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.836515:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d000, rc 0) 00000100:00100000:2.0:1787739053.836573:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d861c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17440:x1874579652419072:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 463us (957us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739053.836581:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23186 00000100:00100000:0.0:1787739053.845489:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739053.845576:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652419200 00000100:00100000:3.0:1787739053.845586:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652419200 00000100:00100000:3.0:1787739053.845647:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23187 00000100:00100000:3.0:1787739053.845658:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d87c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17440:x1874579652419200:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739053.845687:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739053.845710:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab8ea lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094b17 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.845776:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab8f1 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.845789:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab8f1 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.845801:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab8f1 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739053.845844:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x5:0x0]: returning size 4096 00000004:00200000:3.0:1787739053.845859:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739053.845894:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab8ea lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b17 expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.845911:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab8f1 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b17 expref: 7 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.845920:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074331c2000) 00010000:00010000:3.0:1787739053.845922:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739053.845925:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa400, rc 0) 00000100:00100000:3.0:1787739053.845985:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d87c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17440:x1874579652419200:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 328us (505us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.845993:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23187 00000100:00100000:3.0:1787739053.856148:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739053.856316:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652419456 00000100:00100000:0.0:1787739053.856325:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652419456 00000100:00100000:0.0:1787739053.856378:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23188 00000100:00100000:0.0:1787739053.856387:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17440:x1874579652419456:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:0.0:1787739053.856520:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d86940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665546112:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739053.856570:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.856618:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739053.856692:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665546112 00000100:00100000:1.0:1787739053.856697:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665546112 00000100:00100000:1.0:1787739053.856735:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2580 00000100:00100000:1.0:1787739053.856743:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d86580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665546112:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739053.856817:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d86580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665546112:12345-0@lo:1000:mdt00_000.0 Request processed in 74us (245us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.856824:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2580 00000100:00100000:1.0:1787739053.856862:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d86940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665546112:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739053.856907:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f71e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665546240:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739053.856933:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.857651:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e890c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.864268:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665546240 00000100:00100000:0.0:1787739053.864283:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665546240 00000100:00100000:0.0:1787739053.864348:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2581 00000100:00100000:0.0:1787739053.864360:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f712c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665546240:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739053.864521:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f712c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665546240:12345-0@lo:1000:mdt00_000.0 Request processed in 163us (7585us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739053.864529:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2581 00000100:00100000:0.0:1787739053.871233:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f71e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665546240:0@lo:1000:mdt00_000.0 00010000:00010000:0.0:1787739053.871318:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ab8f8 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.871329:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ab8f8 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.871339:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ab8f8 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739053.871359:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d85e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665546368:0@lo:101:mdt00_000.0 00000100:00100000:0.0:1787739053.871442:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.871656:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc840 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.872492:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665546368 00000100:00100000:3.0:1787739053.872500:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665546368 00000100:00100000:3.0:1787739053.872548:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2582 00000100:00100000:3.0:1787739053.872556:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d870c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665546368:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739053.872572:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739053.872587:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab8ff lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab8f8 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.872613:0:17834:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab80a lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aed expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.872623:0:17834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab8ff lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab8f8 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.872647:0:17834:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab80a lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aed expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.872659:0:17834:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab80a lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aed expref: 16 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739053.872670:0:17834:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7480 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:17834:1874579665546496:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739053.873375:0:17834:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739053.882776:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739053.883013:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652419584 00000100:00100000:1.0:1787739053.883021:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652419584 00010000:00010000:1.0:1787739053.883062:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab80a lrc: 5/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aed expref: 17 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739053.883113:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1387 00000100:00100000:1.0:1787739053.883125:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652419584:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739053.883144:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739053.883151:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab80a lrc: 5/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aed expref: 17 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.883165:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab80a lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aed expref: 17 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.883188:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2b00) 00010000:00010000:1.0:1787739053.883194:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab8ff lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab8f8 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.883205:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab8ff lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab8f8 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.883215:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab8ff lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab8f8 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.883249:0:6509:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab8ff lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab8f8 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739053.883262:0:6509:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f70f00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665546624:0@lo:105: 00000100:00100000:1.0:1787739053.883308:0:6509:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.883347:0:6509:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89180 going to sleep for 11 seconds 00000100:00100000:1.0:1787739053.883367:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665546624 00000100:00100000:1.0:1787739053.883371:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665546624 00000100:00100000:1.0:1787739053.883398:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1059 00000100:00100000:1.0:1787739053.883405:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f71680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665546624:12345-0@lo:105: 00010000:00010000:1.0:1787739053.883436:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ab8f8 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab8ff expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.883445:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ab8f8 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab8ff expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.883454:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ab8f8 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab8ff expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.883461:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ab8f8 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab8ff expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.883468:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ab8f8 lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab8ff expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.883475:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ab8f8 lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ab8ff expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.883489:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609a000) 00000100:00100000:1.0:1787739053.883524:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f71680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665546624:12345-0@lo:105: Request processed in 119us (215us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.883531:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1059 00000100:00100000:1.0:1787739053.883572:0:6509:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f70f00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665546624:0@lo:105: 00010000:00010000:1.0:1787739053.883590:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739053.883627:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652419584:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 505us (862us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.883634:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1387 00010000:00010000:3.0:1787739053.886092:0:17834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab80a lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094aed expref: 16 pid: 6524 timeout: 740 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739053.886118:0:17834:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7480 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:17834:1874579665546496:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739053.886148:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab8ff lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab8f8 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.886158:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2b00) 00010000:00010000:3.0:1787739053.886161:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739053.886165:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab000, rc 0) 00000100:00100000:3.0:1787739053.887265:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d870c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665546368:12345-0@lo:101:mdt00_000.0 Request processed in 14702us (15816us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.887286:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2582 00000100:00100000:0.0:1787739053.888109:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d85e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665546368:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739053.888137:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ab8f8 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab8ff expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739053.888163:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739053.888231:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab906 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.888237:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab906 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.888246:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab906 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739053.888266:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f70f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665546752:0@lo:101:mdt00_000.0 00000100:00100000:1.0:1787739053.888316:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.888348:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89180 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.888460:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665546752 00000100:00100000:3.0:1787739053.888467:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665546752 00000100:00100000:3.0:1787739053.888509:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2583 00000100:00100000:3.0:1787739053.888518:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f730c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665546752:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739053.888534:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739053.888550:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab90d lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab906 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.888573:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab90d lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab906 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.888586:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab90d lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab906 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739053.888594:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x836d1894 (ffffa074331c2e00) 00010000:00010000:3.0:1787739053.888596:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739053.888599:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739053.888653:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f730c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665546752:12345-0@lo:101:mdt00_000.0 Request processed in 135us (336us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.888659:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2583 00000100:00100000:0.0:1787739053.888758:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f70f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665546752:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739053.888769:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab906 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab90d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.888778:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab906 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab90d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739053.888798:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:0.0:1787739053.888829:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d85e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665546880:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739053.888861:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.888887:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc600 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.888930:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665546880 00000100:00100000:3.0:1787739053.888934:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665546880 00000100:00100000:3.0:1787739053.889151:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2584 00000100:00100000:3.0:1787739053.889158:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665546880:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739053.889233:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665546880:12345-0@lo:1000:mdt00_000.0 Request processed in 75us (371us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.889240:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2584 00000100:00100000:2.0:1787739053.889319:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d85e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665546880:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739053.889374:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be6d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665547008:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739053.889403:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.889525:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.890044:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665547008 00000100:00100000:3.0:1787739053.890050:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665547008 00000100:00100000:3.0:1787739053.890095:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2585 00000100:00100000:3.0:1787739053.890101:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665547008:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739053.890218:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665547008:12345-0@lo:1000:mdt00_000.0 Request processed in 117us (813us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739053.890225:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2585 00000100:00100000:2.0:1787739053.890570:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be6d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665547008:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739053.890666:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be6d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665547136:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739053.890696:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739053.890723:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.891201:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665547136 00000100:00100000:3.0:1787739053.891207:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665547136 00000100:00100000:3.0:1787739053.891252:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2586 00000100:00100000:3.0:1787739053.891261:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be6940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665547136:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739053.891349:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be6940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665547136:12345-0@lo:1000:mdt00_000.0 Request processed in 89us (651us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739053.891355:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2586 00000100:00100000:3.0:1787739053.891386:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be6d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665547136:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739053.891471:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665547264:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739053.891497:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.891526:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665547264 00000100:00100000:3.0:1787739053.891529:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665547264 00000100:00100000:3.0:1787739053.891544:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2587 00000100:00100000:3.0:1787739053.891549:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f4780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665547264:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739053.891586:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f4780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665547264:12345-0@lo:1000:mdt00_000.0 Request processed in 37us (88us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.891592:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2587 00000100:00100000:3.0:1787739053.891605:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073070540c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.891617:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665547264:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739053.891664:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665547392:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739053.891687:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.891706:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665547392 00000100:00100000:3.0:1787739053.891709:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665547392 00000100:00100000:3.0:1787739053.891723:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2588 00000100:00100000:3.0:1787739053.891728:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f43c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665547392:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739053.891797:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f43c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665547392:12345-0@lo:1000:mdt00_000.0 Request processed in 68us (108us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.891803:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2588 00000100:00100000:3.0:1787739053.891818:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073070540c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.891829:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665547392:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739053.891864:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665547520:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739053.891885:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.891902:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665547520 00000100:00100000:3.0:1787739053.891905:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665547520 00000100:00100000:3.0:1787739053.891918:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2589 00000100:00100000:3.0:1787739053.891923:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f7c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665547520:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739053.892111:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f7c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665547520:12345-0@lo:1000:mdt00_000.0 Request processed in 187us (224us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739053.892117:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2589 00000100:00100000:3.0:1787739053.892131:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073070540c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739053.892143:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665547520:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739053.892171:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665547648:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739053.892190:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739053.892216:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073070540c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739053.892650:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665547648 00000100:00100000:1.0:1787739053.892656:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665547648 00000100:00100000:1.0:1787739053.892693:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2590 00000100:00100000:1.0:1787739053.892700:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f5a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665547648:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739053.892751:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f5a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665547648:12345-0@lo:1000:mdt00_000.0 Request processed in 52us (560us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739053.892757:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2590 00000100:00100000:3.0:1787739053.893210:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665547648:0@lo:1000:mdt00_000.0 00000040:00080000:2.0:1787739053.894747:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].331, 2168 off505304 00000040:00080000:2.0:1787739053.894828:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].331, 2368 off551840 00000004:00080000:2.0:1787739053.894834:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730610ea80 with version 632 00000004:00080000:1.0:1787739053.898753:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 632 rpc_version 632 00000020:00100000:1.0:1787739053.898805:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739053.898819:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303f70b40 x1874579665547904/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739053.898843:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f70b40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665547904:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739053.899034:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739053.899076:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89900 going to sleep for 11 seconds 00000100:00100000:2.0:1787739053.902062:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665547904 00000100:00100000:2.0:1787739053.902074:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665547904 00000100:00100000:2.0:1787739053.902830:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2591 00000100:00100000:2.0:1787739053.902843:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f73480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665547904:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739053.903528:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f73480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:7467:x1874579665547904:12345-0@lo:1000:osp_up0-1.0 Request processed in 687us (4494us total) trans 4294983312 rc 0/0 00000100:00100000:2.0:1787739053.903537:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2591 00000100:00100000:1.0:1787739053.904687:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f70b40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665547904:0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1787739053.904819:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab914 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x15f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.904836:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab914 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x15f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.904858:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab914 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x15f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.904878:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab91b lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.904886:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab91b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.904892:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab91b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.904922:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ab922 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.904929:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ab922 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739053.904939:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ab922 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739053.904957:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f72580 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665548032:0@lo:101:mdt00_000.0 00000100:00100000:1.0:1787739053.905007:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.911762:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665548032 00000100:00100000:0.0:1787739053.911778:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665548032 00000100:00100000:0.0:1787739053.911845:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2592 00000100:00100000:0.0:1787739053.911967:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f70780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665548032:12345-0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739053.911991:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739053.912013:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab929 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab922 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.912041:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab929 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab922 expref: 12 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.912055:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab929 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab922 expref: 12 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.912063:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb5:0x0].0x0 (ffffa073068a9f00) 00010000:00010000:0.0:1787739053.912066:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739053.912069:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b800, rc 0) 00000100:00100000:0.0:1787739053.912144:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f70780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:6523:x1874579665548032:12345-0@lo:101:mdt00_000.0 Request processed in 182us (7137us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739053.912150:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2592 00000100:00100000:2.0:1787739053.914373:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89900 going to sleep for 11 seconds 00000100:00100000:2.0:1787739053.914438:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f72580 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665548032:0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739053.914463:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ab922 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab929 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.914476:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ab922 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab929 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739053.914499:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006af1935e reply state 00000000006dba04 transno 4294968300 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab91b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.914513:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab91b lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.914520:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb5:0x0].0x0 (ffffa07433235800) 00010000:00010000:2.0:1787739053.914523:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739053.914526:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787739053.914531:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ab922 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab929 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.914539:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ab922 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab929 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739053.914550:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xb5:0x0] 00000004:00010000:2.0:1787739053.914557:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006af1935e reply state 00000000006dba04 transno 4294968300 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab914 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x15f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.914564:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab914 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x15f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.914569:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15f:0x0].0x0 (ffffa07433235900) 00010000:00010000:2.0:1787739053.914571:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739053.914573:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:2.0:1787739053.914577:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x15f:0x0]: returning size 4096 00000004:00080000:2.0:1787739053.914585:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739053.914593:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ab8f8 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab8ff expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.914600:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ab8f8 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab8ff expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.914606:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ab8f8 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab8ff expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.914630:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739053.914634:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739053.914638:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab906 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab90d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739053.914646:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab906 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab90d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.914814:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739053.914821:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ab8f8 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab8ff expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.914830:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739053.914835:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ab8f8 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab8ff expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739053.914843:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ab8f8 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab8ff expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.914882:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ab8f8 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab8ff expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.914889:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739053.914912:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ab8f8 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab8ff expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.914919:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ab8f8 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab8ff expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.914935:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739053.914937:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739053.914939:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739053.914953:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d86d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665548160:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739053.915003:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739053.915051:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665548160 00000100:00100000:0.0:1787739053.915058:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665548160 00000100:00100000:0.0:1787739053.915111:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1187 00000100:00100000:0.0:1787739053.915120:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba1850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+12:3651:x1874579665548160:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739053.915132:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739053.915148:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ab8ff lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab8f8 expref: 12 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739053.915161:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2b00) 00010000:00010000:0.0:1787739053.915168:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739053.915196:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba1850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+11:3651:x1874579665548160:12345-0@lo:103:ldlm_bl.0 Request processed in 77us (192us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739053.915203:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1187 00000100:00100000:0.0:1787739053.915257:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d86d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665548160:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739053.915446:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17440:x1874579652419456:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 59058us (59301us total) trans 4294968300 rc 0/0 00000100:00100000:2.0:1787739053.915456:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23188 00000100:00100000:0.0:1787739053.947316:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739053.947399:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652419840 00000100:00100000:1.0:1787739053.947408:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652419840 00000100:00100000:1.0:1787739053.947487:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23189 00000100:00100000:1.0:1787739053.947497:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d86d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17441:x1874579652419840:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:1.0:1787739053.947656:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d86d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17441:x1874579652419840:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 159us (349us total) trans 0 rc -17/-17 00000100:00100000:1.0:1787739053.947663:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23189 00000100:00100000:0.0:1787739054.010673:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739054.012034:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652419968 00000100:00100000:1.0:1787739054.012047:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652419968 00000100:00100000:1.0:1787739054.012119:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23190 00000100:00100000:1.0:1787739054.012130:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17442:x1874579652419968:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739054.012150:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739054.012174:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ab930 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094b1e expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.012240:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab937 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.012255:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab937 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.012266:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab937 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.012289:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab93e lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.012300:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab90d lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab906 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.012308:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab93e lrc: 4/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.012358:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab90d lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ab906 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739054.012374:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f71a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665548288:0@lo:104: 00000100:00100000:1.0:1787739054.012604:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739054.012755:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89600 going to sleep for 11 seconds 00000100:00100000:0.0:1787739054.014967:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665548288 00000100:00100000:0.0:1787739054.014981:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665548288 00000100:00100000:0.0:1787739054.015089:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1060 00000100:00100000:0.0:1787739054.015100:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f73840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665548288:12345-0@lo:104: 00010000:00010000:0.0:1787739054.015118:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab906 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab90d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.015218:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787739054.015225:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f73840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665548288:12345-0@lo:104: Request processed in 128us (2621us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739054.015231:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1060 00010000:00010000:2.0:1787739054.015300:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739054.015308:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab906 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab90d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.015317:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1dc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739054.015322:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab906 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ab90d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739054.015345:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab906 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ab90d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.015380:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab906 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab90d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.015388:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739054.015410:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab906 lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab90d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.015437:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1ab906 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab90d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.015457:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739054.015459:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739054.015461:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739054.019192:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f71a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665548288:0@lo:104: 00010000:00010000:3.0:1787739054.019221:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab93e lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739054.020481:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665548544:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787739054.020547:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739054.020585:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787739054.020591:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665548416:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739054.020617:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739054.020646:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665548416 00000100:00100000:2.0:1787739054.020653:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665548416 00010000:00010000:2.0:1787739054.020681:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab90d lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ab906 expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739054.020716:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1388 00000100:00100000:2.0:1787739054.020736:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+11:3653:x1874579665548416:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739054.020749:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739054.023148:0:6502:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab89d lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab8a4 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739054.023166:0:6502:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab89d lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab8a4 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.023203:0:6502:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab89d lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab8a4 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.023210:0:6502:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739054.023243:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ab89d lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab8a4 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787739054.023261:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983312 is committed 00000001:00080000:3.0:1787739054.023283:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983311 is committed 00000020:00080000:3.0:1787739054.023291:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 345 batchid updated 00000020:00080000:3.0:1787739054.023303:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07410404f00 refcount 2 committed 0 result 0 batchid 346 00000020:00080000:3.0:1787739054.023307:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa074051e2d00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa074104046c0 00000020:00080000:3.0:1787739054.023310:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].343 00000020:00080000:3.0:1787739054.023313:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa074051e2b00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306962000 00000020:00080000:3.0:1787739054.023315:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].344 00000020:00080000:3.0:1787739054.023332:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 346 committed batchid 345 00000004:00080000:3.0:1787739054.023452:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07307054240 with version 657 00000020:00080000:3.0:1787739054.023460:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 345 cancel update log [0x240000401:0x3:0x0].342: rc = 0 00000020:00080000:3.0:1787739054.023495:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 345 cancel update log [0x200000400:0x3:0x0].343: rc = 0 00000020:00080000:3.0:1787739054.023503:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 345 00010000:00010000:3.0:1787739054.023548:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab90d lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ab906 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.023560:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x836d1894 (ffffa074331c2e00) 00010000:00010000:3.0:1787739054.023564:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab93e lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.023573:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab93e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.023581:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab93e lrc: 4/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.023601:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739054.023677:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+6:3653:x1874579665548416:12345-0@lo:103:ldlm_bl.0 Request processed in 2953us (3058us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.023687:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1388 00010000:00010000:3.0:1787739054.023717:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab93e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.023724:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab93e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739054.023806:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f4b40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665548800:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739054.023973:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739054.024002:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739054.024022:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665548544 00000100:00100000:3.0:1787739054.024027:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665548544 00000100:00100000:3.0:1787739054.024057:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665548800 00000100:00100000:3.0:1787739054.024059:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665548800 00000100:00100000:3.0:1787739054.024074:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2593 00000100:00100000:3.0:1787739054.024092:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be7840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:3653:x1874579665548544:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1787739054.024122:0:8447:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1445191680 avail 1298952192 00000100:00100000:3.0:1787739054.024155:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be7840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:3653:x1874579665548544:12345-0@lo:41:osp-pre-1-0.0 Request processed in 75us (3609us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.024161:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2593 00000100:00100000:3.0:1787739054.024173:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2594 00000100:00100000:3.0:1787739054.024178:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07436d8c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665548800:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739054.024223:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07436d8c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665548800:12345-0@lo:1000:mdt00_003.0 Request processed in 45us (375us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.024228:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2594 00000100:00100000:3.0:1787739054.024512:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f4b40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665548800:0@lo:1000:mdt00_003.0 00010000:00010000:3.0:1787739054.024554:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab945 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x15f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.024565:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab945 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.024573:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab945 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739054.024592:0:7944:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab945 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.024604:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab937 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.024610:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab937 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.024616:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab937 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.024622:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab937 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.024628:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739054.024631:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab937 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.024637:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab937 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.024646:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235400) 00010000:00010000:3.0:1787739054.024648:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739054.024652:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab937 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.024658:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ab937 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.024669:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab93e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.024674:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab93e lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.024680:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab93e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.024685:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739054.024688:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab93e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.024694:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab93e lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.024700:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x836d1894 (ffffa074331c2e00) 00010000:00010000:3.0:1787739054.024703:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739054.024706:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab93e lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.024711:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ab93e lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.024744:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ab930 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b1e expref: 16 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.024762:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab945 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x15f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b1e expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.024769:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15f:0x0].0x0 (ffffa074331c2c00) 00010000:00010000:3.0:1787739054.024771:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739054.024774:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aaa00, rc 0) 00000100:00100000:3.0:1787739054.024817:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17442:x1874579652419968:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 12689us (17547us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.024824:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23190 00000100:00100000:2.0:1787739054.024973:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665548672:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787739054.025008:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1787739054.025042:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 657 rpc_version 657 00000020:00100000:2.0:1787739054.025060:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739054.025067:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306be5e00 x1874579665549056/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739054.025093:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be5e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665549056:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739054.025120:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739054.025136:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730638b3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739054.025150:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665549056 00000100:00100000:2.0:1787739054.025153:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665549056 00000100:00100000:2.0:1787739054.025178:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2595 00000100:00100000:2.0:1787739054.025184:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be43c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665549056:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739054.025384:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be43c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665549056:12345-0@lo:1000:osp_up1-0.0 Request processed in 200us (263us total) trans 4294968301 rc 0/0 00000100:00100000:2.0:1787739054.025391:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2595 00000100:00100000:2.0:1787739054.025446:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665549056:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739054.025472:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665548416:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739054.025486:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665548544:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1787739054.025578:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665548672 00000100:00100000:1.0:1787739054.025582:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665548672 00000100:00100000:1.0:1787739054.025612:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1188 00000100:00100000:1.0:1787739054.025619:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+8:3650:x1874579665548672:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:1.0:1787739054.025628:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739054.025645:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ab8a4 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x2153044 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab89d expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.025657:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x2153044 (ffffa074331c2f00) 00010000:00010000:1.0:1787739054.025664:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739054.025694:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+7:3650:x1874579665548672:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 76us (686us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.025701:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1188 00010000:00010000:1.0:1787739054.029543:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab82d lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x14f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.029559:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab82d lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x14f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.029567:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab82d lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x14f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.029577:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633cc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739054.029581:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab82d lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x14f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.029588:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab82d lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x14f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.029602:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f7:0x0].0x0 (ffffa07303c78800) 00010000:00010000:1.0:1787739054.029605:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739054.029610:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab82d lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x14f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.029615:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ab82d lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x14f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.029636:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab834 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.029643:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab834 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.029648:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab834 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.029653:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739054.029655:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab834 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.029660:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab834 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.029670:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb7:0x0].0x0 (ffffa07303c78500) 00010000:00010000:1.0:1787739054.029672:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739054.029674:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab834 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.029679:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ab834 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739054.029921:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665548672:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:1.0:1787739054.039114:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739054.039187:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652420096 00000100:00100000:2.0:1787739054.039196:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652420096 00000100:00100000:2.0:1787739054.039254:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23191 00000100:00100000:2.0:1787739054.039264:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f72940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17442:x1874579652420096:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739054.039299:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739054.039319:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab94c lrc: 2/0,0 mode: --/CR res: [0x240000402:0x15f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094b25 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.039384:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab953 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x15f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.039398:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab953 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.039433:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab953 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x15f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739054.039523:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x15f:0x0]: returning size 4096 00000004:00200000:2.0:1787739054.039527:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739054.039559:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab94c lrc: 0/0,0 mode: --/CR res: [0x240000402:0x15f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b25 expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.039577:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab953 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x15f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b25 expref: 10 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.039586:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15f:0x0].0x0 (ffffa07433235900) 00010000:00010000:2.0:1787739054.039589:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739054.039592:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c400, rc 0) 00000100:00100000:2.0:1787739054.039647:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f72940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17442:x1874579652420096:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 385us (542us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739054.039656:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23191 00000100:00100000:1.0:1787739054.055835:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739054.055927:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652420224 00000100:00100000:3.0:1787739054.055937:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652420224 00000100:00100000:3.0:1787739054.055995:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23192 00000100:00100000:3.0:1787739054.056006:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f72d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17442:x1874579652420224:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739054.056026:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739054.056046:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab95a lrc: 2/0,0 mode: --/CR res: [0x240000400:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094b2c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.056114:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab961 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.056128:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab961 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.056138:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab961 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739054.056189:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xb5:0x0]: returning size 4096 00000004:00200000:3.0:1787739054.056194:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739054.056228:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1ab95a lrc: 0/0,0 mode: --/CR res: [0x240000400:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b2c expref: 12 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.056246:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab961 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b2c expref: 11 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.056254:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb5:0x0].0x0 (ffffa07433235800) 00010000:00010000:3.0:1787739054.056257:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739054.056261:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739054.056328:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f72d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17442:x1874579652420224:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 324us (503us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.056335:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23192 00000100:00100000:3.0:1787739054.059748:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739054.059858:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652420352 00000100:00100000:1.0:1787739054.059868:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652420352 00000100:00100000:1.0:1787739054.059923:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23193 00000100:00100000:1.0:1787739054.059933:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17442:x1874579652420352:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739054.059954:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739054.059972:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab968 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094b33 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.060030:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab96f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.060045:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab929 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab922 expref: 5 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.060053:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab96f lrc: 4/1,0 mode: --/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.060096:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab929 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ab922 expref: 5 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739054.060116:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f721c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665549184:0@lo:104: 00000100:00100000:1.0:1787739054.060165:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739054.060211:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89300 going to sleep for 11 seconds 00000100:00100000:3.0:1787739054.060283:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665549184 00000100:00100000:3.0:1787739054.060288:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665549184 00000100:00100000:3.0:1787739054.060320:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1061 00000100:00100000:3.0:1787739054.060327:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f703c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665549184:12345-0@lo:104: 00010000:00010000:3.0:1787739054.060339:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ab922 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab929 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.060393:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739054.060398:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f703c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665549184:12345-0@lo:104: Request processed in 72us (233us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.060404:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1061 00010000:00010000:1.0:1787739054.060656:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739054.060664:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ab922 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ab929 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.060672:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739054.060677:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ab922 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ab929 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739054.060686:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ab922 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ab929 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.060711:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ab922 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab929 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.060718:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739054.060756:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ab922 lrc: 1/0,0 mode: --/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab929 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.060763:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ab922 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ab929 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.060780:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739054.060782:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739054.060784:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739054.060810:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f721c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665549184:0@lo:104: 00010000:00010000:1.0:1787739054.060828:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab96f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739054.060878:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787739054.060891:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f712c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665549312:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739054.060926:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739054.061056:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665549312 00000100:00100000:1.0:1787739054.061061:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665549312 00010000:00010000:1.0:1787739054.061089:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab929 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ab922 expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739054.061115:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1389 00000100:00100000:1.0:1787739054.061121:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be7c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+6:3650:x1874579665549312:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739054.061131:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00080000:2.0:1787739054.065051:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 346 batchid updated 00000020:00080000:3.0:1787739054.065100:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 346 00000004:00080000:3.0:1787739054.065158:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07307054cc0 with version 658 00000020:00080000:3.0:1787739054.065180:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 346 cancel update log [0x240000401:0x3:0x0].343: rc = 0 00000020:00080000:3.0:1787739054.065250:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 346 cancel update log [0x200000400:0x3:0x0].344: rc = 0 00000020:00080000:3.0:1787739054.065259:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 346 00000004:00080000:1.0:1787739054.065362:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 658 rpc_version 658 00000020:00100000:1.0:1787739054.065395:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739054.065406:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303f721c0 x1874579665549568/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739054.065455:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f721c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665549568:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739054.065513:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739054.065542:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89300 going to sleep for 11 seconds 00000100:00100000:3.0:1787739054.065595:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665549568 00000100:00100000:3.0:1787739054.065601:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665549568 00000100:00100000:3.0:1787739054.065649:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2596 00000100:00100000:3.0:1787739054.065656:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f730c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665549568:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739054.065777:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f730c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665549568:12345-0@lo:1000:osp_up1-0.0 Request processed in 121us (264us total) trans 4294968302 rc 0/0 00000100:00100000:3.0:1787739054.065784:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2596 00010000:00010000:3.0:1787739054.067463:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ab929 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ab922 expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.067486:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb5:0x0].0x0 (ffffa073068a9f00) 00010000:00010000:3.0:1787739054.067491:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab96f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.067501:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab96f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.067508:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab96f lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.067540:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739054.067612:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be7c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+5:3650:x1874579665549312:12345-0@lo:103:ldlm_bl.0 Request processed in 6490us (6684us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.067620:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1389 00000100:00100000:3.0:1787739054.067677:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f712c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665549312:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739054.067797:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f721c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665549568:0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1787739054.068481:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab96f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.068492:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab96f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739054.068550:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb5:0x0]: returning size 4096 00000004:00200000:2.0:1787739054.068556:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739054.068602:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab968 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b33 expref: 16 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.068619:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab96f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b33 expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.068626:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb5:0x0].0x0 (ffffa073068a9f00) 00010000:00010000:2.0:1787739054.068629:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739054.068632:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c800, rc 0) 00000100:00100000:2.0:1787739054.068687:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17442:x1874579652420352:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 8755us (8959us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739054.068695:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23193 00000100:00100000:1.0:1787739054.080872:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739054.080916:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739054.080987:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652420736 00000100:00100000:0.0:1787739054.080997:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652420736 00000100:00100000:0.0:1787739054.081062:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23194 00000100:00100000:0.0:1787739054.081072:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f73480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17442:x1874579652420736:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:0.0:1787739054.081136:0:13793:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739054.081163:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ab953 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x15f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b25 expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.081184:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15f:0x0].0x0 (ffffa07433235900) 00010000:00010000:0.0:1787739054.081189:0:13793:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739054.081236:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ab976 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.081245:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ab976 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.081261:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ab976 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.081291:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d85a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665549696:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739054.081339:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.081363:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739054.081506:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652420480 00000100:00100000:3.0:1787739054.081511:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652420480 00000100:00100000:3.0:1787739054.081543:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1189 00000100:00100000:3.0:1787739054.081550:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f70780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2406:x1874579652420480:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:3.0:1787739054.081560:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739054.081572:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ab945 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x15f:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b1e expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.081584:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15f:0x0].0x0 (ffffa074331c2c00) 00010000:00010000:3.0:1787739054.081591:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739054.081630:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f70780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2406:x1874579652420480:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 80us (713us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.081637:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1189 00000100:00100000:1.0:1787739054.082380:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665549696 00000100:00100000:1.0:1787739054.082386:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665549696 00000100:00100000:1.0:1787739054.082449:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2597 00000100:00100000:1.0:1787739054.082457:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+5:13793:x1874579665549696:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739054.082470:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739054.082487:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab97d lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab976 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.082510:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab97d lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab976 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.082522:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab97d lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab976 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.082529:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235200) 00010000:00010000:1.0:1787739054.082532:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739054.082535:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d600, rc 0) 00000100:00100000:1.0:1787739054.082583:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:13793:x1874579665549696:12345-0@lo:101:mdt00_004.0 Request processed in 127us (1244us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.082589:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2597 00000100:00100000:0.0:1787739054.083510:0:13793:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983312 gen 2 00000004:00080000:0.0:1787739054.083518:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983307 committed transno 4294983312 00000020:00080000:0.0:1787739054.083524:0:13793:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07305fcc300 refcount 2 committed 0 result 0 batchid 337 00000020:00080000:0.0:1787739054.083528:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730424db00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07305fcc540 00000020:00080000:0.0:1787739054.083531:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].330 00000020:00080000:0.0:1787739054.083533:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730424d500 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306063400 00000020:00080000:0.0:1787739054.083536:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].330 00000004:00080000:0.0:1787739054.083570:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983309 committed transno 4294983312 00000004:00080000:0.0:1787739054.083581:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983310 committed transno 4294983312 00000004:00080000:0.0:1787739054.083593:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983312 committed transno 4294983312 00000100:00100000:0.0:1787739054.083610:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d85a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665549696:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739054.083628:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ab976 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab97d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.083640:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ab976 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab97d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739054.083656:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739054.083689:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1ab984 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.083696:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1ab984 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.083705:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1ab984 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.083721:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d85a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665549824:0@lo:101:mdt00_004.0 00000020:00080000:3.0:1787739054.083797:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 337 committed batchid 336 00000020:00080000:3.0:1787739054.083864:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 336 00000100:00100000:0.0:1787739054.084087:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.084114:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739054.084554:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665549824 00000100:00100000:3.0:1787739054.084561:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665549824 00000100:00100000:3.0:1787739054.084607:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2598 00000100:00100000:3.0:1787739054.084616:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d85680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:13793:x1874579665549824:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739054.084631:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739054.084647:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab98b lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab984 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.084669:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab98b lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab984 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.084683:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab98b lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab984 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.084690:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x836d1894 (ffffa074331c2e00) 00010000:00010000:3.0:1787739054.084693:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739054.084696:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa600, rc 0) 00000100:00100000:3.0:1787739054.084976:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d85680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665549824:12345-0@lo:101:mdt00_004.0 Request processed in 360us (897us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.084983:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2598 00000100:00100000:0.0:1787739054.085233:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d85a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665549824:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739054.085248:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1ab984 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab98b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.085258:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1ab984 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab98b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739054.085349:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:0.0:1787739054.085389:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d85a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665549952:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739054.085453:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.085476:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc540 going to sleep for 11 seconds 00000100:00100000:2.0:1787739054.085828:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665549952 00000100:00100000:2.0:1787739054.085834:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665549952 00000100:00100000:2.0:1787739054.086067:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2599 00000100:00100000:2.0:1787739054.086076:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d852c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665549952:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739054.086167:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d852c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665549952:12345-0@lo:1000:mdt00_004.0 Request processed in 93us (713us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739054.086173:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2599 00000100:00100000:0.0:1787739054.086440:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d85a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665549952:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739054.086489:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d85a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665550080:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739054.086515:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.086533:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc540 going to sleep for 11 seconds 00000100:00100000:2.0:1787739054.086961:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665550080 00000100:00100000:2.0:1787739054.086967:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665550080 00000100:00100000:2.0:1787739054.087004:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2600 00000100:00100000:2.0:1787739054.087012:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665550080:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739054.087117:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665550080:12345-0@lo:1000:mdt00_004.0 Request processed in 106us (600us total) trans 0 rc 1/1 00000100:00100000:2.0:1787739054.087124:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2600 00000100:00100000:0.0:1787739054.087582:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d85a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665550080:0@lo:1000:mdt00_004.0 00010000:00010000:0.0:1787739054.087645:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab992 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.087654:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab992 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.087664:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab992 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.087678:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d85a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665550208:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739054.087709:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.087739:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739054.088000:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665550208 00000100:00100000:2.0:1787739054.088006:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665550208 00000100:00100000:2.0:1787739054.088072:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2601 00000100:00100000:2.0:1787739054.088089:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d87840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665550208:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739054.088100:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739054.088117:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab999 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab992 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.088142:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab999 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab992 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.088162:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab999 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab992 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.088170:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15f:0x0].0x0 (ffffa074419d5000) 00010000:00010000:2.0:1787739054.088173:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739054.088176:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c000, rc 0) 00000100:00100000:2.0:1787739054.088218:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d87840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665550208:12345-0@lo:101:mdt00_004.0 Request processed in 138us (507us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739054.088224:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2601 00000100:00100000:0.0:1787739054.088290:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d85a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665550208:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739054.088302:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab992 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab999 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.088311:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab992 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab999 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.088343:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab9a0 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.088353:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab9a0 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.088361:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab9a0 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.088375:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab9a7 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.088386:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab961 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b2c expref: 10 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.088394:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab9a7 lrc: 4/0,1 mode: --/EX res: [0x240000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.088435:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab961 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b2c expref: 10 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.088449:0:13793:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab961 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b2c expref: 10 pid: 6524 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.088460:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d85a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665550336:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739054.088517:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739054.094970:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739054.095002:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652420864 00000100:00100000:1.0:1787739054.095011:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652420864 00010000:00010000:1.0:1787739054.095050:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab961 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b2c expref: 11 pid: 6524 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739054.095094:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1390 00000100:00100000:1.0:1787739054.095105:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f71a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2409:x1874579652420864:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739054.095120:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739054.095127:0:6510:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab961 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b2c expref: 11 pid: 6524 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.095140:0:6510:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab961 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b2c expref: 11 pid: 6524 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.095161:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb5:0x0].0x0 (ffffa07433235800) 00010000:00010000:1.0:1787739054.095166:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab9a7 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.095175:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab9a7 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.095182:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab9a7 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.095198:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739054.095241:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f71a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2409:x1874579652420864:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 138us (278us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.095248:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1390 00010000:00010000:2.0:1787739054.096913:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ab961 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b2c expref: 10 pid: 6524 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739054.096936:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d85a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665550336:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739054.096959:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab9a7 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.096969:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab9a7 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.096977:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab9a7 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.097018:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1ab9ae lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.097025:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1ab9ae lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.097040:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1ab9ae lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739054.097058:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be6580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665550464:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1787739054.097131:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739054.097157:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739054.097262:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665550464 00000100:00100000:3.0:1787739054.097340:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665550464 00000100:00100000:3.0:1787739054.097394:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2602 00000100:00100000:3.0:1787739054.097402:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665550464:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739054.097439:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739054.097454:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab9b5 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ab9ae expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.097483:0:8447:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab96f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b33 expref: 13 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.097491:0:8447:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab9b5 lrc: 4/0,0 mode: --/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab9ae expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.097511:0:8447:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab96f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b33 expref: 13 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.097523:0:8447:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab96f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b33 expref: 13 pid: 7944 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739054.097533:0:8447:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f5680 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8447:1874579665550592:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739054.097580:0:8447:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054000 going to sleep for 11 seconds 00000100:00100000:3.0:1787739054.110667:0:8447:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f5680 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8447:1874579665550592:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739054.110704:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab9b5 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab9ae expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.110718:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb5:0x0].0x0 (ffffa073068a9f00) 00010000:00010000:3.0:1787739054.110721:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739054.110726:0:8447:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab9b5 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab9ae expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.110749:0:8447:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab9b5 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab9ae expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.110762:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abc00, rc 0) 00000100:00100000:3.0:1787739054.110827:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665550464:12345-0@lo:101:mdt00_004.0 Request processed in 13425us (13695us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.110837:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2602 00000100:00100000:2.0:1787739054.110969:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be6580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665550464:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739054.110989:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1ab9ae lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab9b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.110999:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1ab9ae lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab9b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739054.112168:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739054.112367:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652420992 00000100:00100000:0.0:1787739054.112374:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652420992 00010000:00010000:0.0:1787739054.112406:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab96f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b33 expref: 14 pid: 7944 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.112462:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1391 00000100:00100000:0.0:1787739054.112471:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2408:x1874579652420992:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739054.112487:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739054.112493:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab96f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b33 expref: 14 pid: 7944 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.112504:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab96f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b33 expref: 14 pid: 7944 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.112524:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ab96f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b33 expref: 14 pid: 7944 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.112540:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb5:0x0].0x0 (ffffa073068a9f00) 00010000:00010000:0.0:1787739054.112546:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab9b5 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab9ae expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.112555:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab9b5 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab9ae expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.112562:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab9b5 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ab9ae expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.112591:0:6508:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab9b5 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ab9ae expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.112604:0:6508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d85a40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665550720:0@lo:105: 00000100:00100000:0.0:1787739054.112646:0:6508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.112679:0:6508:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc540 going to sleep for 11 seconds 00000100:00100000:2.0:1787739054.112755:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665550720 00000100:00100000:2.0:1787739054.112760:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665550720 00000100:00100000:2.0:1787739054.112785:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1062 00000100:00100000:2.0:1787739054.112791:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d86580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665550720:12345-0@lo:105: 00010000:00010000:2.0:1787739054.112801:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1ab9ae lrc: 5/0,1 mode: --/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab9b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.112810:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1ab9ae lrc: 5/0,1 mode: --/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab9b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.112818:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1ab9ae lrc: 5/0,1 mode: --/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab9b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.112825:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1ab9ae lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab9b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.112831:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1ab9ae lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ab9b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.112839:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1ab9ae lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ab9b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.112861:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633d800) 00000100:00100000:2.0:1787739054.112895:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d86580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665550720:12345-0@lo:105: Request processed in 105us (249us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739054.112902:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1062 00010000:00010000:2.0:1787739054.112924:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1ab9ae lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab9b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.112931:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1ab9ae lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab9b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739054.113562:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be6580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665550848:0@lo:403:mdt00_004.0 00000100:00100000:2.0:1787739054.113607:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739054.113626:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739054.113659:0:6508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d85a40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665550720:0@lo:105: 00010000:00010000:2.0:1787739054.113672:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739054.113717:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:2408:x1874579652420992:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 1245us (1552us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739054.113725:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1391 00000100:00100000:3.0:1787739054.114152:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665550848 00000100:00100000:3.0:1787739054.114158:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665550848 00000100:00100000:3.0:1787739054.114200:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2603 00000100:00100000:3.0:1787739054.114208:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665550848:12345-0@lo:403:mdt00_004.0 00000100:00100000:3.0:1787739054.115982:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665550848:12345-0@lo:403:mdt00_004.0 Request processed in 1767us (2366us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.115994:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2603 00000100:00100000:0.0:1787739054.116296:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be6580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665550848:0@lo:403:mdt00_004.0 00000040:00080000:0.0:1787739054.117200:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].332, 664 off505968 00000040:00080000:0.0:1787739054.117364:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].332, 744 off552584 00000004:00080000:0.0:1787739054.117370:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07305fcc180 with version 633 00000004:00080000:2.0:1787739054.119706:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 633 rpc_version 633 00000020:00100000:2.0:1787739054.119758:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739054.119769:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306be70c0 x1874579665551104/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2064/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739054.119790:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be70c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665551104:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739054.119862:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739054.119894:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539540 going to sleep for 11 seconds 00000100:00100000:0.0:1787739054.120055:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665551104 00000100:00100000:0.0:1787739054.120063:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665551104 00000100:00100000:0.0:1787739054.120128:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2604 00000100:00100000:0.0:1787739054.120137:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be6940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665551104:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739054.121355:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be6940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665551104:12345-0@lo:1000:osp_up0-1.0 Request processed in 1213us (1486us total) trans 4294983313 rc 0/0 00000100:00100000:0.0:1787739054.121367:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2604 00000100:00100000:2.0:1787739054.122823:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be70c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665551104:0@lo:1000:osp_up0-1.0 00000004:00200000:0.0:1787739054.123280:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x15f:0x0]: returning size 4096 00010000:00010000:0.0:1787739054.123304:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab9a7 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123314:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab9a7 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123321:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab9a7 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123328:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739054.123332:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab9a7 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123339:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab9a7 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123353:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb5:0x0].0x0 (ffffa07433235800) 00010000:00010000:0.0:1787739054.123355:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739054.123360:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab9a7 lrc: 2/0,0 mode: --/EX res: [0x240000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123366:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ab9a7 lrc: 0/0,0 mode: --/EX res: [0x240000400:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123377:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1ab9ae lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab9b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123384:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1ab9ae lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab9b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123391:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1ab9ae lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab9b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123408:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739054.123431:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xb5:0x0] 00010000:00010000:0.0:1787739054.123439:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab9a0 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123445:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab9a0 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123451:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab9a0 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123455:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739054.123458:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab9a0 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123464:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab9a0 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123474:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15f:0x0].0x0 (ffffa07433235900) 00010000:00010000:0.0:1787739054.123476:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739054.123479:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab9a0 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123484:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab9a0 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123493:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab992 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab999 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123499:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab992 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab999 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123506:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab992 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab999 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123520:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:0.0:1787739054.123529:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739054.123533:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ab976 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ab97d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123539:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ab976 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab97d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123546:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ab976 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ab97d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123556:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:0.0:1787739054.123559:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739054.123563:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1ab984 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab98b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123569:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1ab984 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ab98b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.123643:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f73480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17442:x1874579652420736:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 42570us (42776us total) trans 4294968303 rc 0/0 00000100:00100000:0.0:1787739054.123651:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23194 00010000:00010000:0.0:1787739054.123683:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1787739054.123688:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1ab9ae lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab9b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123695:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739054.123699:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1ab9ae lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab9b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739054.123708:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1ab9ae lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab9b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123750:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1ab9ae lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab9b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123757:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739054.123777:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1ab9ae lrc: 1/0,0 mode: --/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab9b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123783:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1ab9ae lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab9b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123801:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787739054.123804:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab992 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab999 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123809:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739054.123812:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab992 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab999 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739054.123818:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab992 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab999 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123834:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab992 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab999 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123840:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739054.123845:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab992 lrc: 1/0,0 mode: --/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab999 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123864:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1ab992 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab999 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123875:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739054.123878:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ab976 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ab97d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123883:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739054.123886:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ab976 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab97d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739054.123892:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ab976 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab97d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123906:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ab976 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab97d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123912:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739054.123928:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ab976 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab97d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123935:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ab976 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab97d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.123945:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739054.123947:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739054.123949:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739054.124148:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739054.124163:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d870c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665551232:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739054.124218:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739054.124259:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787739054.124265:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d87480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665551360:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739054.124286:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787739054.124319:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739054.124510:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665551232 00000100:00100000:1.0:1787739054.124518:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665551232 00000100:00100000:1.0:1787739054.124556:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665551360 00000100:00100000:1.0:1787739054.124559:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665551360 00000100:00100000:1.0:1787739054.124579:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1190 00000100:00100000:1.0:1787739054.124588:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba2050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+11:3652:x1874579665551232:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739054.124600:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739054.124618:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ab9b5 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab9ae expref: 11 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.124631:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb5:0x0].0x0 (ffffa073068a9f00) 00010000:00010000:1.0:1787739054.124638:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739054.124674:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba2050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+10:3652:x1874579665551232:12345-0@lo:103:ldlm_bl.0 Request processed in 87us (455us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.124683:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1190 00000100:00100000:1.0:1787739054.124727:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1191 00000100:00100000:1.0:1787739054.124733:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be7840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+9:3652:x1874579665551360:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739054.124740:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739054.124750:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ab999 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x15f:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab992 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.124761:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15f:0x0].0x0 (ffffa074419d5000) 00010000:00010000:1.0:1787739054.124767:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739054.124790:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be7840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+8:3652:x1874579665551360:12345-0@lo:103:ldlm_bl.0 Request processed in 57us (502us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.124795:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1191 00000100:00100000:1.0:1787739054.124826:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d87480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665551360:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739054.124833:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d870c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665551232:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739054.128499:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787739054.128525:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d85e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665551488:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739054.128592:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739054.129178:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665551488 00000100:00100000:1.0:1787739054.129188:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665551488 00000100:00100000:1.0:1787739054.129243:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1192 00000100:00100000:1.0:1787739054.129252:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d86940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+8:3651:x1874579665551488:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739054.129266:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739054.129290:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab97d lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab976 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.129306:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235200) 00010000:00010000:1.0:1787739054.129315:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739054.129368:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d86940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+7:3651:x1874579665551488:12345-0@lo:103:ldlm_bl.0 Request processed in 117us (775us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.129376:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1192 00000100:00100000:0.0:1787739054.130520:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d85e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665551488:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739054.157316:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739054.159397:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652421120 00000100:00100000:0.0:1787739054.159432:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652421120 00000100:00100000:0.0:1787739054.159501:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23195 00000100:00100000:0.0:1787739054.159511:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17443:x1874579652421120:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739054.159530:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739054.159551:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab9bc lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094b3a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.159615:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab9c3 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.159630:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab9c3 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.159640:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab9c3 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.159652:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab9ca lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.159658:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab9ca lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.159664:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab9ca lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.159708:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab9c3 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.159715:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab9c3 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.159721:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab9c3 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.159728:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab9c3 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.159734:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739054.159738:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab9c3 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.159745:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab9c3 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.159756:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9800) 00010000:00010000:0.0:1787739054.159759:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739054.159763:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab9c3 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.159769:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ab9c3 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.159783:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab9ca lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.159789:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab9ca lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.159794:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab9ca lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.159799:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739054.159803:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab9ca lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.159808:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab9ca lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.159816:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x793368a3 (ffffa073068a9000) 00010000:00010000:0.0:1787739054.159818:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739054.159821:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab9ca lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.159827:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ab9ca lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.159856:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab9bc lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b3a expref: 14 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.159866:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9800) 00010000:00010000:0.0:1787739054.159868:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739054.159871:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ab9bc lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b3a expref: 14 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.159881:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a400, rc 0) 00000100:00100000:0.0:1787739054.160021:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17443:x1874579652421120:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 509us (2712us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739054.160029:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23195 00000100:00100000:0.0:1787739054.165264:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739054.165347:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652421248 00000100:00100000:1.0:1787739054.165356:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652421248 00000100:00100000:1.0:1787739054.165442:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23196 00000100:00100000:1.0:1787739054.165452:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d85e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17443:x1874579652421248:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739054.165469:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739054.165489:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab9d1 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094b41 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.165546:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab9d8 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.165560:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab9d8 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.165570:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab9d8 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.165582:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab9df lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.165588:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab9df lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.165593:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab9df lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.165628:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab9d8 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.165634:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab9d8 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.165640:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab9d8 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.165646:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab9d8 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.165652:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ca00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739054.165656:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab9d8 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.165663:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab9d8 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.165674:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235200) 00010000:00010000:1.0:1787739054.165677:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739054.165681:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab9d8 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.165686:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ab9d8 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.165698:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab9df lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.165703:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab9df lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.165708:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab9df lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.165713:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739054.165716:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab9df lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.165721:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab9df lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.165737:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x793368a3 (ffffa07433235600) 00010000:00010000:1.0:1787739054.165739:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739054.165742:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab9df lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.165746:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ab9df lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.165774:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab9d1 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b41 expref: 14 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.165783:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235200) 00010000:00010000:1.0:1787739054.165785:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739054.165788:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ab9d1 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b41 expref: 14 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.165799:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d600, rc 0) 00000100:00100000:1.0:1787739054.165853:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d85e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17443:x1874579652421248:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 402us (598us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739054.165860:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23196 00000100:00100000:3.0:1787739054.174607:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739054.174712:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652421376 00000100:00100000:1.0:1787739054.174734:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652421376 00000100:00100000:1.0:1787739054.174791:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23197 00000100:00100000:1.0:1787739054.174802:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f43c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17443:x1874579652421376:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739054.174823:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739054.174847:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab9e6 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094b48 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.174903:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ab9ed lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.174917:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ab9ed lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.174927:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ab9ed lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739054.174975:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:1.0:1787739054.174981:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739054.175020:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ab9e6 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b48 expref: 15 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.175038:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ab9ed lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b48 expref: 14 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.175047:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235000) 00010000:00010000:1.0:1787739054.175050:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739054.175054:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1ce00, rc 0) 00000100:00100000:1.0:1787739054.175239:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f43c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17443:x1874579652421376:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 438us (641us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.175248:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23197 00000100:00100000:0.0:1787739054.199892:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739054.199974:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652421632 00000100:00100000:2.0:1787739054.199983:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652421632 00000100:00100000:2.0:1787739054.200044:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23198 00000100:00100000:2.0:1787739054.200054:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17443:x1874579652421632:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:2.0:1787739054.200126:0:6523:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739054.200152:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ab9ed lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b48 expref: 14 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.200169:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235000) 00010000:00010000:2.0:1787739054.200178:0:6523:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739054.200234:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab9f4 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.200248:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab9f4 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.200257:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab9f4 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.200269:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab9fb lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x793368a3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.200276:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab9fb lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.200283:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab9fb lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1787739054.201103:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].346, 2168 off532120 00000040:00080000:2.0:1787739054.201169:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].345, 2368 off575272 00000004:00080000:2.0:1787739054.201176:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07435539540 with version 659 00000004:00080000:0.0:1787739054.201341:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 659 rpc_version 659 00000020:00100000:0.0:1787739054.201374:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739054.201383:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304d870c0 x1874579665551744/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 4232/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739054.201405:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d870c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665551744:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739054.201485:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.201528:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739054.201589:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665551744 00000100:00100000:2.0:1787739054.201595:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665551744 00000100:00100000:2.0:1787739054.201747:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2605 00000100:00100000:2.0:1787739054.201755:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665551744:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739054.202385:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665551744:12345-0@lo:1000:osp_up1-0.0 Request processed in 628us (896us total) trans 4294968304 rc 0/0 00000100:00100000:2.0:1787739054.202395:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2605 00000100:00100000:3.0:1787739054.202529:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d870c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665551744:0@lo:1000:osp_up1-0.0 00010000:00010000:3.0:1787739054.207691:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aba02 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x14f9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.207710:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aba02 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14f9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.207733:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aba02 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14f9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.207753:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aba09 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.207765:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aba09 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.207772:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aba09 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.207807:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1aba10 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.207813:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1aba10 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.207823:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1aba10 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739054.207846:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665551872:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739054.207907:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739054.209869:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054000 going to sleep for 11 seconds 00000100:00100000:0.0:1787739054.215879:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665551872 00000100:00100000:0.0:1787739054.215895:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665551872 00000100:00100000:0.0:1787739054.215977:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2606 00000100:00100000:0.0:1787739054.215987:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f5a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6523:x1874579665551872:12345-0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739054.216011:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739054.216033:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1aba17 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aba10 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.216063:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1aba17 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aba10 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.216151:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1aba17 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aba10 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.216161:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb8:0x0].0x0 (ffffa073068a9800) 00010000:00010000:0.0:1787739054.216164:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739054.216168:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a400, rc 0) 00000100:00100000:0.0:1787739054.216248:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f5a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6523:x1874579665551872:12345-0@lo:101:mdt00_000.0 Request processed in 263us (8341us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739054.216255:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2606 00000100:00100000:0.0:1787739054.216327:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665551872:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739054.216344:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1aba10 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aba17 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.216355:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1aba10 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aba17 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739054.216378:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000bcf06b67 reply state 000000002b60206e transno 4294983314 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aba09 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.216396:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aba09 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.216402:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb8:0x0].0x0 (ffffa074331c2700) 00010000:00010000:0.0:1787739054.216404:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739054.216407:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739054.216435:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1aba10 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aba17 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.216443:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1aba10 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aba17 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739054.216455:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xb8:0x0] 00000004:00010000:0.0:1787739054.216463:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000bcf06b67 reply state 000000002b60206e transno 4294983314 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aba02 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14f9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.216471:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aba02 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x14f9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.216477:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f9:0x0].0x0 (ffffa074331c2c00) 00010000:00010000:0.0:1787739054.216479:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739054.216481:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787739054.216486:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14f9:0x0]: returning size 4096 00010000:00010000:0.0:1787739054.216497:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab9f4 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.216503:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab9f4 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.216508:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab9f4 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.216516:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab9f4 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.216521:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633ce00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739054.216525:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab9f4 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.216533:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab9f4 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.216543:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5900) 00010000:00010000:0.0:1787739054.216546:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739054.216549:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab9f4 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.216555:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ab9f4 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739054.216574:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000bcf06b67 reply state 000000002b60206e transno 4294983314 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab9fb lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.216582:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab9fb lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.216588:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x793368a3 (ffffa074419d5300) 00010000:00010000:0.0:1787739054.216591:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739054.216593:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 2 00000100:00100000:0.0:1787739054.216659:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17443:x1874579652421632:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 16605us (16774us total) trans 4294983314 rc 0/0 00000100:00100000:0.0:1787739054.216667:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23198 00000100:00100000:2.0:1787739054.238622:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739054.238726:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652421888 00000100:00100000:3.0:1787739054.238736:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652421888 00000100:00100000:3.0:1787739054.238800:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23199 00000100:00100000:3.0:1787739054.238812:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17444:x1874579652421888:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:3.0:1787739054.239008:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17444:x1874579652421888:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 198us (395us total) trans 0 rc -17/-17 00000100:00100000:3.0:1787739054.239015:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23199 00000100:00100000:3.0:1787739054.255990:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739054.256444:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652422016 00000100:00100000:1.0:1787739054.256453:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652422016 00000100:00100000:1.0:1787739054.256515:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23200 00000100:00100000:1.0:1787739054.256525:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17445:x1874579652422016:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739054.256547:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739054.256573:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1aba1e lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094b4f expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.256639:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1aba25 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.256655:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1aba25 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.256667:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1aba25 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.256678:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba2c lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.256686:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba2c lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.256691:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba2c lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.256765:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1aba33 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14f9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.256774:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1aba33 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14f9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.256782:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1aba33 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14f9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739054.256864:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14f9:0x0]: returning size 4096 00000004:00200000:1.0:1787739054.256869:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:1.0:1787739054.256887:0:6525:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1aba33 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14f9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.256899:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1aba25 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.256906:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1aba25 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.256912:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1aba25 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.256919:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1aba25 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.256925:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739054.256929:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1aba25 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.256936:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1aba25 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.256946:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235200) 00010000:00010000:1.0:1787739054.256949:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739054.256953:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1aba25 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.256958:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1aba25 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.256976:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba2c lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.256982:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba2c lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.256987:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba2c lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.256993:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739054.256995:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba2c lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.257000:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba2c lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.257007:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x793368a3 (ffffa074419d5300) 00010000:00010000:1.0:1787739054.257009:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739054.257012:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba2c lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.257017:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba2c lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.257041:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1aba1e lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b4f expref: 17 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.257058:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1aba33 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14f9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b4f expref: 16 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.257087:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f9:0x0].0x0 (ffffa074331c2c00) 00010000:00010000:1.0:1787739054.257090:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787739054.257093:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1ca00, rc 0) 00000100:00100000:1.0:1787739054.257158:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17445:x1874579652422016:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 634us (1177us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.257166:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23200 00000100:00100000:3.0:1787739054.265207:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739054.265335:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652422144 00000100:00100000:0.0:1787739054.265344:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652422144 00000100:00100000:0.0:1787739054.265402:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23201 00000100:00100000:0.0:1787739054.265443:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f5e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17445:x1874579652422144:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739054.265466:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739054.265489:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1aba3a lrc: 2/0,0 mode: --/CR res: [0x200000400:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094b56 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.265548:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aba41 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.265562:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aba41 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.265573:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aba41 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739054.265614:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xb8:0x0]: returning size 4096 00000004:00200000:0.0:1787739054.265620:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739054.265655:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1aba3a lrc: 0/0,0 mode: --/CR res: [0x200000400:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b56 expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.265671:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aba41 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b56 expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.265680:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb8:0x0].0x0 (ffffa074331c2700) 00010000:00010000:0.0:1787739054.265684:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739054.265687:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609bc00, rc 0) 00000100:00100000:0.0:1787739054.265747:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f5e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17445:x1874579652422144:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 336us (547us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739054.265758:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23201 00000100:00100000:0.0:1787739054.272589:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739054.272657:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652422272 00000100:00100000:1.0:1787739054.272667:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652422272 00000100:00100000:1.0:1787739054.272716:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23202 00000100:00100000:1.0:1787739054.272727:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d85a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17445:x1874579652422272:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739054.272750:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739054.272770:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1aba48 lrc: 2/0,0 mode: --/CR res: [0x240000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094b5d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.272828:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba4f lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.272842:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1aba17 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aba10 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.272851:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba4f lrc: 4/1,0 mode: --/PR res: [0x240000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.272882:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1aba17 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1aba10 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739054.272897:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f70780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665552000:0@lo:104: 00000100:00100000:1.0:1787739054.272949:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739054.273002:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739054.273062:0:15907:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787739054.273088:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665552000 00000100:00100000:0.0:1787739054.273093:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665552000 00000100:00100000:0.0:1787739054.273135:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1063 00000100:00100000:0.0:1787739054.273142:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f4f050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:13793:x1874579665552000:12345-0@lo:104: 00010000:00010000:0.0:1787739054.273155:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1aba10 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aba17 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.273199:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787739054.273205:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f4f050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:13793:x1874579665552000:12345-0@lo:104: Request processed in 65us (257us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739054.273212:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1063 00010000:00010000:1.0:1787739054.273347:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739054.273354:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1aba10 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aba17 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.273363:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739054.273368:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1aba10 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xb8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1aba17 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739054.273376:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1aba10 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xb8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1aba17 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.273399:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1aba10 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xb8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aba17 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.273406:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739054.273452:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1aba10 lrc: 1/0,0 mode: --/PW res: [0x240000401:0xb8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aba17 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.273459:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1aba10 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xb8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aba17 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.273475:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739054.273476:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739054.273478:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739054.273500:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f70780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665552000:0@lo:104: 00010000:00010000:1.0:1787739054.273514:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba4f lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.273798:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f71a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665552128:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739054.273827:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739054.273892:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665552128 00000100:00100000:1.0:1787739054.273896:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665552128 00010000:00010000:1.0:1787739054.273913:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1aba17 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1aba10 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739054.273950:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1392 00000100:00100000:1.0:1787739054.273956:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d843c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+10:3651:x1874579665552128:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739054.273965:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739054.280375:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1ab984 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ab98b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739054.280396:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1ab984 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ab98b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.280461:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1ab984 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab98b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.280469:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739054.280508:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1ab984 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ab98b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787739054.280526:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968304 is committed 00000001:00080000:3.0:1787739054.280542:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968303 is committed 00000020:00080000:3.0:1787739054.280551:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 337 batchid updated 00000020:00080000:3.0:1787739054.280567:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa0730610e300 refcount 2 committed 0 result 0 batchid 338 00000020:00080000:3.0:1787739054.280570:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730415fc00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730610ea80 00000020:00080000:3.0:1787739054.280574:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].331 00000020:00080000:3.0:1787739054.280576:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730415f600 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304010400 00000020:00080000:3.0:1787739054.280579:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].331 00000100:00100000:3.0:1787739054.280600:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787739054.280621:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665552256:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:3.0:1787739054.280705:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787739054.280979:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab914 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x15f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.280989:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab914 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x15f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.281000:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab914 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x15f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.281007:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1cc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739054.281012:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab914 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x15f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.281018:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab914 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x15f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.281030:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x15f:0x0].0x0 (ffffa07433235900) 00010000:00010000:3.0:1787739054.281034:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739054.281038:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab914 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x15f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.281044:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ab914 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x15f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.281060:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab91b lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.281065:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab91b lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.281076:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab91b lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.281092:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1de00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739054.281094:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab91b lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.281103:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab91b lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.281111:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb5:0x0].0x0 (ffffa07433235800) 00010000:00010000:3.0:1787739054.281113:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739054.281116:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab91b lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.281122:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ab91b lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1787739054.281144:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 338 committed batchid 337 00000100:00100000:0.0:1787739054.281197:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665552256 00000100:00100000:0.0:1787739054.281206:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665552256 00000004:00080000:3.0:1787739054.281253:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07307054000 with version 634 00000100:00100000:0.0:1787739054.281253:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1193 00000100:00100000:0.0:1787739054.281262:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+7:3653:x1874579665552256:12345-0@lo:103:jbd2/dm-1-8.0 00000020:00080000:3.0:1787739054.281263:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 337 cancel update log [0x200000401:0x3:0x0].330: rc = 0 00010000:00010000:0.0:1787739054.281275:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739054.281291:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ab98b lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x836d1894 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ab984 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.281302:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x836d1894 (ffffa074331c2e00) 00010000:00010000:0.0:1787739054.281309:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000020:00080000:3.0:1787739054.281313:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 337 cancel update log [0x240000400:0x3:0x0].330: rc = 0 00000020:00080000:3.0:1787739054.281320:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 337 00000004:00080000:3.0:1787739054.281328:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 634 rpc_version 634 00000020:00100000:3.0:1787739054.281345:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739054.281349:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+6:3653:x1874579665552256:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 88us (644us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.281352:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074074f61c0 x1874579665552512/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739054.281356:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1193 00000100:00100000:3.0:1787739054.281368:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665552512:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739054.281402:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f52c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665552256:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:3.0:1787739054.281490:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739054.281518:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054180 going to sleep for 11 seconds 00000100:00100000:0.0:1787739054.282273:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665552512 00000100:00100000:0.0:1787739054.282280:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665552512 00000100:00100000:0.0:1787739054.282331:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2607 00000100:00100000:0.0:1787739054.282339:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f6d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:7467:x1874579665552512:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739054.282500:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f6d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665552512:12345-0@lo:1000:osp_up0-1.0 Request processed in 162us (1009us total) trans 4294983315 rc 0/0 00000100:00100000:0.0:1787739054.282507:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2607 00000100:00100000:0.0:1787739054.282551:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665552512:0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1787739054.283515:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1aba17 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1aba10 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.283535:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb8:0x0].0x0 (ffffa073068a9800) 00010000:00010000:0.0:1787739054.283541:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba4f lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.283550:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba4f lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.283558:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba4f lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.283584:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739054.283637:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d843c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665552128:12345-0@lo:103:ldlm_bl.0 Request processed in 9681us (9807us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739054.283650:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1392 00000100:00100000:0.0:1787739054.283695:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f71a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665552128:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739054.286467:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba4f lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.286483:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba4f lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739054.286543:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xb8:0x0]: returning size 4096 00000004:00200000:0.0:1787739054.286548:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739054.286600:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1aba48 lrc: 0/0,0 mode: --/CR res: [0x240000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b5d expref: 9 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.286618:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba4f lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b5d expref: 8 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.286627:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb8:0x0].0x0 (ffffa073068a9800) 00010000:00010000:0.0:1787739054.286629:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739054.286633:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c000, rc 0) 00000100:00100000:0.0:1787739054.286705:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d85a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17445:x1874579652422272:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 13979us (14123us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739054.286713:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23202 00000100:00100000:1.0:1787739054.298956:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739054.299037:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652422528 00000100:00100000:2.0:1787739054.299047:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652422528 00000100:00100000:2.0:1787739054.299178:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23203 00000100:00100000:2.0:1787739054.299190:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f70780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17445:x1874579652422528:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:2.0:1787739054.299246:0:6525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739054.299274:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1aba33 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14f9:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b4f expref: 17 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.299294:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f9:0x0].0x0 (ffffa074331c2c00) 00010000:00010000:2.0:1787739054.299299:0:6525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739054.299325:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1aba56 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.299339:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1aba56 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.299348:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1aba56 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.299359:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aba5d lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x793368a3 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.299367:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aba5d lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.299374:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aba5d lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.299484:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aba64 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14f9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.299493:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aba64 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14f9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.299500:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aba64 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14f9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.299517:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aba6b lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.299528:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aba41 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b56 expref: 16 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.299535:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aba6b lrc: 4/0,1 mode: --/EX res: [0x200000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.299567:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aba41 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b56 expref: 16 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.299581:0:6525:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aba41 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b56 expref: 16 pid: 6523 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739054.299594:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be6d00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6525:1874579665552640:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739054.299671:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ecc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739054.308837:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be6d00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6525:1874579665552640:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739054.308870:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aba6b lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739054.313022:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739054.313102:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652422656 00000100:00100000:3.0:1787739054.313112:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652422656 00010000:00010000:3.0:1787739054.313149:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aba41 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b56 expref: 17 pid: 6523 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739054.313188:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1393 00000100:00100000:3.0:1787739054.313199:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f72580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652422656:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739054.313215:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739054.313221:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aba41 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b56 expref: 17 pid: 6523 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.313235:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aba41 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b56 expref: 17 pid: 6523 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.313259:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1aba41 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b56 expref: 17 pid: 6523 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.313286:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb8:0x0].0x0 (ffffa074331c2700) 00010000:00010000:3.0:1787739054.313291:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aba6b lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.313300:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aba6b lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.313308:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aba6b lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.313344:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739054.313394:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f72580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:2409:x1874579652422656:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 198us (380us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.313402:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1393 00010000:00010000:0.0:1787739054.313539:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aba6b lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.313548:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aba6b lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.313594:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1aba72 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.313602:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1aba72 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.313611:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1aba72 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.313629:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d86580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665552768:0@lo:101:mdt00_002.0 00000100:00100000:0.0:1787739054.313668:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.313701:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcce40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739054.314481:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665552768 00000100:00100000:2.0:1787739054.314489:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665552768 00000100:00100000:2.0:1787739054.314527:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2608 00000100:00100000:2.0:1787739054.314534:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c4170c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+5:6525:x1874579665552768:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787739054.314552:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739054.314567:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aba79 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aba72 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.314590:0:18535:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba4f lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b5d expref: 7 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.314598:0:18535:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aba79 lrc: 4/0,0 mode: --/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aba72 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.314625:0:18535:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba4f lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b5d expref: 7 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.314637:0:18535:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba4f lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b5d expref: 7 pid: 13793 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739054.314647:0:18535:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be61c0 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:18535:1874579665552896:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739054.314703:0:18535:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e840 going to sleep for 11 seconds 00000100:00100000:1.0:1787739054.329247:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739054.329385:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652422784 00000100:00100000:0.0:1787739054.329394:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652422784 00010000:00010000:0.0:1787739054.329461:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba4f lrc: 5/0,0 mode: PR/PR res: [0x240000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b5d expref: 8 pid: 13793 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.329499:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1394 00000100:00100000:0.0:1787739054.329508:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f70f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2409:x1874579652422784:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739054.329523:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739054.329529:0:6510:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba4f lrc: 5/0,0 mode: PR/PR res: [0x240000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b5d expref: 8 pid: 13793 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739054.329532:0:18535:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be61c0 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:18535:1874579665552896:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739054.329540:0:6510:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba4f lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b5d expref: 8 pid: 13793 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.329613:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba4f lrc: 0/0,0 mode: --/PR res: [0x240000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b5d expref: 8 pid: 13793 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.331962:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aba79 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aba72 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.331979:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb8:0x0].0x0 (ffffa073068a9800) 00010000:00010000:3.0:1787739054.331982:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739054.331986:0:18535:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aba79 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aba72 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.331998:0:18535:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aba79 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1aba72 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.332008:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aba79 lrc: 4/0,0 mode: EX/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1aba72 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.332048:0:18535:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aba79 lrc: 4/0,0 mode: EX/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aba72 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739054.332063:0:18535:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7840 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:18535:1874579665553024:0@lo:105: 00000100:00100000:3.0:1787739054.332138:0:18535:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739054.332192:0:18535:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054540 going to sleep for 11 seconds 00000100:00100000:2.0:1787739054.336883:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665553024 00000100:00100000:2.0:1787739054.336896:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665553024 00000100:00100000:2.0:1787739054.336958:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1064 00000100:00100000:2.0:1787739054.336968:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f52c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:18535:x1874579665553024:12345-0@lo:105: 00010000:00010000:2.0:1787739054.336984:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1aba72 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aba79 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.336997:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1aba72 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aba79 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.337006:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1aba72 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aba79 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.337015:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1aba72 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1aba79 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.337023:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1aba72 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1aba79 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.337038:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1aba72 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1aba79 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.337055:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609a400) 00000100:00100000:2.0:1787739054.337330:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f52c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:18535:x1874579665553024:12345-0@lo:105: Request processed in 301us (5129us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739054.337338:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1064 00010000:00010000:0.0:1787739054.339197:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb8:0x0].0x0 (ffffa073068a9800) 00010000:00010000:0.0:1787739054.339208:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739054.339264:0:18535:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7840 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:18535:1874579665553024:0@lo:105: 00010000:00010000:3.0:1787739054.339286:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d600, rc 0) 00000100:00100000:3.0:1787739054.339353:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c4170c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6525:x1874579665552768:12345-0@lo:101:mdt00_002.0 Request processed in 24819us (25683us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.339361:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2608 00000100:00100000:2.0:1787739054.340672:0:6525:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968304 gen 2 00000004:00080000:2.0:1787739054.340682:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968299 committed transno 4294968304 00000020:00080000:2.0:1787739054.340688:0:6525:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa0730638ba80 refcount 2 committed 0 result 0 batchid 347 00000020:00080000:2.0:1787739054.340693:0:6525:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88200 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730638b0c0 00000020:00080000:2.0:1787739054.340696:0:6525:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].344 00000020:00080000:2.0:1787739054.340699:0:6525:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa074051e2700 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306962800 00000020:00080000:2.0:1787739054.340702:0:6525:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].345 00000004:00080000:2.0:1787739054.340725:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968301 committed transno 4294968304 00000004:00080000:2.0:1787739054.340734:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968302 committed transno 4294968304 00000004:00080000:2.0:1787739054.340744:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968304 committed transno 4294968304 00000100:00100000:2.0:1787739054.340758:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d86580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665552768:0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787739054.340779:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1aba72 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aba79 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739054.343400:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 347 committed batchid 346 00000020:00080000:2.0:1787739054.343487:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 346 00000100:00100000:2.0:1787739054.343832:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be5e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665553152:0@lo:403:mdt00_002.0 00000100:00100000:2.0:1787739054.343905:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739054.343954:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739054.343977:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665553152 00000100:00100000:3.0:1787739054.343984:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665553152 00000100:00100000:3.0:1787739054.344035:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2609 00000100:00100000:3.0:1787739054.344044:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6525:x1874579665553152:12345-0@lo:403:mdt00_002.0 00000100:00100000:3.0:1787739054.346350:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6525:x1874579665553152:12345-0@lo:403:mdt00_002.0 Request processed in 2299us (2438us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.346363:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2609 00000100:00100000:1.0:1787739054.346745:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f70f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2409:x1874579652422784:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 17234us (17504us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.346755:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1394 00000100:00100000:2.0:1787739054.347231:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665553152:0@lo:403:mdt00_002.0 00000040:00080000:2.0:1787739054.353497:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].347, 664 off532784 00000040:00080000:2.0:1787739054.353661:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].346, 744 off576016 00000004:00080000:2.0:1787739054.353668:0:6525:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730610e900 with version 660 00000004:00080000:2.0:1787739054.353694:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 660 rpc_version 660 00000020:00100000:2.0:1787739054.353727:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739054.353738:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306be5e00 x1874579665553408/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1448/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739054.353760:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be5e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665553408:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739054.353824:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739054.353861:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e780 going to sleep for 11 seconds 00000100:00100000:1.0:1787739054.354027:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665553408 00000100:00100000:1.0:1787739054.354034:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665553408 00000100:00100000:1.0:1787739054.354350:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2610 00000100:00100000:1.0:1787739054.354361:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665553408:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739054.354606:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665553408:12345-0@lo:1000:osp_up1-0.0 Request processed in 247us (781us total) trans 4294968305 rc 0/0 00000100:00100000:1.0:1787739054.354614:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2610 00000100:00100000:3.0:1787739054.354864:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665553408:0@lo:1000:osp_up1-0.0 00000004:00200000:2.0:1787739054.357398:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14f9:0x0]: returning size 4096 00010000:00010000:2.0:1787739054.357456:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aba6b lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357468:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aba6b lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357475:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aba6b lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357482:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633cc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739054.357487:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aba6b lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357495:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aba6b lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357510:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb8:0x0].0x0 (ffffa074331c2700) 00010000:00010000:2.0:1787739054.357514:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739054.357518:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aba6b lrc: 2/0,0 mode: --/EX res: [0x200000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357523:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1aba6b lrc: 0/0,0 mode: --/EX res: [0x200000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357537:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1aba72 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aba79 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357545:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1aba72 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aba79 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357552:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1aba72 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aba79 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357572:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739054.357580:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xb8:0x0] 00010000:00010000:2.0:1787739054.357589:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aba64 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14f9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357594:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aba64 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x14f9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357600:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aba64 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14f9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357604:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739054.357607:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aba64 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14f9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357613:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aba64 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x14f9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357620:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f9:0x0].0x0 (ffffa074331c2c00) 00010000:00010000:2.0:1787739054.357623:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739054.357625:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aba64 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14f9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357630:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1aba64 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14f9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357646:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1aba56 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357653:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1aba56 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357658:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1aba56 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357663:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1aba56 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357668:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739054.357671:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1aba56 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357677:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1aba56 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357684:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5200) 00010000:00010000:2.0:1787739054.357686:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739054.357689:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1aba56 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357699:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1aba56 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739054.357713:0:6525:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f5abb802 reply state 0000000063f10556 transno 4294983316 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aba5d lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357724:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aba5d lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.357731:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x793368a3 (ffffa074419d5300) 00010000:00010000:2.0:1787739054.357733:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739054.357736:0:6525:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787739054.357829:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f70780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17445:x1874579652422528:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 58640us (58892us total) trans 4294983316 rc 0/0 00000100:00100000:2.0:1787739054.357838:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23203 00010000:00010000:0.0:1787739054.358191:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739054.358200:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1aba72 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aba79 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.358207:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739054.358212:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1aba72 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aba79 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739054.358220:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1aba72 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aba79 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.358252:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1aba72 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aba79 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.358259:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739054.358301:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1aba72 lrc: 1/0,0 mode: --/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aba79 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.358308:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1aba72 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aba79 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.358323:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739054.358324:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739054.358326:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739054.358333:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787739054.358350:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c4161c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665553536:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739054.358408:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.358466:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665553536 00000100:00100000:0.0:1787739054.358474:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665553536 00000100:00100000:0.0:1787739054.358514:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1194 00000100:00100000:0.0:1787739054.358522:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba3050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+7:3652:x1874579665553536:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739054.358535:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739054.358552:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1aba79 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aba72 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.358564:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb8:0x0].0x0 (ffffa073068a9800) 00010000:00010000:0.0:1787739054.358571:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739054.358602:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba3050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+6:3652:x1874579665553536:12345-0@lo:103:ldlm_bl.0 Request processed in 80us (194us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739054.358608:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1194 00000100:00100000:0.0:1787739054.358680:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c4161c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665553536:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739054.404393:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739054.407600:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739054.407641:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652422912 00000100:00100000:1.0:1787739054.407650:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652422912 00000100:00100000:1.0:1787739054.407702:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23204 00000100:00100000:1.0:1787739054.407712:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f71e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17446:x1874579652422912:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739054.407736:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739054.407760:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1aba80 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094b64 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.407825:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba87 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.407838:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba87 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.407850:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba87 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.407863:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1aba8e lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.407870:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1aba8e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.407876:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1aba8e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.407916:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba87 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.407924:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba87 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.407930:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba87 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.407947:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba87 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.407955:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739054.407959:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba87 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.407966:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba87 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.407978:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235600) 00010000:00010000:1.0:1787739054.407982:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739054.407986:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba87 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.407992:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aba87 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.408003:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1aba8e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.408009:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1aba8e lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.408015:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1aba8e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.408019:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739054.408022:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1aba8e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.408028:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1aba8e lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.408039:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xef411af6 (ffffa07433235f00) 00010000:00010000:1.0:1787739054.408042:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739054.408045:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1aba8e lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.408051:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1aba8e lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.408091:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1aba80 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b64 expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.408101:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235600) 00010000:00010000:1.0:1787739054.408104:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739054.408107:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1aba80 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b64 expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.408118:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1dc00, rc 0) 00000100:00100000:1.0:1787739054.408169:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f71e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17446:x1874579652422912:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 458us (578us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739054.408176:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23204 00000100:00100000:2.0:1787739054.417815:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739054.418136:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652423040 00000100:00100000:0.0:1787739054.418146:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652423040 00000100:00100000:0.0:1787739054.418202:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23205 00000100:00100000:0.0:1787739054.418212:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17446:x1874579652423040:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739054.418233:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739054.418255:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1aba95 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094b6b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.418321:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1aba9c lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.418334:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1aba9c lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.418345:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1aba9c lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.418355:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1abaa3 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.418361:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1abaa3 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.418367:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1abaa3 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.418404:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1aba9c lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.418438:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1aba9c lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.418445:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1aba9c lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.418451:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1aba9c lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.418456:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739054.418460:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1aba9c lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.418466:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1aba9c lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.418477:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9800) 00010000:00010000:0.0:1787739054.418480:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739054.418485:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1aba9c lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.418490:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1aba9c lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.418501:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1abaa3 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.418506:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1abaa3 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.418511:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1abaa3 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.418516:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609aa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739054.418518:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1abaa3 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.418523:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1abaa3 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.418528:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xef411af6 (ffffa073068a9000) 00010000:00010000:0.0:1787739054.418531:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739054.418533:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1abaa3 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.418538:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1abaa3 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.418567:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1aba95 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b6b expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.418577:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9800) 00010000:00010000:0.0:1787739054.418580:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739054.418583:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1aba95 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b6b expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.418594:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a400, rc 0) 00000100:00100000:0.0:1787739054.418653:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17446:x1874579652423040:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 442us (1720us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739054.418661:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23205 00000100:00100000:3.0:1787739054.436500:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739054.436617:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652423168 00000100:00100000:2.0:1787739054.436628:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652423168 00000100:00100000:2.0:1787739054.436678:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23206 00000100:00100000:2.0:1787739054.436690:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17446:x1874579652423168:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739054.436709:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739054.436742:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abaaa lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094b72 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.436801:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1abab1 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.436815:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1abab1 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.436827:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1abab1 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739054.436874:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:2.0:1787739054.436878:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739054.436913:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abaaa lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b72 expref: 19 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.436932:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1abab1 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b72 expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.436953:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5200) 00010000:00010000:2.0:1787739054.436957:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739054.436960:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633cc00, rc 0) 00000100:00100000:2.0:1787739054.437056:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17446:x1874579652423168:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 370us (567us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739054.437065:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23206 00000100:00100000:2.0:1787739054.468476:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739054.468525:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652423424 00000100:00100000:2.0:1787739054.468534:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652423424 00000100:00100000:2.0:1787739054.468590:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23207 00000100:00100000:2.0:1787739054.468601:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17446:x1874579652423424:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:2.0:1787739054.468713:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be6d00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665553664:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739054.468773:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739054.468816:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ef00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739054.469517:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665553664 00000100:00100000:2.0:1787739054.469524:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665553664 00000100:00100000:2.0:1787739054.469557:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2611 00000100:00100000:2.0:1787739054.469564:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665553664:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739054.469641:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665553664:12345-0@lo:1000:mdt00_003.0 Request processed in 77us (867us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739054.469648:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2611 00000100:00100000:2.0:1787739054.469690:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be6d00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665553664:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739054.469750:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be6d00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665553792:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739054.469773:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739054.469794:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ee40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739054.469804:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665553792 00000100:00100000:2.0:1787739054.469807:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665553792 00000100:00100000:2.0:1787739054.469827:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2612 00000100:00100000:2.0:1787739054.469832:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665553792:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739054.469911:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665553792:12345-0@lo:1000:mdt00_003.0 Request processed in 78us (137us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739054.469917:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2612 00000100:00100000:2.0:1787739054.469943:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be6d00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665553792:0@lo:1000:mdt00_003.0 00010000:00010000:2.0:1787739054.469990:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1abab8 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.470001:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1abab8 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.470010:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1abab8 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739054.470022:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be6d00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665553920:0@lo:101:mdt00_003.0 00000100:00100000:2.0:1787739054.470048:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739054.470062:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ee40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739054.470072:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665553920 00000100:00100000:2.0:1787739054.470083:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665553920 00000100:00100000:2.0:1787739054.470105:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2613 00000100:00100000:2.0:1787739054.470110:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be7c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665553920:12345-0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739054.470119:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739054.470133:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ababf lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abab8 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.470155:0:8448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1abab1 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b72 expref: 17 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.470163:0:8448:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ababf lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abab8 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.470185:0:8448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1abab1 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b72 expref: 17 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.470198:0:8448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1abab1 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b72 expref: 17 pid: 6525 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739054.470209:0:8448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be6580 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8448:1874579665554048:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739054.470259:0:8448:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ef00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739054.478342:0:8448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be6580 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8448:1874579665554048:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739054.478380:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ababf lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abab8 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.478393:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5200) 00010000:00010000:1.0:1787739054.478396:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739054.478400:0:8448:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ababf lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abab8 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.478410:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d000, rc 0) 00000100:00100000:1.0:1787739054.478501:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be7c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665553920:12345-0@lo:101:mdt00_003.0 Request processed in 8389us (8450us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.478509:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2613 00000100:00100000:3.0:1787739054.478595:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be6d00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665553920:0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739054.478611:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1abab8 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ababf expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.478622:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1abab8 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ababf expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739054.488973:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739054.489141:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652423552 00000100:00100000:3.0:1787739054.489150:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652423552 00010000:00010000:3.0:1787739054.489188:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1abab1 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b72 expref: 18 pid: 6525 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739054.489232:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1395 00000100:00100000:3.0:1787739054.489242:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be6580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2409:x1874579652423552:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739054.489257:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739054.489264:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1abab1 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b72 expref: 18 pid: 6525 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.489278:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1abab1 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b72 expref: 18 pid: 6525 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.489300:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1abab1 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b72 expref: 18 pid: 6525 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.489313:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5200) 00010000:00010000:3.0:1787739054.489319:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ababf lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abab8 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.489329:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ababf lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1abab8 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.489338:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ababf lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1abab8 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.489370:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ababf lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abab8 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739054.489384:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665554176:0@lo:105: 00000100:00100000:3.0:1787739054.489462:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739054.489510:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054540 going to sleep for 11 seconds 00000100:00100000:2.0:1787739054.489566:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665554176 00000100:00100000:2.0:1787739054.489571:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665554176 00000100:00100000:2.0:1787739054.489606:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1065 00000100:00100000:2.0:1787739054.489614:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665554176:12345-0@lo:105: 00010000:00010000:2.0:1787739054.489626:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1abab8 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ababf expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.489635:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1abab8 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ababf expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.489644:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1abab8 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ababf expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.489651:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1abab8 lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ababf expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.489659:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1abab8 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ababf expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.489667:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1abab8 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ababf expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.489691:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633dc00) 00000100:00100000:2.0:1787739054.489735:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665554176:12345-0@lo:105: Request processed in 111us (262us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739054.489742:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1065 00010000:00010000:0.0:1787739054.489858:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1abab8 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ababf expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.489868:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1abab8 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ababf expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739054.489894:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739054.489927:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1abac6 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.489934:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1abac6 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.489944:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1abac6 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.489960:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c4161c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665554304:0@lo:101:mdt00_003.0 00000100:00100000:0.0:1787739054.489988:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.490014:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fccf00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739054.490120:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665554304 00000100:00100000:2.0:1787739054.490125:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665554304 00000100:00100000:2.0:1787739054.490160:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2614 00000100:00100000:2.0:1787739054.490167:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c4152c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665554304:12345-0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739054.490182:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739054.490197:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abacd lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abac6 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.490216:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abacd lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abac6 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.490227:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abacd lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abac6 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.490235:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xef411af6 (ffffa074419d5d00) 00010000:00010000:2.0:1787739054.490238:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739054.490241:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d200, rc 0) 00000100:00100000:2.0:1787739054.490272:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c4152c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665554304:12345-0@lo:101:mdt00_003.0 Request processed in 106us (283us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739054.490277:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2614 00000100:00100000:0.0:1787739054.490584:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c4161c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665554304:0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739054.490599:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1abac6 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abacd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.490608:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1abac6 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abacd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739054.490622:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:0.0:1787739054.490653:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c4161c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665554432:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.490680:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.490704:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739054.490804:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665554432 00000100:00100000:2.0:1787739054.490809:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665554432 00000100:00100000:2.0:1787739054.490842:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2615 00000100:00100000:2.0:1787739054.490858:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c416580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665554432:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739054.490929:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c416580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665554432:12345-0@lo:1000:mdt00_003.0 Request processed in 71us (247us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739054.490936:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2615 00000100:00100000:3.0:1787739054.491125:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665554176:0@lo:105: 00010000:00010000:3.0:1787739054.491138:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739054.491187:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be6580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652423552:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 1946us (2222us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.491195:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1395 00000100:00100000:0.0:1787739054.491714:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c4161c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665554432:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.491771:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c4161c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665554560:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.491801:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.491825:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739054.491947:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665554560 00000100:00100000:1.0:1787739054.491952:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665554560 00000100:00100000:1.0:1787739054.491987:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2616 00000100:00100000:1.0:1787739054.491995:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c414b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665554560:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739054.492085:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c414b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665554560:12345-0@lo:1000:mdt00_003.0 Request processed in 83us (276us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739054.492092:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2616 00000100:00100000:3.0:1787739054.492152:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c4161c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665554560:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739054.492221:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665554688:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739054.492249:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739054.492279:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073070549c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739054.492320:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665554688 00000100:00100000:1.0:1787739054.492325:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665554688 00000100:00100000:1.0:1787739054.492356:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2617 00000100:00100000:1.0:1787739054.492363:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665554688:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739054.492437:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665554688:12345-0@lo:1000:mdt00_003.0 Request processed in 51us (163us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739054.492443:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2617 00000100:00100000:3.0:1787739054.492509:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665554688:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739054.492554:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665554816:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739054.492576:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739054.492601:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073070549c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739054.492637:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665554816 00000100:00100000:1.0:1787739054.492641:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665554816 00000100:00100000:1.0:1787739054.492669:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2618 00000100:00100000:1.0:1787739054.492675:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665554816:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739054.492717:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665554816:12345-0@lo:1000:mdt00_003.0 Request processed in 42us (140us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.492736:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2618 00000100:00100000:3.0:1787739054.492827:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665554816:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739054.492872:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665554944:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739054.492895:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739054.492915:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073070549c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739054.493001:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665554944 00000100:00100000:1.0:1787739054.493006:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665554944 00000100:00100000:1.0:1787739054.493039:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2619 00000100:00100000:1.0:1787739054.493045:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665554944:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739054.493116:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665554944:12345-0@lo:1000:mdt00_003.0 Request processed in 70us (220us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.493122:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2619 00000100:00100000:3.0:1787739054.493551:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665554944:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739054.493595:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ee940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665555072:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739054.493621:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739054.493650:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073070549c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739054.493698:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665555072 00000100:00100000:1.0:1787739054.493702:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665555072 00000100:00100000:1.0:1787739054.493739:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2620 00000100:00100000:1.0:1787739054.493746:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665555072:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739054.493797:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665555072:12345-0@lo:1000:mdt00_003.0 Request processed in 51us (175us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.493804:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2620 00000100:00100000:0.0:1787739054.493980:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ee940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665555072:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.494012:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c417480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665555200:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.494035:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.494057:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739054.494107:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665555200 00000100:00100000:2.0:1787739054.494112:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665555200 00000100:00100000:2.0:1787739054.494145:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2621 00000100:00100000:2.0:1787739054.494151:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c415e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665555200:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739054.494208:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c415e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665555200:12345-0@lo:1000:mdt00_003.0 Request processed in 57us (172us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739054.494218:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2621 00000100:00100000:0.0:1787739054.494279:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c417480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665555200:0@lo:1000:mdt00_003.0 00000040:00080000:0.0:1787739054.497251:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].333, 2168 off508136 00000040:00080000:0.0:1787739054.497371:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].333, 2368 off554952 00000004:00080000:0.0:1787739054.497378:0:7944:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07305fccf00 with version 635 00000004:00080000:3.0:1787739054.497475:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 635 rpc_version 635 00000020:00100000:3.0:1787739054.497525:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739054.497534:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074156ed2c0 x1874579665555456/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739054.497558:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ed2c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665555456:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739054.497625:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739054.497667:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739054.497965:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665555456 00000100:00100000:1.0:1787739054.497972:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665555456 00000100:00100000:1.0:1787739054.498017:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2622 00000100:00100000:1.0:1787739054.498025:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665555456:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739054.499236:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665555456:12345-0@lo:1000:osp_up0-1.0 Request processed in 1209us (1607us total) trans 4294983317 rc 0/0 00000100:00100000:0.0:1787739054.499250:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2622 00000100:00100000:3.0:1787739054.499370:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ed2c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665555456:0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1787739054.499528:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abad4 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x160:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.499545:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abad4 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x160:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.499557:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abad4 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x160:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.499577:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1abadb lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.499584:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1abadb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.499590:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1abadb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.499617:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1abae2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.499624:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1abae2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.499633:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1abae2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.499649:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c417480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665555584:0@lo:101:mdt00_003.0 00000100:00100000:0.0:1787739054.499680:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.499707:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4dcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739054.499823:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665555584 00000100:00100000:3.0:1787739054.499829:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665555584 00000100:00100000:3.0:1787739054.499875:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2623 00000100:00100000:3.0:1787739054.499883:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c416d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665555584:12345-0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739054.499900:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739054.499916:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1abae9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abae2 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.499936:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1abae9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abae2 expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.499948:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1abae9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abae2 expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.499957:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb6:0x0].0x0 (ffffa0730608b500) 00010000:00010000:3.0:1787739054.499960:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739054.499963:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abe00, rc 0) 00000100:00100000:3.0:1787739054.500012:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c416d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:7944:x1874579665555584:12345-0@lo:101:mdt00_003.0 Request processed in 130us (331us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.500019:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2623 00000100:00100000:0.0:1787739054.500157:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c417480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665555584:0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739054.500172:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1abae2 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abae9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.500180:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1abae2 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abae9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739054.500198:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000fe3c2d9 reply state 0000000089405e0c transno 4294968306 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1abadb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.500209:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1abadb lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.500215:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb6:0x0].0x0 (ffffa073068a9400) 00010000:00010000:0.0:1787739054.500218:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739054.500221:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739054.500226:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1abae2 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abae9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.500233:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1abae2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abae9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739054.500244:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xb6:0x0] 00000004:00010000:0.0:1787739054.500249:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000fe3c2d9 reply state 0000000089405e0c transno 4294968306 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abad4 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x160:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.500256:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abad4 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x160:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.500261:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x160:0x0].0x0 (ffffa073068a9000) 00010000:00010000:0.0:1787739054.500264:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739054.500266:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787739054.500270:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x160:0x0]: returning size 4096 00000004:00080000:0.0:1787739054.500277:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739054.500284:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1abab8 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ababf expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.500290:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1abab8 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ababf expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.500296:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1abab8 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ababf expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.500319:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739054.500323:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739054.500327:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1abac6 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abacd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.500334:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1abac6 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abacd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.500437:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17446:x1874579652423424:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 31810us (31942us total) trans 4294968306 rc 0/0 00000100:00100000:0.0:1787739054.500444:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23207 00010000:00010000:2.0:1787739054.500513:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739054.500520:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1abab8 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ababf expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.500528:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633dc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739054.500533:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1abab8 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ababf expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739054.500542:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1abab8 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ababf expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.500568:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1abab8 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ababf expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.500574:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739054.500599:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1abab8 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ababf expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.500606:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1abab8 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ababf expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.500622:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739054.500624:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739054.500626:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739054.500741:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665555712:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739054.500776:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739054.500953:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665555712 00000100:00100000:1.0:1787739054.500958:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665555712 00000100:00100000:1.0:1787739054.500999:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1195 00000100:00100000:1.0:1787739054.501006:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c417480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3650:x1874579665555712:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739054.501017:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739054.501032:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ababf lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abab8 expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.501050:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5200) 00010000:00010000:1.0:1787739054.501056:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739054.501104:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c417480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+10:3650:x1874579665555712:12345-0@lo:103:ldlm_bl.0 Request processed in 98us (327us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.501111:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1195 00000100:00100000:2.0:1787739054.501169:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665555712:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739054.532177:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787739054.532198:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c415680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665555840:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1787739054.532271:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.535342:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665555840 00000100:00100000:0.0:1787739054.535356:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665555840 00000100:00100000:0.0:1787739054.535437:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2624 00000100:00100000:0.0:1787739054.535449:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f72580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:3650:x1874579665555840:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1787739054.535485:0:18535:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1444085760 avail 1297846272 00000100:00100000:0.0:1787739054.535566:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f72580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:3650:x1874579665555840:12345-0@lo:41:osp-pre-0-1.0 Request processed in 120us (3295us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739054.535574:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2624 00000100:00100000:2.0:1787739054.537141:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c415680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665555840:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1787739054.551916:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739054.552265:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652423808 00000100:00100000:0.0:1787739054.552274:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652423808 00000100:00100000:0.0:1787739054.552329:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23208 00000100:00100000:0.0:1787739054.552338:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be6d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17447:x1874579652423808:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:0.0:1787739054.552518:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be6d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17447:x1874579652423808:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 181us (609us total) trans 0 rc -17/-17 00000100:00100000:0.0:1787739054.552525:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23208 00000100:00100000:3.0:1787739054.568789:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739054.569006:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652423936 00000100:00100000:0.0:1787739054.569015:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652423936 00000100:00100000:0.0:1787739054.569085:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23209 00000100:00100000:0.0:1787739054.569095:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156eda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17448:x1874579652423936:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739054.569114:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739054.569133:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1abaf0 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094b79 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.569191:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abaf7 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.569204:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abaf7 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.569214:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abaf7 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.569225:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1abafe lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.569235:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abacd lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abac6 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.569242:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1abafe lrc: 4/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.569269:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abacd lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1abac6 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.569282:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c415a40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665555968:0@lo:104: 00000100:00100000:0.0:1787739054.569327:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.569373:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d840 going to sleep for 11 seconds 00000100:00100000:3.0:1787739054.569566:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665555968 00000100:00100000:3.0:1787739054.569572:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665555968 00000100:00100000:3.0:1787739054.569613:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1066 00000100:00100000:3.0:1787739054.569620:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c416940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665555968:12345-0@lo:104: 00010000:00010000:3.0:1787739054.569632:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1abac6 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1abacd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.575555:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739054.575567:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c416940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665555968:12345-0@lo:104: Request processed in 5944us (6236us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.575579:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1066 00000100:00100000:0.0:1787739054.575685:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c415a40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665555968:0@lo:104: 00010000:00010000:0.0:1787739054.575710:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1abafe lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.576790:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739054.576803:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1abac6 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1abacd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.576814:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609aa00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739054.576821:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1abac6 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1abacd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739054.576830:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1abac6 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1abacd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.576870:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1abac6 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abacd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.576878:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739054.576911:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1abac6 lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abacd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.576919:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1abac6 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abacd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.576950:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739054.576952:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739054.576954:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739054.576962:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787739054.576979:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f70f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665556096:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739054.577035:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.577605:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665556096 00000100:00100000:0.0:1787739054.577614:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665556096 00010000:00010000:0.0:1787739054.577646:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abacd lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1abac6 expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.577686:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1396 00000100:00100000:0.0:1787739054.577695:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f721c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+10:3653:x1874579665556096:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739054.577708:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:2.0:1787739054.582245:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983317 is committed 00000001:00080000:2.0:1787739054.582281:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983316 is committed 00000020:00080000:2.0:1787739054.582290:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 347 batchid updated 00000020:00080000:2.0:1787739054.582305:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07435539d80 refcount 2 committed 0 result 0 batchid 348 00000020:00080000:2.0:1787739054.582310:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730415f400 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07435539540 00000020:00080000:2.0:1787739054.582314:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].345 00000020:00080000:2.0:1787739054.582316:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730415f200 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa0730541fa00 00000020:00080000:2.0:1787739054.582319:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].346 00000020:00080000:2.0:1787739054.582349:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 348 committed batchid 347 00000004:00080000:2.0:1787739054.582479:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0743c0ef540 with version 661 00000020:00080000:2.0:1787739054.582489:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 347 cancel update log [0x240000401:0x3:0x0].344: rc = 0 00000020:00080000:2.0:1787739054.582526:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 347 cancel update log [0x200000400:0x3:0x0].345: rc = 0 00000020:00080000:2.0:1787739054.582536:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 347 00000004:00080000:2.0:1787739054.582545:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 661 rpc_version 661 00000020:00100000:2.0:1787739054.582576:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739054.582586:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304d86940 x1874579665556352/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739054.582609:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d86940 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665556352:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739054.582675:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739054.582705:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0efd80 going to sleep for 11 seconds 00010000:00010000:2.0:1787739054.582764:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abacd lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1abac6 expref: 7 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.582782:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xef411af6 (ffffa074419d5d00) 00010000:00010000:2.0:1787739054.582787:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1abafe lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.582791:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab9fb lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.582799:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab9fb lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.582801:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1abafe lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.582807:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab9fb lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.582809:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1abafe lrc: 4/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.582813:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739054.582817:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab9fb lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.582824:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab9fb lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.582829:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739054.582833:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x793368a3 (ffffa074419d5300) 00010000:00010000:0.0:1787739054.582835:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739054.582838:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab9fb lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739054.582873:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f721c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+6:3653:x1874579665556096:12345-0@lo:103:ldlm_bl.0 Request processed in 5179us (5836us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739054.582882:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1396 00010000:00010000:0.0:1787739054.582920:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ab9fb lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.582935:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aba02 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x14f9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.582942:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aba02 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x14f9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.582948:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aba02 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x14f9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.582957:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739054.582961:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aba02 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x14f9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.582967:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aba02 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x14f9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.582977:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14f9:0x0].0x0 (ffffa074331c2c00) 00010000:00010000:0.0:1787739054.582979:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739054.582982:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aba02 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x14f9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.582987:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aba02 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x14f9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.583001:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aba09 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.583007:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aba09 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.583011:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aba09 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.583016:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aaa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739054.583018:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aba09 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.583023:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aba09 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.583033:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb8:0x0].0x0 (ffffa074331c2700) 00010000:00010000:0.0:1787739054.583035:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739054.583038:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aba09 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.583043:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aba09 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.583057:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aba5d lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.583063:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aba5d lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739054.583146:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f70f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665556096:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739054.583512:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665556352 00000100:00100000:2.0:1787739054.583520:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665556352 00000100:00100000:2.0:1787739054.583559:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2625 00000100:00100000:2.0:1787739054.583568:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d843c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665556352:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739054.583703:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d843c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665556352:12345-0@lo:1000:osp_up1-0.0 Request processed in 136us (1027us total) trans 4294968307 rc 0/0 00000100:00100000:2.0:1787739054.583711:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2625 00000100:00100000:2.0:1787739054.583750:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d86940 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665556352:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787739054.584161:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aba5d lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.584171:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739054.584175:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aba5d lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.584183:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aba5d lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.584195:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x793368a3 (ffffa074419d5300) 00010000:00010000:0.0:1787739054.584197:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739054.584201:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aba5d lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.584207:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aba5d lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x793368a3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.585230:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1abafe lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.585238:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1abafe lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.585357:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c415a40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665556480:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739054.585431:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.585461:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4db40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739054.585483:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665556480 00000100:00100000:0.0:1787739054.585489:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665556480 00000100:00100000:0.0:1787739054.585532:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2626 00000100:00100000:0.0:1787739054.585539:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c415680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665556480:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739054.585612:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c415680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665556480:12345-0@lo:1000:mdt00_001.0 Request processed in 72us (180us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739054.585619:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2626 00000100:00100000:0.0:1787739054.585658:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c415a40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665556480:0@lo:1000:mdt00_001.0 00010000:00010000:0.0:1787739054.585702:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abb05 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x160:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.585716:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abb05 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x160:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.588986:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abb05 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x160:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739054.589023:0:6524:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abb05 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x160:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.589045:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abaf7 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.589053:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abaf7 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.589059:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abaf7 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.589067:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abaf7 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.589535:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609be00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739054.589539:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abaf7 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.589548:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abaf7 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.589563:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9700) 00010000:00010000:0.0:1787739054.589566:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739054.589570:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abaf7 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.589577:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abaf7 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.589590:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1abafe lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.589597:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1abafe lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.589604:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1abafe lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.589609:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739054.589611:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1abafe lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.589617:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1abafe lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.589624:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xef411af6 (ffffa074419d5d00) 00010000:00010000:0.0:1787739054.589626:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739054.589629:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1abafe lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.589633:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1abafe lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.589684:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1abaf0 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b79 expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.589700:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abb05 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x160:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b79 expref: 14 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.589707:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x160:0x0].0x0 (ffffa07307064d00) 00010000:00010000:0.0:1787739054.589709:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739054.589712:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609bc00, rc 0) 00000100:00100000:0.0:1787739054.590298:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156eda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17448:x1874579652423936:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 21201us (21513us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739054.590313:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23209 00000100:00100000:3.0:1787739054.597200:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739054.597332:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652424064 00000100:00100000:1.0:1787739054.597342:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652424064 00000100:00100000:1.0:1787739054.597401:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23210 00000100:00100000:1.0:1787739054.597443:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ec780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17448:x1874579652424064:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739054.597472:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739054.597493:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1abb0c lrc: 2/0,0 mode: --/CR res: [0x240000402:0x160:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094b80 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.597554:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1abb13 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x160:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.597568:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1abb13 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x160:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.597580:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1abb13 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x160:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739054.597675:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x160:0x0]: returning size 4096 00000004:00200000:1.0:1787739054.597680:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739054.597718:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1abb0c lrc: 0/0,0 mode: --/CR res: [0x240000402:0x160:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b80 expref: 11 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.597737:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1abb13 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x160:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b80 expref: 10 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.597746:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x160:0x0].0x0 (ffffa073068a9000) 00010000:00010000:1.0:1787739054.597748:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739054.597751:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1de00, rc 0) 00000100:00100000:1.0:1787739054.597811:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ec780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17448:x1874579652424064:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 402us (623us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.597819:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23210 00000100:00100000:2.0:1787739054.606367:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739054.606504:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652424192 00000100:00100000:3.0:1787739054.606514:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652424192 00000100:00100000:3.0:1787739054.606567:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23211 00000100:00100000:3.0:1787739054.606577:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d86580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17448:x1874579652424192:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739054.606601:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739054.606622:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1abb1a lrc: 2/0,0 mode: --/CR res: [0x240000400:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094b87 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.606795:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1abb21 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.606809:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1abb21 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.606820:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1abb21 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739054.606930:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xb6:0x0]: returning size 4096 00000004:00200000:3.0:1787739054.606935:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739054.606971:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1abb1a lrc: 0/0,0 mode: --/CR res: [0x240000400:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b87 expref: 12 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.606985:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1abb21 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b87 expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.606995:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb6:0x0].0x0 (ffffa073068a9400) 00010000:00010000:3.0:1787739054.606998:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739054.607001:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab200, rc 0) 00000100:00100000:3.0:1787739054.607111:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d86580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17448:x1874579652424192:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 534us (754us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.607118:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23211 00000100:00100000:0.0:1787739054.612787:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739054.612875:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652424320 00000100:00100000:2.0:1787739054.612885:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652424320 00000100:00100000:2.0:1787739054.612946:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23212 00000100:00100000:2.0:1787739054.612956:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c415a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17448:x1874579652424320:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739054.612977:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739054.612999:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abb28 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094b8e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.613062:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abb2f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.613094:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1abae9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abae2 expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.613102:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abb2f lrc: 4/1,0 mode: --/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.613133:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1abae9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1abae2 expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739054.613148:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d870c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665556608:0@lo:104: 00000100:00100000:2.0:1787739054.613197:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739054.613240:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739054.613368:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665556608 00000100:00100000:1.0:1787739054.613373:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665556608 00000100:00100000:1.0:1787739054.613523:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1067 00000100:00100000:1.0:1787739054.613533:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c380780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665556608:12345-0@lo:104: 00010000:00010000:1.0:1787739054.613545:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1abae2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1abae9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.613595:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787739054.613601:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c380780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665556608:12345-0@lo:104: Request processed in 70us (403us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.613607:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1067 00010000:00010000:0.0:1787739054.613830:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739054.613837:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1abae2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1abae9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.613845:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739054.613850:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1abae2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1abae9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739054.613857:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1abae2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1abae9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.613882:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1abae2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abae9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.613889:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739054.613915:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1abae2 lrc: 1/0,0 mode: --/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abae9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.613923:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1abae2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abae9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.613948:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739054.613950:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739054.613952:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739054.614172:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c4161c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665556736:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739054.614205:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.614279:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665556736 00000100:00100000:0.0:1787739054.614284:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665556736 00010000:00010000:0.0:1787739054.614306:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1abae9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1abae2 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.614334:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1397 00000100:00100000:0.0:1787739054.614340:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f70f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+6:3653:x1874579665556736:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739054.614350:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1787739054.615905:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d870c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665556608:0@lo:104: 00010000:00010000:2.0:1787739054.615931:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abb2f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739054.620857:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 348 batchid updated 00000020:00080000:3.0:1787739054.620993:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 348 00000004:00080000:3.0:1787739054.621054:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa073070540c0 with version 662 00000020:00080000:3.0:1787739054.621083:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 348 cancel update log [0x240000401:0x3:0x0].345: rc = 0 00000020:00080000:3.0:1787739054.621168:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 348 cancel update log [0x200000400:0x3:0x0].346: rc = 0 00000020:00080000:3.0:1787739054.621177:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 348 00010000:00010000:1.0:1787739054.621576:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1abae9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1abae2 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.621595:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb6:0x0].0x0 (ffffa0730608b500) 00010000:00010000:1.0:1787739054.621602:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abb2f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.621611:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abb2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.621619:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abb2f lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.621658:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739054.621731:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f70f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+5:3653:x1874579665556736:12345-0@lo:103:ldlm_bl.0 Request processed in 7389us (7523us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.621741:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1397 00010000:00010000:3.0:1787739054.621785:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abb2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.621793:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abb2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739054.621794:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c4161c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665556736:0@lo:103:ldlm_bl.0 00000004:00200000:3.0:1787739054.621856:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb6:0x0]: returning size 4096 00000004:00200000:3.0:1787739054.621861:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739054.621903:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abb28 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b8e expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.621919:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abb2f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b8e expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.621926:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb6:0x0].0x0 (ffffa0730608b500) 00010000:00010000:3.0:1787739054.621930:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739054.621932:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c600, rc 0) 00000100:00100000:3.0:1787739054.621982:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c415a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17448:x1874579652424320:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 9028us (9205us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.621990:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23212 00000004:00080000:2.0:1787739054.622301:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 662 rpc_version 662 00000020:00100000:2.0:1787739054.622329:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739054.622337:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743c380000 x1874579665556992/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739054.622356:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665556992:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739054.622398:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739054.622446:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef300 going to sleep for 11 seconds 00000100:00100000:3.0:1787739054.622513:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665556992 00000100:00100000:3.0:1787739054.622519:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665556992 00000100:00100000:3.0:1787739054.622555:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2627 00000100:00100000:3.0:1787739054.622564:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665556992:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739054.622688:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665556992:12345-0@lo:1000:osp_up1-0.0 Request processed in 125us (289us total) trans 4294968308 rc 0/0 00000100:00100000:3.0:1787739054.622694:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2627 00000100:00100000:2.0:1787739054.623801:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665556992:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739054.630556:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739054.630589:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739054.630664:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652424704 00000100:00100000:3.0:1787739054.630673:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652424704 00000100:00100000:3.0:1787739054.630731:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23213 00000100:00100000:3.0:1787739054.630742:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c383c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17448:x1874579652424704:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:3.0:1787739054.630793:0:7944:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739054.630818:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1abb13 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x160:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b80 expref: 11 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.630834:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x160:0x0].0x0 (ffffa073068a9000) 00000100:00100000:0.0:1787739054.630834:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652424448 00010000:00010000:3.0:1787739054.630838:0:7944:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739054.630838:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652424448 00000100:00100000:0.0:1787739054.630881:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1196 00010000:00010000:3.0:1787739054.630883:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1abb36 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.630888:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c381a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2409:x1874579652424448:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:3.0:1787739054.630891:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1abb36 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.630897:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739054.630905:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1abb36 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.630908:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abb05 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x160:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b79 expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.630918:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x160:0x0].0x0 (ffffa07307064d00) 00000100:00100000:3.0:1787739054.630925:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665557120:0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739054.630925:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739054.630984:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c381a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652424448:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 95us (436us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739054.630992:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1196 00000100:00100000:3.0:1787739054.634584:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739054.634649:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739054.634767:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665557120 00000100:00100000:1.0:1787739054.634776:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665557120 00000100:00100000:1.0:1787739054.634827:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2628 00000100:00100000:1.0:1787739054.634836:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ecf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+5:7944:x1874579665557120:12345-0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739054.634852:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739054.634874:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abb3d lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abb36 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.634900:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abb3d lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abb36 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.634913:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abb3d lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abb36 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.634921:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235600) 00010000:00010000:1.0:1787739054.634924:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739054.634928:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1cc00, rc 0) 00000100:00100000:1.0:1787739054.634984:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ecf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:7944:x1874579665557120:12345-0@lo:101:mdt00_003.0 Request processed in 148us (407us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.634991:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2628 00000100:00100000:3.0:1787739054.635062:0:7944:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983317 gen 2 00000004:00080000:3.0:1787739054.635130:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983313 committed transno 4294983317 00000020:00080000:3.0:1787739054.635135:0:7944:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07305fcc240 refcount 2 committed 0 result 0 batchid 339 00000020:00080000:3.0:1787739054.635139:0:7944:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88100 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07305fcc180 00000020:00080000:3.0:1787739054.635142:0:7944:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].332 00000020:00080000:3.0:1787739054.635144:0:7944:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88600 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306962e00 00000020:00080000:3.0:1787739054.635147:0:7944:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].332 00000004:00080000:3.0:1787739054.635176:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983315 committed transno 4294983317 00000004:00080000:3.0:1787739054.635187:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983317 committed transno 4294983317 00000100:00100000:3.0:1787739054.635204:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665557120:0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739054.635220:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1abb36 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abb3d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.635230:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1abb36 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abb3d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739054.635249:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739054.635276:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1abb44 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.635283:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1abb44 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.635292:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1abb44 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739054.635310:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665557248:0@lo:101:mdt00_003.0 00000100:00100000:3.0:1787739054.635349:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739054.635367:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739054.635380:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665557248 00000100:00100000:3.0:1787739054.635383:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665557248 00000100:00100000:3.0:1787739054.635408:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2629 00000100:00100000:3.0:1787739054.635436:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:7944:x1874579665557248:12345-0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739054.635444:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739054.635456:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1abb4b lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abb44 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.635470:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1abb4b lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abb44 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.635480:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1abb4b lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abb44 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.635487:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xef411af6 (ffffa0730608bd00) 00010000:00010000:3.0:1787739054.635490:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739054.635492:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab600, rc 0) 00000100:00100000:3.0:1787739054.635520:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665557248:12345-0@lo:101:mdt00_003.0 Request processed in 85us (170us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.635527:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2629 00000100:00100000:3.0:1787739054.635552:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665557248:0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739054.635562:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1abb44 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abb4b expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.635570:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1abb44 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abb4b expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739054.635581:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:3.0:1787739054.635611:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665557376:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739054.635635:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739054.635664:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054000 going to sleep for 11 seconds 00000100:00100000:3.0:1787739054.635675:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665557376 00000100:00100000:3.0:1787739054.635679:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665557376 00000100:00100000:3.0:1787739054.635700:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2630 00000100:00100000:3.0:1787739054.635707:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665557376:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739054.635760:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665557376:12345-0@lo:1000:mdt00_003.0 Request processed in 53us (123us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.635766:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2630 00000100:00100000:3.0:1787739054.635791:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665557376:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739054.635823:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665557504:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739054.635844:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739054.635859:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054000 going to sleep for 11 seconds 00000100:00100000:3.0:1787739054.635870:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665557504 00000100:00100000:3.0:1787739054.635873:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665557504 00000100:00100000:3.0:1787739054.635893:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2631 00000100:00100000:3.0:1787739054.635898:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665557504:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739054.635970:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665557504:12345-0@lo:1000:mdt00_003.0 Request processed in 71us (124us total) trans 0 rc 1/1 00000100:00100000:3.0:1787739054.635976:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2631 00000100:00100000:3.0:1787739054.636000:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665557504:0@lo:1000:mdt00_003.0 00010000:00010000:3.0:1787739054.636034:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abb52 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x160:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.636042:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abb52 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x160:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.636049:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abb52 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x160:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739054.636061:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665557632:0@lo:101:mdt00_003.0 00000020:00080000:1.0:1787739054.636363:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 339 committed batchid 337 00000020:00080000:1.0:1787739054.636424:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 337 00000100:00100000:3.0:1787739054.638232:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739054.638279:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739054.640307:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665557632 00000100:00100000:0.0:1787739054.640321:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665557632 00000100:00100000:0.0:1787739054.640377:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2632 00000100:00100000:0.0:1787739054.640387:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ee940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665557632:12345-0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739054.640406:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739054.640443:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1abb59 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x160:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abb52 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.640477:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1abb59 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x160:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abb52 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.640490:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1abb59 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x160:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abb52 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.640499:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x160:0x0].0x0 (ffffa07307064500) 00010000:00010000:0.0:1787739054.640502:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739054.640505:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609ba00, rc 0) 00000100:00100000:0.0:1787739054.640564:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ee940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665557632:12345-0@lo:101:mdt00_003.0 Request processed in 179us (2337us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739054.640571:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2632 00000100:00100000:3.0:1787739054.640762:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665557632:0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739054.640780:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abb52 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x160:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abb59 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.640790:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abb52 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x160:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abb59 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.640818:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1abb60 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x160:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.640827:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1abb60 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x160:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.640834:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1abb60 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x160:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.640855:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb67 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.640867:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1abb21 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b87 expref: 10 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.640875:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb67 lrc: 4/0,1 mode: --/EX res: [0x240000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.640902:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1abb21 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b87 expref: 10 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.640914:0:7944:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1abb21 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b87 expref: 10 pid: 6525 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739054.640927:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665557760:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739054.642194:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739054.651896:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739054.653580:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652424832 00000100:00100000:1.0:1787739054.653591:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652424832 00010000:00010000:1.0:1787739054.653630:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1abb21 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b87 expref: 11 pid: 6525 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739054.653672:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1398 00000100:00100000:1.0:1787739054.653682:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2409:x1874579652424832:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739054.653696:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739054.653702:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1abb21 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b87 expref: 11 pid: 6525 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.653714:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1abb21 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b87 expref: 11 pid: 6525 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.653745:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb6:0x0].0x0 (ffffa073068a9400) 00010000:00010000:1.0:1787739054.653750:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb67 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.653759:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb67 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.653766:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb67 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.653783:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739054.653835:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2409:x1874579652424832:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 154us (1947us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.653842:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1398 00010000:00010000:0.0:1787739054.654298:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1abb21 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b87 expref: 10 pid: 6525 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.654314:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665557760:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739054.654347:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb67 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.654357:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb67 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.654364:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb67 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.654403:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1abb6e lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.654443:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1abb6e lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.654453:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1abb6e lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.654472:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c4161c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665557888:0@lo:101:mdt00_003.0 00000100:00100000:0.0:1787739054.654517:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.654544:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739054.655307:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665557888 00000100:00100000:3.0:1787739054.655315:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665557888 00000100:00100000:3.0:1787739054.655365:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2633 00000100:00100000:3.0:1787739054.655374:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156efc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665557888:12345-0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739054.655389:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739054.655404:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1abb75 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abb6e expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.655452:0:8448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abb2f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b8e expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.655463:0:8448:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1abb75 lrc: 4/0,0 mode: --/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abb6e expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.655490:0:8448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abb2f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b8e expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.655503:0:8448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abb2f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b8e expref: 13 pid: 6524 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739054.655513:0:8448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8448:1874579665558016:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739054.655573:0:8448:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054000 going to sleep for 11 seconds 00000100:00100000:3.0:1787739054.665293:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739054.665337:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652424960 00000100:00100000:3.0:1787739054.665344:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652424960 00010000:00010000:3.0:1787739054.665378:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abb2f lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b8e expref: 14 pid: 6524 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739054.665430:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1399 00000100:00100000:3.0:1787739054.665441:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba0450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2407:x1874579652424960:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739054.665455:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739054.665461:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abb2f lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b8e expref: 14 pid: 6524 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.665472:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abb2f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b8e expref: 14 pid: 6524 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.665497:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb6:0x0].0x0 (ffffa0730608b500) 00010000:00010000:3.0:1787739054.665503:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1abb75 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abb6e expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.665513:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1abb75 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1abb6e expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.665522:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1abb75 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1abb6e expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.665550:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1abb75 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abb6e expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739054.665563:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665558144:0@lo:105: 00000100:00100000:3.0:1787739054.665601:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739054.665640:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054780 going to sleep for 11 seconds 00000100:00100000:3.0:1787739054.665656:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665558144 00000100:00100000:3.0:1787739054.665659:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665558144 00000100:00100000:3.0:1787739054.665686:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1068 00000100:00100000:3.0:1787739054.665693:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665558144:12345-0@lo:105: 00010000:00010000:3.0:1787739054.665703:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1abb6e lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abb75 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.665712:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1abb6e lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abb75 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.665726:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1abb6e lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abb75 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.665733:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1abb6e lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1abb75 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.665740:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1abb6e lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1abb75 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.665747:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1abb6e lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1abb75 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.665760:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609a400) 00000100:00100000:3.0:1787739054.665795:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665558144:12345-0@lo:105: Request processed in 102us (192us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.665802:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1068 00000100:00100000:3.0:1787739054.665845:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665558144:0@lo:105: 00010000:00010000:1.0:1787739054.667019:0:8448:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abb2f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b8e expref: 14 pid: 6524 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739054.667041:0:8448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8448:1874579665558016:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739054.667067:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1abb75 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abb6e expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.667087:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb6:0x0].0x0 (ffffa0730608b500) 00010000:00010000:1.0:1787739054.667090:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739054.667092:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aac00, rc 0) 00000100:00100000:1.0:1787739054.667150:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156efc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665557888:12345-0@lo:101:mdt00_003.0 Request processed in 11777us (12632us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.667158:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2633 00000100:00100000:1.0:1787739054.667195:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c4161c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665557888:0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739054.667210:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1abb6e lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abb75 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739054.667838:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f71a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665558272:0@lo:403:mdt00_003.0 00010000:00010000:3.0:1787739054.668231:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739054.668287:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba0450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:2407:x1874579652424960:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 2846us (3002us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.668297:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1399 00000100:00100000:1.0:1787739054.668789:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739054.668816:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739054.670129:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665558272 00000100:00100000:3.0:1787739054.670140:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665558272 00000100:00100000:3.0:1787739054.670195:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2634 00000100:00100000:3.0:1787739054.670206:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f71680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665558272:12345-0@lo:403:mdt00_003.0 00000100:00100000:3.0:1787739054.671842:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f71680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665558272:12345-0@lo:403:mdt00_003.0 Request processed in 1632us (3048us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.671853:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2634 00000100:00100000:3.0:1787739054.671919:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f71a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665558272:0@lo:403:mdt00_003.0 00000040:00080000:3.0:1787739054.678718:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].334, 952 off509088 00000040:00080000:3.0:1787739054.679090:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].334, 1032 off555984 00000004:00080000:3.0:1787739054.679096:0:7944:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07307054240 with version 636 00000004:00080000:0.0:1787739054.680253:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 636 rpc_version 636 00000020:00100000:0.0:1787739054.680295:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739054.680305:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07415700b40 x1874579665558528/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739054.680340:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415700b40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665558528:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739054.680753:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739054.680940:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739054.683843:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665558528 00000100:00100000:3.0:1787739054.683862:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665558528 00000100:00100000:3.0:1787739054.683914:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2635 00000100:00100000:3.0:1787739054.683923:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415701e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665558528:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739054.684331:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415701e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665558528:12345-0@lo:1000:osp_up0-1.0 Request processed in 408us (3578us total) trans 4294983318 rc 0/0 00000100:00100000:3.0:1787739054.684339:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2635 00000100:00100000:1.0:1787739054.686478:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415700b40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665558528:0@lo:1000:osp_up0-1.0 00000004:00200000:3.0:1787739054.687017:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x160:0x0]: returning size 4096 00010000:00010000:3.0:1787739054.687043:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb67 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.687053:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb67 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.687061:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb67 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.687142:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739054.687148:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb67 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.687157:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb67 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.687174:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb6:0x0].0x0 (ffffa073068a9400) 00010000:00010000:3.0:1787739054.687177:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739054.687182:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb67 lrc: 2/0,0 mode: --/EX res: [0x240000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.687188:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb67 lrc: 0/0,0 mode: --/EX res: [0x240000400:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.687202:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1abb6e lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abb75 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.687210:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1abb6e lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abb75 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.687217:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1abb6e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abb75 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.687245:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739054.687255:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xb6:0x0] 00010000:00010000:3.0:1787739054.687262:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1abb60 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x160:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.687269:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1abb60 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x160:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.687274:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1abb60 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x160:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.687280:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739054.687283:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1abb60 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x160:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.687289:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1abb60 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x160:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.687301:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x160:0x0].0x0 (ffffa073068a9000) 00010000:00010000:3.0:1787739054.687303:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739054.687306:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1abb60 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x160:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.687312:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1abb60 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x160:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.687320:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abb52 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x160:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abb59 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.687327:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abb52 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x160:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abb59 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.687334:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abb52 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x160:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abb59 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.687357:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:3.0:1787739054.687368:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739054.687372:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1abb36 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abb3d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.687379:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1abb36 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abb3d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.687385:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1abb36 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abb3d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.687397:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:3.0:1787739054.687400:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739054.687404:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1abb44 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abb4b expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.687435:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1abb44 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abb4b expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739054.687520:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c383c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17448:x1874579652424704:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 56777us (56926us total) trans 4294968309 rc 0/0 00010000:00010000:0.0:1787739054.687820:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1787739054.687828:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1abb6e lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1abb75 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.687836:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739054.687840:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1abb6e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abb75 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739054.687922:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1abb6e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abb75 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.687956:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1abb6e lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abb75 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.687963:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739054.687986:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1abb6e lrc: 1/0,0 mode: --/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abb75 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.687994:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1abb6e lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abb75 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.688012:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787739054.688016:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abb52 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x160:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1abb59 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.688022:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aae00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739054.688026:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abb52 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x160:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abb59 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739054.688032:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abb52 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x160:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abb59 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.688049:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abb52 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x160:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abb59 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.688055:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739054.688061:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abb52 lrc: 1/0,0 mode: --/EX res: [0x240000402:0x160:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abb59 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739054.688147:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23213 00010000:00010000:1.0:1787739054.688237:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739054.688243:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1abb36 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1abb3d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.688250:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739054.688254:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1abb36 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abb3d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739054.688262:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1abb36 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abb3d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.688280:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1abb36 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abb3d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.688286:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739054.688309:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1abb36 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abb3d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.688317:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1abb36 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abb3d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739054.688499:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f73840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665558912:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739054.688557:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739054.688618:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665558912 00000100:00100000:3.0:1787739054.688624:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665558912 00010000:00010000:0.0:1787739054.688719:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abb52 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x160:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abb59 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.688811:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739054.688814:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739054.688815:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739054.688821:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739054.692774:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739054.692777:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739054.692780:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739054.692808:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415702d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665558656:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739054.693243:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.693293:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787739054.693301:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157030c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665558784:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739054.693324:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739054.693510:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665558656 00000100:00100000:3.0:1787739054.693517:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665558656 00000100:00100000:3.0:1787739054.693544:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665558784 00000100:00100000:3.0:1787739054.693547:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665558784 00000100:00100000:3.0:1787739054.693560:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1197 00000100:00100000:3.0:1787739054.693568:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f712940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+12:3652:x1874579665558912:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739054.693581:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739054.693603:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abb3d lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abb36 expref: 12 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.693617:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235600) 00010000:00010000:3.0:1787739054.693625:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739054.694275:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1198 00000100:00100000:0.0:1787739054.694287:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415702580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+11:3650:x1874579665558656:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739054.694299:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739054.694313:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1abb75 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abb6e expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739054.694741:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1199 00000100:00100000:1.0:1787739054.694752:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415700780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+10:3650:x1874579665558784:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739054.694761:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739054.694775:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1abb59 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x160:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abb52 expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.694787:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x160:0x0].0x0 (ffffa07307064500) 00010000:00010000:1.0:1787739054.694796:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739054.696990:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb6:0x0].0x0 (ffffa0730608b500) 00010000:00010000:0.0:1787739054.697014:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739054.700744:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415700780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+9:3650:x1874579665558784:12345-0@lo:103:ldlm_bl.0 Request processed in 5984us (7408us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.700759:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1199 00000100:00100000:2.0:1787739054.701235:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074157030c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665558784:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739054.711833:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f712940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+8:3652:x1874579665558912:12345-0@lo:103:ldlm_bl.0 Request processed in 18255us (23265us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.711862:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1197 00000100:00100000:3.0:1787739054.711936:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f73840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665558912:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739054.712729:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415702580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+7:3650:x1874579665558656:12345-0@lo:103:ldlm_bl.0 Request processed in 18442us (19484us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739054.712739:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1198 00000100:00100000:3.0:1787739054.714148:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415702d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665558656:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739054.718057:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739054.718193:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652425088 00000100:00100000:1.0:1787739054.718204:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652425088 00000100:00100000:1.0:1787739054.718267:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23214 00000100:00100000:1.0:1787739054.718278:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f711680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17450:x1874579652425088:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739054.718307:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739054.718338:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abb7c lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094b95 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.718404:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1abb83 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.718448:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1abb83 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.718460:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1abb83 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.718476:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abb8a lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.718482:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abb8a lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.718492:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abb8a lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.718535:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1abb83 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.718542:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1abb83 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.718547:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1abb83 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.718553:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1abb83 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.718559:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739054.718563:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1abb83 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.718570:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1abb83 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.718581:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074331c2000) 00010000:00010000:1.0:1787739054.718584:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739054.718588:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1abb83 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.718594:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1abb83 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.718607:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abb8a lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.718615:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abb8a lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.718620:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abb8a lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.718625:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739054.718628:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abb8a lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.718634:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abb8a lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.718642:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xcf4bce4d (ffffa07433235f00) 00010000:00010000:1.0:1787739054.718644:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739054.718647:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abb8a lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.718653:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abb8a lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.718686:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abb7c lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b95 expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.718697:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074331c2000) 00010000:00010000:1.0:1787739054.718699:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739054.718702:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abb7c lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b95 expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.718711:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c800, rc 0) 00000100:00100000:1.0:1787739054.718776:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f711680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17450:x1874579652425088:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 500us (729us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739054.718783:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23214 00000100:00100000:1.0:1787739054.727511:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739054.727954:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652425216 00000100:00100000:3.0:1787739054.727963:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652425216 00000100:00100000:3.0:1787739054.728025:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23215 00000100:00100000:3.0:1787739054.728036:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f703c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17450:x1874579652425216:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739054.728057:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739054.728088:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1abb91 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094b9c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.728151:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb98 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.728164:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb98 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.728175:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb98 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739054.728223:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:3.0:1787739054.728229:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739054.728262:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1abb91 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b9c expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.728277:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb98 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b9c expref: 14 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.728285:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730608b600) 00010000:00010000:3.0:1787739054.728288:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739054.728292:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab400, rc 0) 00000100:00100000:3.0:1787739054.728993:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f703c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17450:x1874579652425216:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 956us (1486us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.729003:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23215 00000100:00100000:3.0:1787739054.740326:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739054.740450:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652425472 00000100:00100000:0.0:1787739054.740461:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652425472 00000100:00100000:0.0:1787739054.740515:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23216 00000100:00100000:0.0:1787739054.740526:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7103c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17450:x1874579652425472:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:0.0:1787739054.740635:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415703480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665559040:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.740687:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.740742:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739054.740762:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665559040 00000100:00100000:0.0:1787739054.740765:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665559040 00000100:00100000:0.0:1787739054.740794:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2636 00000100:00100000:0.0:1787739054.740801:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415701a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665559040:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.740985:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415701a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665559040:12345-0@lo:1000:mdt00_003.0 Request processed in 183us (296us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739054.740992:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2636 00000100:00100000:0.0:1787739054.741034:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415703480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665559040:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.741087:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415703480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665559168:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.741110:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.741126:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739054.741140:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665559168 00000100:00100000:0.0:1787739054.741143:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665559168 00000100:00100000:0.0:1787739054.741168:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2637 00000100:00100000:0.0:1787739054.741176:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415701680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665559168:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.741249:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415701680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665559168:12345-0@lo:1000:mdt00_003.0 Request processed in 74us (137us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739054.741256:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2637 00000100:00100000:0.0:1787739054.741280:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415703480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665559168:0@lo:1000:mdt00_003.0 00010000:00010000:0.0:1787739054.741341:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1abb9f lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.741351:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1abb9f lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.741360:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1abb9f lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.741373:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415703480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665559296:0@lo:101:mdt00_003.0 00000100:00100000:0.0:1787739054.741397:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.741434:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739054.741445:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665559296 00000100:00100000:0.0:1787739054.741449:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665559296 00000100:00100000:0.0:1787739054.741472:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2638 00000100:00100000:0.0:1787739054.741478:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07436d8dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665559296:12345-0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739054.741487:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739054.741501:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abba6 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abb9f expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.741524:0:18535:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab8f1 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b17 expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.741534:0:18535:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abba6 lrc: 4/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abb9f expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.741558:0:18535:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab8f1 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b17 expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.741572:0:18535:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab8f1 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b17 expref: 9 pid: 7944 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.741583:0:18535:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157012c0 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:18535:1874579665559424:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739054.741633:0:18535:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07406576480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739054.753341:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739054.753707:0:18535:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074157012c0 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:18535:1874579665559424:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739054.753751:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abba6 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abb9f expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.753765:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074331c2000) 00010000:00010000:2.0:1787739054.753768:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739054.753772:0:18535:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abba6 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abb9f expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.753782:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609bc00, rc 0) 00000100:00100000:1.0:1787739054.753971:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415703480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665559296:0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739054.753988:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1abb9f lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abba6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.753998:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1abb9f lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abba6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739054.754032:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07436d8dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665559296:12345-0@lo:101:mdt00_003.0 Request processed in 12369us (12448us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739054.754040:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2638 00000100:00100000:3.0:1787739054.754328:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652425600 00000100:00100000:3.0:1787739054.754335:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652425600 00010000:00010000:3.0:1787739054.754362:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab8f1 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b17 expref: 10 pid: 7944 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739054.754403:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1400 00000100:00100000:3.0:1787739054.754471:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f70b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652425600:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739054.754485:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739054.754491:0:6510:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab8f1 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b17 expref: 10 pid: 7944 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.754502:0:6510:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab8f1 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b17 expref: 10 pid: 7944 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.754523:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ab8f1 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b17 expref: 10 pid: 7944 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.754535:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074331c2000) 00010000:00010000:3.0:1787739054.754540:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abba6 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abb9f expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.754548:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abba6 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1abb9f expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.754556:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abba6 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1abb9f expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.754584:0:6510:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abba6 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abb9f expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739054.754597:0:6510:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f712d00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6510:1874579665559552:0@lo:105: 00000100:00100000:3.0:1787739054.754631:0:6510:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739054.754760:0:6510:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739054.759921:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665559552 00000100:00100000:1.0:1787739054.759935:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665559552 00000100:00100000:1.0:1787739054.760387:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1069 00000100:00100000:1.0:1787739054.760399:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6510:x1874579665559552:12345-0@lo:105: 00010000:00010000:1.0:1787739054.760438:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1abb9f lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abba6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.760452:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1abb9f lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abba6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.760460:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1abb9f lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abba6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.760469:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1abb9f lrc: 5/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1abba6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.760482:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1abb9f lrc: 6/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1abba6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.760490:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1abb9f lrc: 6/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1abba6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.760513:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609a400) 00000100:00100000:1.0:1787739054.760584:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6510:x1874579665559552:12345-0@lo:105: Request processed in 189us (5952us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.760591:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1069 00010000:00010000:0.0:1787739054.760647:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1abb9f lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abba6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.760658:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1abb9f lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abba6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739054.760681:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739054.760715:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1abbad lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.760732:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1abbad lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.760741:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1abbad lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.760758:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415703480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665559680:0@lo:101:mdt00_003.0 00000100:00100000:0.0:1787739054.760828:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.760860:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07406576480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739054.760875:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665559680 00000100:00100000:0.0:1787739054.760879:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665559680 00000100:00100000:0.0:1787739054.760906:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2639 00000100:00100000:0.0:1787739054.760912:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415701e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665559680:12345-0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739054.760925:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739054.760940:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1abbb4 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abbad expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.760960:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1abbb4 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abbad expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.760972:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1abbb4 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abbad expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.760980:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xcf4bce4d (ffffa07307064300) 00010000:00010000:0.0:1787739054.760983:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739054.760986:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a800, rc 0) 00000100:00100000:0.0:1787739054.761015:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415701e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665559680:12345-0@lo:101:mdt00_003.0 Request processed in 103us (187us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739054.761020:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2639 00000100:00100000:0.0:1787739054.761055:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415703480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665559680:0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739054.761068:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1abbad lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abbb4 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.761087:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1abbad lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abbb4 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739054.761101:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:0.0:1787739054.761130:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415703480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665559808:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.761154:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.761169:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739054.761182:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665559808 00000100:00100000:0.0:1787739054.761186:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665559808 00000100:00100000:0.0:1787739054.761211:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2640 00000100:00100000:0.0:1787739054.761217:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415701a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665559808:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.761279:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415701a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665559808:12345-0@lo:1000:mdt00_003.0 Request processed in 62us (123us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739054.761285:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2640 00000100:00100000:0.0:1787739054.761312:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415703480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665559808:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.761343:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415703480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665559936:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.761362:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.761372:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07406576480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739054.761383:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665559936 00000100:00100000:0.0:1787739054.761385:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665559936 00000100:00100000:0.0:1787739054.761403:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2641 00000100:00100000:0.0:1787739054.761408:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415701680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665559936:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.761482:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415701680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665559936:12345-0@lo:1000:mdt00_003.0 Request processed in 73us (118us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739054.761488:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2641 00000100:00100000:0.0:1787739054.761512:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415703480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665559936:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.761573:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415703480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665560064:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.761594:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.761607:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739054.761616:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665560064 00000100:00100000:0.0:1787739054.761619:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665560064 00000100:00100000:0.0:1787739054.761636:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2642 00000100:00100000:0.0:1787739054.761642:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157012c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665560064:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.761689:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157012c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665560064:12345-0@lo:1000:mdt00_003.0 Request processed in 46us (93us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739054.761695:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2642 00000100:00100000:0.0:1787739054.761720:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415703480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665560064:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.761758:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415703480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665560192:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.761778:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.761789:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739054.761800:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665560192 00000100:00100000:0.0:1787739054.761803:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665560192 00000100:00100000:0.0:1787739054.761820:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2643 00000100:00100000:0.0:1787739054.761825:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415702d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665560192:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.761864:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415702d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665560192:12345-0@lo:1000:mdt00_003.0 Request processed in 39us (86us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739054.761870:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2643 00000100:00100000:0.0:1787739054.761891:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415703480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665560192:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.761923:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415703480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665560320:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.761944:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.761956:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07406576480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739054.761965:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665560320 00000100:00100000:0.0:1787739054.761968:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665560320 00000100:00100000:0.0:1787739054.761987:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2644 00000100:00100000:0.0:1787739054.761991:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157030c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665560320:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.762043:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157030c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665560320:12345-0@lo:1000:mdt00_003.0 Request processed in 50us (97us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739054.762048:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2644 00000100:00100000:0.0:1787739054.762070:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415703480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665560320:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.762115:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415703480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665560448:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.762138:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.762151:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739054.762162:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665560448 00000100:00100000:0.0:1787739054.762165:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665560448 00000100:00100000:0.0:1787739054.762184:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2645 00000100:00100000:0.0:1787739054.762190:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157ebc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665560448:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.762234:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157ebc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665560448:12345-0@lo:1000:mdt00_003.0 Request processed in 44us (94us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739054.762239:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2645 00000100:00100000:0.0:1787739054.762265:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415703480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665560448:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.762294:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157ead00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665560576:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.762315:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.762328:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739054.762338:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665560576 00000100:00100000:0.0:1787739054.762341:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665560576 00000100:00100000:0.0:1787739054.762359:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2646 00000100:00100000:0.0:1787739054.762365:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157e92c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665560576:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739054.762410:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157e92c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665560576:12345-0@lo:1000:mdt00_003.0 Request processed in 44us (93us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739054.762437:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2646 00000100:00100000:0.0:1787739054.762462:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074157ead00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665560576:0@lo:1000:mdt00_003.0 00000040:00080000:0.0:1787739054.763202:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].348, 2168 off534952 00000040:00080000:0.0:1787739054.763280:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].347, 2368 off578384 00000004:00080000:0.0:1787739054.763286:0:7944:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07307054000 with version 663 00000004:00080000:2.0:1787739054.763878:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 663 rpc_version 663 00000020:00100000:2.0:1787739054.763914:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739054.763923:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743c382940 x1874579665560832/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739054.763942:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c382940 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665560832:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739054.763999:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739054.764021:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355eda80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739054.765553:0:6510:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f712d00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6510:1874579665559552:0@lo:105: 00010000:00010000:2.0:1787739054.765576:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739054.765639:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f70b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:2408:x1874579652425600:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 11226us (12306us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739054.765650:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1400 00000100:00100000:0.0:1787739054.768235:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665560832 00000100:00100000:0.0:1787739054.768253:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665560832 00000100:00100000:0.0:1787739054.768362:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2647 00000100:00100000:0.0:1787739054.768372:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c383840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665560832:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739054.769169:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c383840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:7740:x1874579665560832:12345-0@lo:1000:osp_up1-0.0 Request processed in 798us (5167us total) trans 4294968310 rc 0/0 00000100:00100000:0.0:1787739054.769178:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2647 00000100:00100000:3.0:1787739054.769569:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c382940 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665560832:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787739054.771953:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abbbb lrc: 3/0,1 mode: --/PW res: [0x200000402:0x14fc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.771973:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abbbb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14fc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.771984:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abbbb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14fc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.772004:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1abbc2 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.772011:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1abbc2 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.772017:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1abbc2 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.772050:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1abbc9 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.772057:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1abbc9 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.772067:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1abbc9 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.772098:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157ead00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665560960:0@lo:101:mdt00_003.0 00000100:00100000:0.0:1787739054.772158:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.772189:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739054.772347:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665560960 00000100:00100000:1.0:1787739054.772355:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665560960 00000100:00100000:1.0:1787739054.772407:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2648 00000100:00100000:1.0:1787739054.772438:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157eb0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:7944:x1874579665560960:12345-0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739054.772458:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739054.772474:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abbd0 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abbc9 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.772494:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abbd0 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abbc9 expref: 12 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.772506:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abbd0 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abbc9 expref: 12 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.772513:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb9:0x0].0x0 (ffffa07433235f00) 00010000:00010000:1.0:1787739054.772516:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739054.772520:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c800, rc 0) 00000100:00100000:1.0:1787739054.772562:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157eb0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:7944:x1874579665560960:12345-0@lo:101:mdt00_003.0 Request processed in 126us (404us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.772569:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2648 00000100:00100000:0.0:1787739054.772914:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074157ead00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665560960:0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739054.772932:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1abbc9 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abbd0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.772942:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1abbc9 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abbd0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739054.772966:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000659afa75 reply state 00000000ced30451 transno 4294983319 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1abbc2 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.772977:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1abbc2 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.772983:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb9:0x0].0x0 (ffffa07307064f00) 00010000:00010000:0.0:1787739054.772985:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739054.772988:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739054.772993:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1abbc9 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abbd0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.773001:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1abbc9 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abbd0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739054.773010:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xb9:0x0] 00000004:00010000:0.0:1787739054.773016:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000659afa75 reply state 00000000ced30451 transno 4294983319 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abbbb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14fc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.773023:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abbbb lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x14fc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.773030:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14fc:0x0].0x0 (ffffa07307064000) 00010000:00010000:0.0:1787739054.773032:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739054.773034:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787739054.773038:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14fc:0x0]: returning size 4096 00000004:00080000:0.0:1787739054.773046:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739054.773054:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1abb9f lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abba6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.773060:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1abb9f lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abba6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.773066:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1abb9f lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abba6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.773141:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739054.773147:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739054.773152:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1abbad lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abbb4 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.773159:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1abbad lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abbb4 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.773231:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7103c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17450:x1874579652425472:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 32706us (32919us total) trans 4294983319 rc 0/0 00000100:00100000:0.0:1787739054.773238:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23216 00010000:00010000:1.0:1787739054.787465:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739054.787480:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1abb9f lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1abba6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.787494:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739054.787501:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1abb9f lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abba6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739054.787512:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1abb9f lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abba6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.787649:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1abb9f lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abba6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.787657:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739054.787692:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1abb9f lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abba6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.787700:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a400/0x7ee4453e4a1abb9f lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abba6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.787730:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739054.787732:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739054.787734:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739054.787824:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787739054.787855:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157ead00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665561088:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739054.787923:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.788051:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665561088 00000100:00100000:0.0:1787739054.788058:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665561088 00000100:00100000:0.0:1787739054.788119:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1200 00000100:00100000:0.0:1787739054.788129:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c380f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+12:3652:x1874579665561088:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739054.788141:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739054.788159:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abba6 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abb9f expref: 12 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.788172:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074331c2000) 00010000:00010000:0.0:1787739054.788182:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739054.788220:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c380f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+11:3652:x1874579665561088:12345-0@lo:103:ldlm_bl.0 Request processed in 93us (297us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739054.788227:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1200 00000100:00100000:0.0:1787739054.788265:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074157ead00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665561088:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739054.840060:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739054.840280:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652425856 00000100:00100000:3.0:1787739054.840291:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652425856 00000100:00100000:3.0:1787739054.840359:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23217 00000100:00100000:3.0:1787739054.840369:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f70000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17451:x1874579652425856:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:3.0:1787739054.840554:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f70000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17451:x1874579652425856:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 186us (503us total) trans 0 rc -17/-17 00000100:00100000:3.0:1787739054.840561:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23217 00000100:00100000:3.0:1787739054.866476:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739054.884621:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652425984 00000100:00100000:2.0:1787739054.884636:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652425984 00000100:00100000:2.0:1787739054.884763:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23218 00000100:00100000:2.0:1787739054.884772:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f711a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17453:x1874579652425984:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739054.884791:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739054.884813:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1abbd7 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094ba3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.885015:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abbde lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.885029:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abbde lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.885039:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abbde lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.885051:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1abbe5 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.885061:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1abbb4 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abbad expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.885070:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1abbe5 lrc: 4/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.885111:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1abbb4 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1abbad expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739054.885126:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c383480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665561216:0@lo:104: 00000100:00100000:2.0:1787739054.885196:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739054.885253:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355ed780 going to sleep for 11 seconds 00000100:00100000:1.0:1787739054.885498:0:6507:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787739054.885511:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665561216 00000100:00100000:1.0:1787739054.885515:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665561216 00000100:00100000:1.0:1787739054.885553:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1070 00000100:00100000:1.0:1787739054.885559:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f4d850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6524:x1874579665561216:12345-0@lo:104: 00010000:00010000:1.0:1787739054.885570:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1abbad lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1abbb4 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.885617:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787739054.885622:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f4d850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6524:x1874579665561216:12345-0@lo:104: Request processed in 64us (427us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.885628:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1070 00000100:00100000:1.0:1787739054.885681:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c383480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665561216:0@lo:104: 00010000:00010000:1.0:1787739054.885698:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1abbe5 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.885712:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739054.885728:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1abbad lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1abbb4 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.885736:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609be00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739054.885740:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1abbad lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1abbb4 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739054.885749:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1abbad lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1abbb4 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.885772:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1abbad lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abbb4 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.885778:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739054.885796:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1abbad lrc: 1/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abbb4 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.885802:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1abbad lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abbb4 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.885816:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739054.885818:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739054.885820:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739054.885826:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787739054.885834:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f712c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665561344:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739054.885874:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.886031:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665561344 00000100:00100000:0.0:1787739054.886037:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665561344 00010000:00010000:0.0:1787739054.886059:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1abbb4 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1abbad expref: 11 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739054.886095:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1401 00000100:00100000:0.0:1787739054.886102:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f730c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+11:3651:x1874579665561344:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739054.886111:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739054.890820:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1abb44 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abb4b expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739054.890840:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1abb44 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abb4b expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.890904:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1abb44 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abb4b expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.890912:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739054.890937:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1abb44 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abb4b expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787739054.890957:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968310 is committed 00000001:00080000:0.0:1787739054.890974:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968309 is committed 00000020:00080000:0.0:1787739054.890981:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 339 batchid updated 00000020:00080000:0.0:1787739054.890992:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07305fcc6c0 refcount 2 committed 0 result 0 batchid 340 00000020:00080000:0.0:1787739054.890995:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88900 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07305fccf00 00000020:00080000:0.0:1787739054.890998:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].333 00000020:00080000:0.0:1787739054.891000:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88800 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304011e00 00000020:00080000:0.0:1787739054.891003:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].333 00000020:00080000:0.0:1787739054.891013:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 338 batchid updated 00010000:00010000:0.0:1787739054.891084:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1abbb4 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1abbad expref: 7 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.891098:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xcf4bce4d (ffffa07307064300) 00010000:00010000:0.0:1787739054.891103:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1abbe5 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.891112:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1abbe5 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.891120:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1abbe5 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.891144:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739054.891211:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f730c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+6:3651:x1874579665561344:12345-0@lo:103:ldlm_bl.0 Request processed in 5108us (5334us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739054.891220:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1401 00000100:00100000:3.0:1787739054.891566:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787739054.891587:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157ead00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665561472:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:3.0:1787739054.891629:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787739054.891692:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abad4 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x160:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.891701:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abad4 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x160:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.891709:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abad4 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x160:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:1.0:1787739054.891816:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 340 committed batchid 339 00080000:02000000:1.0:1787739054.891838:0:7475:0:(osd_handler.c:2218:osd_trans_start()) *** cfs_fail_loc=19a, val=0*** 00000004:00080000:1.0:1787739054.891908:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa073068b4900 with version 637 00000020:00080000:1.0:1787739054.891918:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 338 cancel update log [0x200000401:0x3:0x0].331: rc = 0 00000020:00080000:1.0:1787739054.891981:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 338 cancel update log [0x240000400:0x3:0x0].331: rc = 0 00000004:00080000:1.0:1787739054.892003:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa073068b4540 with version 638 00000020:00080000:1.0:1787739054.892006:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 339 cancel update log [0x200000401:0x3:0x0].332: rc = 0 00000020:00080000:1.0:1787739054.892024:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 339 cancel update log [0x240000400:0x3:0x0].332: rc = 0 00000020:00080000:1.0:1787739054.892031:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 340 committed batchid 339 00000004:00080000:1.0:1787739054.892052:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 637 rpc_version 637 00000020:00100000:1.0:1787739054.892085:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739054.892092:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303f72580 x1874579665561728/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739054.892109:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f72580 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665561728:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739054.892143:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739054.892163:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4240 going to sleep for 11 seconds 00000100:00100000:1.0:1787739054.892181:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665561728 00000100:00100000:1.0:1787739054.892188:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665561728 00000100:00100000:1.0:1787739054.892233:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2649 00000100:00100000:1.0:1787739054.892241:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f71680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665561728:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739054.892373:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f71680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665561728:12345-0@lo:1000:osp_up0-1.0 Request processed in 133us (228us total) trans 4294983320 rc 0/0 00000100:00100000:1.0:1787739054.892380:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2649 00010000:00010000:1.0:1787739054.892408:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1abbe5 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.892445:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1abbe5 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739054.892519:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f73840 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665561856:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739054.892545:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739054.892560:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b43c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739054.892574:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665561856 00000100:00100000:1.0:1787739054.892577:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665561856 00000100:00100000:1.0:1787739054.892604:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2650 00000100:00100000:1.0:1787739054.892610:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f71a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6524:x1874579665561856:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739054.892663:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f71a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6524:x1874579665561856:12345-0@lo:1000:mdt00_001.0 Request processed in 53us (117us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.892670:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2650 00000100:00100000:1.0:1787739054.892711:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f712c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665561344:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739054.892767:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f72580 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665561728:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787739054.892775:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 638 rpc_version 638 00000020:00100000:1.0:1787739054.892786:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739054.892791:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303f712c0 x1874579665562112/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739054.892805:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f712c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665562112:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739054.892830:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739054.892845:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739054.892862:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f73840 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665561856:0@lo:1000:mdt00_001.0 00010000:00010000:1.0:1787739054.892897:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1abbec lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14fc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.892908:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1abbec lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14fc:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.892917:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1abbec lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14fc:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739054.892936:0:6524:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1abbec lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14fc:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.892949:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abbde lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.892956:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abbde lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.892961:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abbde lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.892967:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abbde lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.892973:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739054.892976:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abbde lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.892983:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abbde lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.892993:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074419d5200) 00010000:00010000:1.0:1787739054.892996:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739054.893000:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abbde lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.893006:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abbde lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.893020:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1abbe5 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.893025:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1abbe5 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.893031:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1abbe5 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.893035:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739054.893038:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1abbe5 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.893044:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1abbe5 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.893050:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xcf4bce4d (ffffa07307064300) 00010000:00010000:1.0:1787739054.893053:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739054.893056:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1abbe5 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.893061:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1abbe5 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.893107:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1abbd7 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ba3 expref: 9 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.893125:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1abbec lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14fc:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ba3 expref: 8 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739054.893133:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14fc:0x0].0x0 (ffffa07433235b00) 00010000:00010000:1.0:1787739054.893135:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739054.893137:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1de00, rc 0) 00000100:00100000:1.0:1787739054.893179:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f711a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17453:x1874579652425984:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 8408us (26713us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739054.893186:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23218 00000100:00100000:1.0:1787739054.893307:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665562112 00000100:00100000:1.0:1787739054.893311:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665562112 00000100:00100000:1.0:1787739054.893332:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2651 00000100:00100000:1.0:1787739054.893337:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f721c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665562112:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739054.893434:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f721c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665562112:12345-0@lo:1000:osp_up0-1.0 Request processed in 96us (603us total) trans 4294983321 rc 0/0 00000100:00100000:1.0:1787739054.893441:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2651 00000100:00100000:1.0:1787739054.893470:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f712c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665562112:0@lo:1000:osp_up0-1.0 00010000:00010000:3.0:1787739054.894502:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739054.894513:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abad4 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x160:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.894524:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abad4 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x160:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.894540:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x160:0x0].0x0 (ffffa073068a9000) 00010000:00010000:3.0:1787739054.894543:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739054.894548:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abad4 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x160:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.894555:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abad4 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x160:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.894571:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1abadb lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.894578:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1abadb lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.894584:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1abadb lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.894594:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739054.894596:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1abadb lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.894602:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1abadb lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.894608:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb6:0x0].0x0 (ffffa073068a9400) 00010000:00010000:3.0:1787739054.894611:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739054.894613:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1abadb lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.894618:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1abadb lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xb6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739054.894649:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665561472 00000100:00100000:3.0:1787739054.894655:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665561472 00000100:00100000:3.0:1787739054.894702:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1201 00000100:00100000:3.0:1787739054.894710:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba1050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+9:3650:x1874579665561472:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:3.0:1787739054.895446:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739054.895468:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1abb4b lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xef411af6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abb44 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.895482:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xef411af6 (ffffa0730608bd00) 00010000:00010000:3.0:1787739054.895489:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739054.895565:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba1050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+8:3650:x1874579665561472:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 856us (3935us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.895572:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1201 00000100:00100000:0.0:1787739054.898568:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074157ead00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665561472:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787739054.907673:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739054.919193:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652426112 00000100:00100000:0.0:1787739054.919208:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652426112 00000100:00100000:0.0:1787739054.919280:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23219 00000100:00100000:0.0:1787739054.919291:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f73840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17453:x1874579652426112:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739054.919315:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739054.919351:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abbf3 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x14fc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094baa expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.919406:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1abbfa lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14fc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.919433:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1abbfa lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14fc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.919444:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1abbfa lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14fc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739054.919527:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14fc:0x0]: returning size 4096 00000004:00200000:0.0:1787739054.919533:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739054.919564:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abbf3 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x14fc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094baa expref: 18 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.919581:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1abbfa lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14fc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094baa expref: 17 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.919589:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14fc:0x0].0x0 (ffffa07307064000) 00010000:00010000:0.0:1787739054.919592:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739054.919595:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a400, rc 0) 00000100:00100000:0.0:1787739054.919653:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f73840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17453:x1874579652426112:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 364us (11990us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739054.919661:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23219 00000100:00100000:1.0:1787739054.927647:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739054.927957:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652426240 00000100:00100000:2.0:1787739054.927966:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652426240 00000100:00100000:2.0:1787739054.928028:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23220 00000100:00100000:2.0:1787739054.928039:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f70f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17453:x1874579652426240:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739054.928059:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739054.928089:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1abc01 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094bb1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.928147:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abc08 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.928160:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abc08 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.928171:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abc08 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739054.928214:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xb9:0x0]: returning size 4096 00000004:00200000:2.0:1787739054.928220:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739054.928253:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1abc01 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bb1 expref: 19 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.928277:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abc08 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bb1 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.928285:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb9:0x0].0x0 (ffffa07307064f00) 00010000:00010000:2.0:1787739054.928287:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739054.928291:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633dc00, rc 0) 00000100:00100000:2.0:1787739054.928350:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f70f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17453:x1874579652426240:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 313us (711us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739054.928357:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23220 00000100:00100000:2.0:1787739054.939803:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739054.939997:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652426368 00000100:00100000:3.0:1787739054.940007:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652426368 00000100:00100000:3.0:1787739054.940066:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23221 00000100:00100000:3.0:1787739054.940084:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c3803c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17453:x1874579652426368:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739054.940103:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739054.940123:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1abc0f lrc: 2/0,0 mode: --/CR res: [0x240000401:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094bb8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.940185:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abc16 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.940199:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abbd0 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abbc9 expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.940208:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abc16 lrc: 4/1,0 mode: --/PR res: [0x240000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.940243:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abbd0 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1abbc9 expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739054.940259:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f712940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665562240:0@lo:104: 00000100:00100000:3.0:1787739054.940306:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739054.940360:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739054.940486:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665562240 00000100:00100000:2.0:1787739054.940490:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665562240 00000100:00100000:2.0:1787739054.940528:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1071 00000100:00100000:2.0:1787739054.940536:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6524:x1874579665562240:12345-0@lo:104: 00010000:00010000:2.0:1787739054.940548:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1abbc9 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1abbd0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.940598:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739054.940604:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6524:x1874579665562240:12345-0@lo:104: Request processed in 70us (297us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739054.940611:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1071 00010000:00010000:2.0:1787739054.940632:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739054.940638:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1abbc9 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1abbd0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.940647:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739054.940653:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1abbc9 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xb9:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1abbd0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739054.940661:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1abbc9 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xb9:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1abbd0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.940685:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1abbc9 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xb9:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abbd0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.940692:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739054.940712:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1abbc9 lrc: 1/0,0 mode: --/PW res: [0x240000401:0xb9:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abbd0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.940923:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1abbc9 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xb9:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abbd0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.940943:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739054.940945:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739054.940947:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739054.940960:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c381680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665562368:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739054.940990:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739054.941045:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f712940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665562240:0@lo:104: 00010000:00010000:3.0:1787739054.941063:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abc16 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739054.951286:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665562368 00000100:00100000:1.0:1787739054.951300:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665562368 00010000:00010000:2.0:1787739054.964575:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abbd0 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1abbc9 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739054.964634:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1402 00000100:00100000:2.0:1787739054.964644:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c3821c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+6:3652:x1874579665562368:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739054.964660:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00080000:0.0:1787739054.971460:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 340 batchid updated 00000020:00080000:1.0:1787739054.971492:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 340 00000004:00080000:1.0:1787739054.971549:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa073068b4300 with version 639 00000020:00080000:1.0:1787739054.971568:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 340 cancel update log [0x200000401:0x3:0x0].333: rc = 0 00000004:00080000:0.0:1787739054.971623:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 639 rpc_version 639 00000020:00080000:1.0:1787739054.971640:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 340 cancel update log [0x240000400:0x3:0x0].333: rc = 0 00000020:00080000:1.0:1787739054.971648:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 340 00000020:00100000:0.0:1787739054.971655:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739054.971666:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074157ead00 x1874579665562624/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739054.971690:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157ead00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665562624:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739054.971764:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739054.971789:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e240 going to sleep for 11 seconds 00010000:00010000:2.0:1787739054.971886:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abbd0 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1abbc9 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.971909:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb9:0x0].0x0 (ffffa07433235f00) 00010000:00010000:2.0:1787739054.971914:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abc16 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.971924:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abc16 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.971932:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abc16 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739054.971967:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739054.972010:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c3821c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+5:3652:x1874579665562368:12345-0@lo:103:ldlm_bl.0 Request processed in 7368us (31018us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739054.972020:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1402 00000100:00100000:2.0:1787739054.972064:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c381680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665562368:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739054.972154:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abc16 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.972164:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abc16 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739054.972190:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665562624 00000100:00100000:1.0:1787739054.972196:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665562624 00000004:00200000:0.0:1787739054.972216:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xb9:0x0]: returning size 4096 00000004:00200000:0.0:1787739054.972221:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000100:00100000:1.0:1787739054.972239:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2652 00000100:00100000:1.0:1787739054.972247:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157ea940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665562624:12345-0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1787739054.972261:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1abc0f lrc: 0/0,0 mode: --/CR res: [0x240000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bb8 expref: 9 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.972281:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abc16 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bb8 expref: 8 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739054.972289:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb9:0x0].0x0 (ffffa07433235f00) 00010000:00010000:0.0:1787739054.972292:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739054.972294:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa400, rc 0) 00000100:00100000:0.0:1787739054.972337:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c3803c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17453:x1874579652426368:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 32262us (32543us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739054.972344:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23221 00000100:00100000:1.0:1787739054.972619:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157ea940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665562624:12345-0@lo:1000:osp_up0-1.0 Request processed in 372us (854us total) trans 4294983322 rc 0/0 00000100:00100000:1.0:1787739054.972625:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2652 00000100:00100000:0.0:1787739054.972633:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074157ead00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665562624:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739054.984225:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739054.984257:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652426496 00000100:00100000:3.0:1787739054.984266:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652426496 00000100:00100000:3.0:1787739054.984333:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1202 00000100:00100000:3.0:1787739054.984342:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157eb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2409:x1874579652426496:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:3.0:1787739054.984357:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739054.984380:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1abbec lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14fc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ba3 expref: 8 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.984396:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14fc:0x0].0x0 (ffffa07433235b00) 00010000:00010000:3.0:1787739054.984406:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739054.984493:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157eb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2409:x1874579652426496:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 152us (276us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.984501:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1202 00000100:00100000:3.0:1787739054.997542:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739054.997574:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652426752 00000100:00100000:3.0:1787739054.997582:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652426752 00000100:00100000:3.0:1787739054.997631:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23222 00000100:00100000:3.0:1787739054.997642:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f712940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17453:x1874579652426752:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:3.0:1787739054.997693:0:6523:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739054.997718:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1abbfa lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14fc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094baa expref: 18 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.997735:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14fc:0x0].0x0 (ffffa07307064000) 00010000:00010000:3.0:1787739054.997740:0:6523:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739054.997786:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1abc1d lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.997795:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1abc1d lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.997811:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1abc1d lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739054.997833:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f7130c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665562752:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739054.997880:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739054.997918:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739054.997937:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665562752 00000100:00100000:3.0:1787739054.997941:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665562752 00000100:00100000:3.0:1787739054.998051:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2653 00000100:00100000:3.0:1787739054.998058:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f711e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+5:6523:x1874579665562752:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739054.998068:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739054.998091:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1abc24 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abc1d expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.998112:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1abc24 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abc1d expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.998124:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1abc24 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abc1d expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.998132:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305d56f00) 00010000:00010000:3.0:1787739054.998134:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739054.998137:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa800, rc 0) 00000100:00100000:3.0:1787739054.998176:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f711e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6523:x1874579665562752:12345-0@lo:101:mdt00_000.0 Request processed in 119us (295us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.998183:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2653 00000100:00100000:3.0:1787739054.998217:0:6523:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968310 gen 2 00000004:00080000:3.0:1787739054.998223:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968305 committed transno 4294968310 00000020:00080000:3.0:1787739054.998229:0:6523:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa0730610e840 refcount 2 committed 0 result 0 batchid 349 00000020:00080000:3.0:1787739054.998232:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730415f900 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730610e900 00000020:00080000:3.0:1787739054.998235:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].346 00000020:00080000:3.0:1787739054.998238:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730415f100 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa0730541e400 00000020:00080000:3.0:1787739054.998240:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].347 00000004:00080000:3.0:1787739054.998260:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968307 committed transno 4294968310 00000004:00080000:3.0:1787739054.998269:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968308 committed transno 4294968310 00000004:00080000:3.0:1787739054.998279:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968310 committed transno 4294968310 00000100:00100000:3.0:1787739054.998298:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f7130c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665562752:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739054.998312:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1abc1d lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abc24 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.998322:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1abc1d lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abc24 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739054.998336:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739054.998355:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1abc2b lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.998363:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1abc2b lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.998370:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1abc2b lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739054.998380:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f7130c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665562880:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739054.998401:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739054.998439:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc480 going to sleep for 11 seconds 00000020:00080000:3.0:1787739054.998449:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 349 committed batchid 348 00000020:00080000:3.0:1787739054.998503:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 348 00000100:00100000:3.0:1787739054.998512:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665562880 00000100:00100000:3.0:1787739054.998516:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665562880 00000100:00100000:3.0:1787739054.998538:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2654 00000100:00100000:3.0:1787739054.998544:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7112c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6523:x1874579665562880:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739054.998552:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739054.998563:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1abc32 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abc2b expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.998577:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1abc32 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abc2b expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.998588:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1abc32 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abc2b expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.998595:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xcf4bce4d (ffffa07305d56b00) 00010000:00010000:3.0:1787739054.998597:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739054.998600:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa200, rc 0) 00000100:00100000:3.0:1787739054.998628:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7112c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665562880:12345-0@lo:101:mdt00_000.0 Request processed in 83us (225us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.998634:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2654 00000100:00100000:3.0:1787739054.998662:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f7130c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665562880:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739054.998672:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1abc2b lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abc32 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739054.998680:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1abc2b lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abc32 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739054.998696:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:3.0:1787739054.998725:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f7130c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665563008:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739054.998749:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739054.998765:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc780 going to sleep for 11 seconds 00000100:00100000:3.0:1787739054.998776:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665563008 00000100:00100000:3.0:1787739054.998779:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665563008 00000100:00100000:3.0:1787739054.998799:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2655 00000100:00100000:3.0:1787739054.998806:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665563008:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739054.998861:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665563008:12345-0@lo:1000:mdt00_000.0 Request processed in 57us (111us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739054.998868:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2655 00000100:00100000:3.0:1787739054.998892:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f7130c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665563008:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739054.998924:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f7130c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665563136:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739054.998944:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.003827:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc780 going to sleep for 10 seconds 00000100:00100000:3.0:1787739055.003873:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665563136 00000100:00100000:3.0:1787739055.003882:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665563136 00000100:00100000:3.0:1787739055.004876:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2656 00000100:00100000:3.0:1787739055.004891:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f712580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665563136:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739055.005165:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f712580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665563136:12345-0@lo:1000:mdt00_000.0 Request processed in 278us (6218us total) trans 0 rc 1/1 00000100:00100000:3.0:1787739055.005173:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2656 00000100:00100000:2.0:1787739055.005333:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f7130c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665563136:0@lo:1000:mdt00_000.0 00010000:00010000:2.0:1787739055.005406:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1abc39 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.005434:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1abc39 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.005444:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1abc39 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.005462:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c381680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665563264:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1787739055.005500:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.021060:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355ed780 going to sleep for 11 seconds 00000100:00100000:1.0:1787739055.022535:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665563264 00000100:00100000:1.0:1787739055.022550:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665563264 00000100:00100000:1.0:1787739055.022637:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2657 00000100:00100000:1.0:1787739055.022647:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c3812c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665563264:12345-0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739055.022667:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739055.022691:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1abc40 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abc39 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.022723:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1abc40 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abc39 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.022737:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1abc40 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abc39 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.022746:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14fc:0x0].0x0 (ffffa07433235300) 00010000:00010000:1.0:1787739055.022749:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739055.022753:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d600, rc 0) 00000100:00100000:1.0:1787739055.022830:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c3812c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6523:x1874579665563264:12345-0@lo:101:mdt00_000.0 Request processed in 184us (17329us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.022837:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2657 00000100:00100000:1.0:1787739055.025110:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c381680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665563264:0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739055.025138:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1abc39 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abc40 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.025153:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1abc39 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abc40 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.025189:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abc47 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.025200:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abc47 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.025210:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abc47 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.025228:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abc4e lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.025239:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abc08 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bb1 expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.025249:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abc4e lrc: 4/0,1 mode: --/EX res: [0x200000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.025284:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abc08 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bb1 expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.025300:0:6523:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abc08 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bb1 expref: 17 pid: 13793 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739055.025313:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f70b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665563392:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739055.025405:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.036449:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739055.036545:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652426880 00000100:00100000:3.0:1787739055.036554:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652426880 00010000:00010000:3.0:1787739055.036614:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abc08 lrc: 5/0,0 mode: PR/PR res: [0x200000400:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bb1 expref: 18 pid: 13793 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739055.036653:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1403 00000100:00100000:3.0:1787739055.036662:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c3830c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2408:x1874579652426880:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739055.036677:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739055.036684:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abc08 lrc: 5/0,0 mode: PR/PR res: [0x200000400:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bb1 expref: 18 pid: 13793 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.036698:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abc08 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bb1 expref: 18 pid: 13793 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.036722:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb9:0x0].0x0 (ffffa07307064f00) 00010000:00010000:3.0:1787739055.036728:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abc4e lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.036738:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abc4e lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.036745:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abc4e lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.036763:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739055.036814:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c3830c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2408:x1874579652426880:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 152us (375us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.036822:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1403 00000100:00080000:2.0:1787739055.037716:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1787739055.041807:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abc08 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bb1 expref: 17 pid: 13793 timeout: 741 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739055.041833:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f70b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665563392:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739055.042039:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abc4e lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.042050:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abc4e lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.042056:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abc4e lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.042109:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1abc55 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.042117:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1abc55 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.042128:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1abc55 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739055.042145:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f70b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665563520:0@lo:101:mdt00_000.0 00000100:00100000:1.0:1787739055.042208:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739055.042292:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.048525:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665563520 00000100:00100000:2.0:1787739055.048541:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665563520 00000100:00100000:0.0:1787739055.048817:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2658 00000100:00100000:0.0:1787739055.048829:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f730c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6523:x1874579665563520:12345-0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739055.048859:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739055.048882:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1abc5c lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abc55 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.048912:0:8447:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abc16 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bb8 expref: 6 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.048923:0:8447:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1abc5c lrc: 4/0,0 mode: --/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abc55 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.049026:0:8447:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abc16 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bb8 expref: 6 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.049047:0:8447:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abc16 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bb8 expref: 6 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.049061:0:8447:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157e9a40 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8447:1874579665563648:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739055.049161:0:8447:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e240 going to sleep for 11 seconds 00000100:00100000:1.0:1787739055.063176:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739055.063216:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652427008 00000100:00100000:1.0:1787739055.063225:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652427008 00010000:00010000:1.0:1787739055.063276:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abc16 lrc: 5/0,0 mode: PR/PR res: [0x240000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bb8 expref: 7 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739055.063316:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1404 00000100:00100000:1.0:1787739055.063326:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c416940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652427008:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739055.063341:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739055.063348:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abc16 lrc: 5/0,0 mode: PR/PR res: [0x240000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bb8 expref: 7 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.063361:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abc16 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bb8 expref: 7 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.063386:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb9:0x0].0x0 (ffffa07433235f00) 00010000:00010000:1.0:1787739055.063392:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1abc5c lrc: 3/0,0 mode: --/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abc55 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.063402:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1abc5c lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1abc55 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.063427:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1abc5c lrc: 4/0,0 mode: EX/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1abc55 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.063457:0:10340:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1abc5c lrc: 4/0,0 mode: EX/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abc55 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739055.063470:0:10340:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c4170c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665563776:0@lo:105: 00000100:00100000:1.0:1787739055.063510:0:10340:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739055.063543:0:10340:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539a80 going to sleep for 11 seconds 00000100:00080000:1.0:1787739055.063558:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:3.0:1787739055.063655:0:8447:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abc16 lrc: 0/0,0 mode: --/PR res: [0x240000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bb8 expref: 7 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739055.063670:0:8447:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074157e9a40 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8447:1874579665563648:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739055.063694:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1abc5c lrc: 4/0,0 mode: EX/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abc55 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.063703:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb9:0x0].0x0 (ffffa07433235f00) 00010000:00010000:3.0:1787739055.063705:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739055.063708:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a400, rc 0) 00000100:00100000:3.0:1787739055.063764:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f730c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6523:x1874579665563520:12345-0@lo:101:mdt00_000.0 Request processed in 14938us (21556us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.063771:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2658 00000100:00100000:1.0:1787739055.063979:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f70b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665563520:0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739055.063995:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1abc55 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abc5c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.064005:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1abc55 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abc5c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.064538:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665563776 00000100:00100000:0.0:1787739055.064545:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665563776 00000100:00100000:0.0:1787739055.064585:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1072 00000100:00100000:0.0:1787739055.064593:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c4152c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665563776:12345-0@lo:105: 00010000:00010000:0.0:1787739055.064603:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1abc55 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abc5c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.064613:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1abc55 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abc5c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.064622:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1abc55 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abc5c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.064629:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1abc55 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1abc5c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.064637:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1abc55 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1abc5c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.064645:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1abc55 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1abc5c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.064678:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07441a1c000) 00000100:00100000:0.0:1787739055.064731:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c4152c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665563776:12345-0@lo:105: Request processed in 139us (1220us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.064738:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1072 00010000:00010000:3.0:1787739055.064852:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1abc55 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abc5c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.064863:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1abc55 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abc5c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739055.065547:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665563904:0@lo:403:mdt00_000.0 00000100:00100000:3.0:1787739055.065598:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.065631:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739055.065959:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665563904 00000100:00100000:0.0:1787739055.065966:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665563904 00000100:00100000:0.0:1787739055.066007:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2659 00000100:00100000:0.0:1787739055.066015:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6523:x1874579665563904:12345-0@lo:403:mdt00_000.0 00000100:00100000:0.0:1787739055.067665:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6523:x1874579665563904:12345-0@lo:403:mdt00_000.0 Request processed in 1645us (2060us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.067677:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2659 00000100:00100000:0.0:1787739055.067745:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665563904:0@lo:403:mdt00_000.0 00000040:00080000:0.0:1787739055.068626:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].349, 664 off535616 00000040:00080000:0.0:1787739055.068794:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].348, 744 off579128 00000004:00080000:0.0:1787739055.068801:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730610e3c0 with version 664 00000004:00080000:0.0:1787739055.068826:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 664 rpc_version 664 00000020:00100000:0.0:1787739055.068855:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739055.068864:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074157e9e00 x1874579665564160/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2064/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739055.068884:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157e9e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665564160:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739055.068929:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739055.068951:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739055.070838:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665564160 00000100:00100000:3.0:1787739055.070861:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665564160 00000100:00100000:3.0:1787739055.070917:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2660 00000100:00100000:3.0:1787739055.070927:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157e9a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665564160:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739055.071375:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157e9a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665564160:12345-0@lo:1000:osp_up1-0.0 Request processed in 449us (2443us total) trans 4294968311 rc 0/0 00000100:00100000:3.0:1787739055.071385:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2660 00000100:00100000:1.0:1787739055.071789:0:10340:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c4170c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665563776:0@lo:105: 00010000:00010000:1.0:1787739055.071813:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739055.071873:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c416940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2408:x1874579652427008:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 8546us (8703us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.071883:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1404 00000100:00100000:0.0:1787739055.076375:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074157e9e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665564160:0@lo:1000:osp_up1-0.0 00000004:00200000:1.0:1787739055.076536:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14fc:0x0]: returning size 4096 00010000:00010000:1.0:1787739055.076560:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abc4e lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076569:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abc4e lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076577:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abc4e lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076583:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739055.076587:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abc4e lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076595:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abc4e lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076613:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb9:0x0].0x0 (ffffa07307064f00) 00010000:00010000:1.0:1787739055.076616:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739055.076621:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abc4e lrc: 2/0,0 mode: --/EX res: [0x200000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076628:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abc4e lrc: 0/0,0 mode: --/EX res: [0x200000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076640:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1abc55 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abc5c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076649:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1abc55 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abc5c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076656:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1abc55 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abc5c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076683:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739055.076692:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xb9:0x0] 00010000:00010000:1.0:1787739055.076699:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abc47 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076706:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abc47 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076712:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abc47 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076728:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739055.076731:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abc47 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076738:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abc47 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076746:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14fc:0x0].0x0 (ffffa07307064000) 00010000:00010000:1.0:1787739055.076748:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739055.076751:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abc47 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076756:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abc47 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076766:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1abc39 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abc40 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076772:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1abc39 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abc40 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076778:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1abc39 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abc40 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076790:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:1.0:1787739055.076800:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739055.076805:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1abc1d lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abc24 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076812:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1abc1d lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abc24 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076819:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1abc1d lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abc24 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076835:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:1.0:1787739055.076839:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739055.076855:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1abc2b lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abc32 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076863:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1abc2b lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abc32 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739055.076940:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f712940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17453:x1874579652426752:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 79297us (79402us total) trans 4294983323 rc 0/0 00000100:00100000:1.0:1787739055.076949:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23222 00010000:00010000:1.0:1787739055.076987:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1787739055.076992:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1abc55 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1abc5c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.076999:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739055.077003:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1abc55 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abc5c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739055.077011:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1abc55 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abc5c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.077040:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1abc55 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abc5c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.077046:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739055.077066:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1abc55 lrc: 1/0,0 mode: --/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abc5c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.077083:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1abc55 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abc5c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.077099:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1787739055.077102:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1abc39 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1abc40 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.077108:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739055.077111:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1abc39 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abc40 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739055.077118:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1abc39 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abc40 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.077134:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1abc39 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abc40 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.077140:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739055.077154:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1abc39 lrc: 1/0,0 mode: --/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abc40 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.077160:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d400/0x7ee4453e4a1abc39 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abc40 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.077172:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739055.077176:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1abc1d lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1abc24 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.077187:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739055.077190:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1abc1d lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abc24 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739055.077198:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1abc1d lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abc24 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.077212:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1abc1d lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abc24 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.077218:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739055.077223:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1abc1d lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abc24 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.077230:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1abc1d lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abc24 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.077241:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739055.077243:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739055.077245:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739055.077365:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c4170c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665564288:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739055.077438:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739055.077482:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c414b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665564544:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739055.077506:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739055.077524:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787739055.077530:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c416580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665564416:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739055.077551:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739055.077579:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665564288 00000100:00100000:1.0:1787739055.077584:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665564288 00000100:00100000:1.0:1787739055.077619:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665564544 00000100:00100000:1.0:1787739055.077622:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665564544 00000100:00100000:1.0:1787739055.077637:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665564416 00000100:00100000:1.0:1787739055.077640:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665564416 00000100:00100000:1.0:1787739055.077654:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1203 00000100:00100000:1.0:1787739055.077661:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c415e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+12:3653:x1874579665564288:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739055.077674:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739055.077690:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1abc5c lrc: 0/0,0 mode: --/EX res: [0x240000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abc55 expref: 12 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.077702:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb9:0x0].0x0 (ffffa07433235f00) 00010000:00010000:1.0:1787739055.077711:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739055.077738:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c415e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+11:3653:x1874579665564288:12345-0@lo:103:ldlm_bl.0 Request processed in 78us (300us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.077745:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1203 00000100:00100000:1.0:1787739055.077758:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1204 00000100:00100000:1.0:1787739055.077763:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c416d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+10:3651:x1874579665564544:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739055.077771:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739055.077781:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1abc24 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abc1d expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.077790:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305d56f00) 00010000:00010000:1.0:1787739055.077796:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739055.077819:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c416d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+9:3651:x1874579665564544:12345-0@lo:103:ldlm_bl.0 Request processed in 57us (313us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.077825:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1204 00000100:00100000:1.0:1787739055.077837:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1205 00000100:00100000:1.0:1787739055.077843:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba3450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+8:3651:x1874579665564416:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739055.077851:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739055.077860:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1abc40 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14fc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abc39 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.077869:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14fc:0x0].0x0 (ffffa07433235300) 00010000:00010000:1.0:1787739055.077874:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739055.077894:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba3450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+7:3651:x1874579665564416:12345-0@lo:103:ldlm_bl.0 Request processed in 52us (342us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.077900:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1205 00000100:00100000:1.0:1787739055.077949:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c416580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665564416:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739055.077959:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c414b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665564544:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739055.077985:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c4170c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665564288:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739055.079306:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739055.080562:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739055.158517:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739055.158554:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652427136 00000100:00100000:0.0:1787739055.158563:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652427136 00000100:00100000:0.0:1787739055.158609:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23223 00000100:00100000:0.0:1787739055.158619:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157eb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17462:x1874579652427136:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739055.158641:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739055.158662:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abc63 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094bbf expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.158735:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abc6a lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.158751:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abc6a lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.158764:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abc6a lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.158778:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abc71 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.158785:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abc71 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.158792:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abc71 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.158854:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abc6a lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.158863:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abc6a lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.158869:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abc6a lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.158875:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abc6a lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.158881:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609bc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739055.158885:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abc6a lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.158892:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abc6a lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.158904:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730608b600) 00010000:00010000:0.0:1787739055.158907:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739055.158913:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abc6a lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.158919:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abc6a lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.158931:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abc71 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.158937:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abc71 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.158943:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abc71 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.158949:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739055.158952:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abc71 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.159313:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abc71 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.159323:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x4dc682e0 (ffffa07307064d00) 00010000:00010000:0.0:1787739055.159338:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739055.159342:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abc71 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.159348:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abc71 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.159384:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abc63 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bbf expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.159394:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730608b600) 00010000:00010000:0.0:1787739055.159397:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739055.159401:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abc63 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bbf expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.159409:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a800, rc 0) 00000100:00100000:0.0:1787739055.159493:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157eb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17462:x1874579652427136:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 875us (984us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739055.159501:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23223 00000100:00100000:1.0:1787739055.164871:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739055.165829:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652427264 00000100:00100000:0.0:1787739055.165851:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652427264 00000100:00100000:0.0:1787739055.165916:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23224 00000100:00100000:0.0:1787739055.165927:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c4170c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17462:x1874579652427264:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739055.165946:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739055.166093:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1abc78 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094bc6 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.166157:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1abc7f lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.166172:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1abc7f lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.166183:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1abc7f lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739055.166227:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x5:0x0]: returning size 4096 00000004:00200000:0.0:1787739055.166232:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739055.166268:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1abc78 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bc6 expref: 8 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.166285:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1abc7f lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bc6 expref: 7 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.166294:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064300) 00010000:00010000:0.0:1787739055.166297:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739055.166301:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b600, rc 0) 00000100:00100000:0.0:1787739055.166447:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c4170c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17462:x1874579652427264:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 521us (1584us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.166455:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23224 00000100:00100000:3.0:1787739055.188268:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739055.188306:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652427520 00000100:00100000:3.0:1787739055.188315:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652427520 00000100:00100000:3.0:1787739055.188386:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23225 00000100:00100000:3.0:1787739055.188396:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7121c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17462:x1874579652427520:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:3.0:1787739055.188520:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665564672:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739055.188567:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.188615:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fccc00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739055.190467:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665564672 00000100:00100000:0.0:1787739055.190483:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665564672 00000100:00100000:0.0:1787739055.190539:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2661 00000100:00100000:0.0:1787739055.190549:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7130c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665564672:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739055.190668:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7130c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665564672:12345-0@lo:1000:mdt00_000.0 Request processed in 121us (2099us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.190676:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2661 00000100:00100000:0.0:1787739055.190793:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665564672:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739055.190937:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415700780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665564800:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739055.191046:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739055.191083:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ecc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739055.191440:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665564800 00000100:00100000:3.0:1787739055.191445:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665564800 00000100:00100000:3.0:1787739055.191479:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2662 00000100:00100000:3.0:1787739055.191486:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415702580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665564800:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739055.191571:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415702580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665564800:12345-0@lo:1000:mdt00_000.0 Request processed in 85us (524us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739055.191578:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2662 00000100:00100000:0.0:1787739055.191732:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415700780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665564800:0@lo:1000:mdt00_000.0 00010000:00010000:0.0:1787739055.191789:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1abc86 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.191799:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1abc86 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.191809:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1abc86 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.191822:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415700780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665564928:0@lo:101:mdt00_000.0 00000100:00100000:0.0:1787739055.191862:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739055.191885:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739055.191897:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665564928 00000100:00100000:0.0:1787739055.191900:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665564928 00000100:00100000:0.0:1787739055.191926:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2663 00000100:00100000:0.0:1787739055.191932:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415703480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665564928:12345-0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739055.191941:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739055.191965:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1abc8d lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abc86 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.191991:0:8448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb98 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b9c expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.192001:0:8448:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1abc8d lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abc86 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.192025:0:8448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb98 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b9c expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.192038:0:8448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb98 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b9c expref: 16 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.192048:0:8448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8448:1874579665565056:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739055.192109:0:8448:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ecc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739055.209167:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739055.212617:0:8448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8448:1874579665565056:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739055.212653:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1abc8d lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abc86 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.212667:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730608b600) 00010000:00010000:1.0:1787739055.212670:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739055.212674:0:8448:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1abc8d lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abc86 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.212685:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609ac00, rc 0) 00000100:00100000:1.0:1787739055.212774:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415703480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665564928:12345-0@lo:101:mdt00_000.0 Request processed in 20841us (20910us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.212783:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2663 00000100:00100000:1.0:1787739055.212835:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415700780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665564928:0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739055.212852:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1abc86 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abc8d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.212863:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1abc86 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abc8d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.213175:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652427648 00000100:00100000:2.0:1787739055.213182:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652427648 00010000:00010000:2.0:1787739055.213216:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb98 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b9c expref: 17 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.213254:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1405 00000100:00100000:2.0:1787739055.213264:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652427648:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739055.213277:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739055.213284:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb98 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b9c expref: 17 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.213296:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb98 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b9c expref: 17 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.213316:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1abb98 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094b9c expref: 17 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.213329:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730608b600) 00010000:00010000:2.0:1787739055.213335:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1abc8d lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abc86 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.213345:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1abc8d lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1abc86 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.213359:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1abc8d lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1abc86 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.213389:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1abc8d lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abc86 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.213403:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c3821c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665565184:0@lo:105: 00000100:00100000:2.0:1787739055.213464:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.213492:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355eda80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739055.213561:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665565184 00000100:00100000:1.0:1787739055.213566:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665565184 00000100:00100000:1.0:1787739055.213600:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1073 00000100:00100000:1.0:1787739055.213607:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c3830c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665565184:12345-0@lo:105: 00010000:00010000:1.0:1787739055.213618:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1abc86 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abc8d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.213627:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1abc86 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abc8d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.213636:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1abc86 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abc8d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.213644:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1abc86 lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1abc8d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.213650:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1abc86 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1abc8d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.213663:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1abc86 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1abc8d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.213681:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609a000) 00000100:00100000:1.0:1787739055.213729:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c3830c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665565184:12345-0@lo:105: Request processed in 110us (253us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.213736:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1073 00000100:00100000:1.0:1787739055.213762:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c3821c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665565184:0@lo:105: 00010000:00010000:1.0:1787739055.213777:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739055.213814:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:2409:x1874579652427648:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 552us (4658us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.213822:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1405 00010000:00010000:1.0:1787739055.213937:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1abc86 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abc8d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.213946:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1abc86 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abc8d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739055.214191:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739055.214224:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1abc94 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.214232:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1abc94 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.214241:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1abc94 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739055.214258:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c414b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665565312:0@lo:101:mdt00_000.0 00000100:00100000:1.0:1787739055.214287:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739055.214312:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.214362:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665565312 00000100:00100000:2.0:1787739055.214367:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665565312 00000100:00100000:2.0:1787739055.214404:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2664 00000100:00100000:2.0:1787739055.214434:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c416580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665565312:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739055.214449:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739055.214464:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abc9b lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abc94 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.214484:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abc9b lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abc94 expref: 11 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.214495:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abc9b lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abc94 expref: 11 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.214501:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x4dc682e0 (ffffa074419d5d00) 00010000:00010000:2.0:1787739055.214504:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739055.214506:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d800, rc 0) 00000100:00100000:2.0:1787739055.214537:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c416580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665565312:12345-0@lo:101:mdt00_000.0 Request processed in 127us (250us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739055.214543:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2664 00000100:00100000:1.0:1787739055.214615:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c414b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665565312:0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739055.214626:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1abc94 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abc9b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.214635:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1abc94 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abc9b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739055.214646:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:1.0:1787739055.214677:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c414b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665565440:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739055.214703:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739055.215163:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.215894:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665565440 00000100:00100000:2.0:1787739055.215903:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665565440 00000100:00100000:2.0:1787739055.216056:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2665 00000100:00100000:2.0:1787739055.216067:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c416940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665565440:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739055.216169:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c416940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665565440:12345-0@lo:1000:mdt00_000.0 Request processed in 104us (1464us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.219665:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c414b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665565440:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739055.219754:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c414b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665565568:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739055.219810:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739055.219839:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073070540c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739055.219856:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665565568 00000100:00100000:1.0:1787739055.219861:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665565568 00000100:00100000:2.0:1787739055.220099:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2665 00000100:00100000:1.0:1787739055.221552:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2666 00000100:00100000:1.0:1787739055.221566:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c415e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665565568:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739055.221692:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c415e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665565568:12345-0@lo:1000:mdt00_000.0 Request processed in 130us (1882us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739055.221700:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2666 00000100:00100000:2.0:1787739055.222866:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c414b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665565568:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739055.222981:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c381680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665565696:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739055.223028:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.223710:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355ed540 going to sleep for 11 seconds 00000100:00100000:0.0:1787739055.223787:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665565696 00000100:00100000:0.0:1787739055.223794:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665565696 00000100:00100000:0.0:1787739055.224161:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2667 00000100:00100000:0.0:1787739055.224172:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665565696:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739055.224269:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665565696:12345-0@lo:1000:mdt00_000.0 Request processed in 99us (1240us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739055.224277:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2667 00000100:00100000:3.0:1787739055.224439:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c381680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665565696:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739055.224509:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f713480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665565824:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739055.224540:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.224693:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fccf00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739055.228726:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665565824 00000100:00100000:1.0:1787739055.228743:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665565824 00000100:00100000:1.0:1787739055.228801:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2668 00000100:00100000:1.0:1787739055.228810:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f712d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665565824:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739055.228949:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f712d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665565824:12345-0@lo:1000:mdt00_000.0 Request processed in 140us (4407us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.228963:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2668 00000100:00100000:3.0:1787739055.232529:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f713480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665565824:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739055.232615:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f713480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665565952:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739055.232677:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.232727:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fccf00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739055.233359:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665565952 00000100:00100000:0.0:1787739055.233366:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665565952 00000100:00100000:0.0:1787739055.233446:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2669 00000100:00100000:0.0:1787739055.233457:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665565952:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739055.233563:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665565952:12345-0@lo:1000:mdt00_000.0 Request processed in 107us (885us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.233570:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2669 00000100:00100000:3.0:1787739055.233679:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f713480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665565952:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739055.233743:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f713480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665566080:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739055.233773:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.233794:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fccf00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739055.233926:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665566080 00000100:00100000:0.0:1787739055.233930:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665566080 00000100:00100000:0.0:1787739055.234022:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2670 00000100:00100000:0.0:1787739055.234032:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f73c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665566080:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739055.234132:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f73c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665566080:12345-0@lo:1000:mdt00_000.0 Request processed in 102us (357us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.234140:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2670 00000100:00100000:3.0:1787739055.237098:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f713480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665566080:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739055.237177:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f70b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665566208:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739055.237232:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.237278:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fccf00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739055.237941:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665566208 00000100:00100000:0.0:1787739055.237949:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665566208 00000100:00100000:0.0:1787739055.238067:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2671 00000100:00100000:0.0:1787739055.238088:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665566208:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739055.238175:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665566208:12345-0@lo:1000:mdt00_000.0 Request processed in 89us (942us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.238182:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2671 00000100:00100000:3.0:1787739055.238290:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f70b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665566208:0@lo:1000:mdt00_000.0 00000040:00080000:3.0:1787739055.240571:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].335, 2168 off511256 00000040:00080000:3.0:1787739055.240658:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].336, 2368 off559424 00000004:00080000:3.0:1787739055.240664:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07305fcc540 with version 640 00000004:00080000:0.0:1787739055.240742:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 640 rpc_version 640 00000020:00100000:0.0:1787739055.240781:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739055.240791:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07305f7a580 x1874579665566464/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739055.240813:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7a580 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665566464:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739055.241064:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739055.241111:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.241186:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665566464 00000100:00100000:2.0:1787739055.241194:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665566464 00000100:00100000:2.0:1787739055.241245:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2672 00000100:00100000:2.0:1787739055.241254:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f783c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:7467:x1874579665566464:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739055.243222:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f783c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:7467:x1874579665566464:12345-0@lo:1000:osp_up0-1.0 Request processed in 1964us (2153us total) trans 4294983324 rc 0/0 00000100:00100000:2.0:1787739055.243235:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2672 00000100:00100000:0.0:1787739055.243394:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7a580 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665566464:0@lo:1000:osp_up0-1.0 00010000:00010000:3.0:1787739055.243622:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abca2 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x162:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.243638:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abca2 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x162:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.243650:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abca2 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x162:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.243668:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1abca9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.243676:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1abca9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.243682:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1abca9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.243711:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abcb0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.243816:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abcb0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.243826:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abcb0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739055.244022:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f70b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665566592:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739055.244061:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.245094:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739055.248189:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665566592 00000100:00100000:0.0:1787739055.248203:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665566592 00000100:00100000:1.0:1787739055.248376:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2673 00000100:00100000:1.0:1787739055.248386:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:6523:x1874579665566592:12345-0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739055.248410:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739055.248456:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abcb7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abcb0 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.248486:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abcb7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abcb0 expref: 13 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.248500:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abcb7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abcb0 expref: 13 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.248509:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb7:0x0].0x0 (ffffa07433235f00) 00010000:00010000:1.0:1787739055.248512:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739055.248516:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1dc00, rc 0) 00000100:00100000:1.0:1787739055.248586:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+13:6523:x1874579665566592:12345-0@lo:101:mdt00_000.0 Request processed in 202us (4525us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.248593:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2673 00000100:00100000:3.0:1787739055.248761:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f70b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665566592:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739055.248779:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abcb0 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abcb7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.248790:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abcb0 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abcb7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739055.248815:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000eb2c6bc8 reply state 00000000c11df1d8 transno 4294968312 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1abca9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.248827:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1abca9 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.248834:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb7:0x0].0x0 (ffffa07305d56800) 00010000:00010000:3.0:1787739055.248836:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739055.249027:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787739055.249037:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abcb0 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abcb7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.249045:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abcb0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abcb7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739055.249058:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xb7:0x0] 00000004:00010000:3.0:1787739055.249064:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000eb2c6bc8 reply state 00000000c11df1d8 transno 4294968312 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abca2 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x162:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.249083:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abca2 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x162:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.249089:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x162:0x0].0x0 (ffffa07305d56600) 00010000:00010000:3.0:1787739055.249092:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739055.249094:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:3.0:1787739055.249099:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x162:0x0]: returning size 4096 00000004:00080000:3.0:1787739055.249107:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739055.249116:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1abc86 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abc8d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.249124:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1abc86 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abc8d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.249131:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1abc86 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abc8d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.249153:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739055.249157:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739055.249161:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1abc94 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abc9b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.249168:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1abc94 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abc9b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739055.249243:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7121c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17462:x1874579652427520:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 60847us (60982us total) trans 4294968312 rc 0/0 00000100:00100000:3.0:1787739055.249251:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23225 00010000:00010000:1.0:1787739055.253240:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739055.253255:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1abc86 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1abc8d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.253274:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739055.253282:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1abc86 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abc8d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739055.253291:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1abc86 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abc8d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.253347:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1abc86 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abc8d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.253354:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739055.253381:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1abc86 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abc8d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.253389:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1abc86 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abc8d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.253430:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739055.253433:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739055.253435:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739055.253491:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787739055.253514:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c416d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665566720:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739055.253580:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739055.253856:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665566720 00000100:00100000:1.0:1787739055.253863:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665566720 00000100:00100000:1.0:1787739055.253922:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1206 00000100:00100000:1.0:1787739055.253932:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f79680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+13:3653:x1874579665566720:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739055.253945:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739055.254121:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1abc8d lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abc86 expref: 13 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.254139:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730608b600) 00010000:00010000:1.0:1787739055.254149:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739055.254197:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f79680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+12:3653:x1874579665566720:12345-0@lo:103:ldlm_bl.0 Request processed in 266us (616us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.254204:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1206 00000100:00100000:0.0:1787739055.254283:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c416d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665566720:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739055.292506:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739055.292602:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652427904 00000100:00100000:0.0:1787739055.292612:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652427904 00000100:00100000:0.0:1787739055.292780:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23226 00000100:00100000:0.0:1787739055.292791:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be43c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17463:x1874579652427904:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:0.0:1787739055.292944:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be43c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17463:x1874579652427904:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 154us (447us total) trans 0 rc -17/-17 00000100:00100000:0.0:1787739055.292952:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23226 00000100:00100000:0.0:1787739055.335041:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739055.335131:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652428032 00000100:00100000:3.0:1787739055.335140:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652428032 00000100:00100000:3.0:1787739055.335198:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23227 00000100:00100000:3.0:1787739055.335209:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17464:x1874579652428032:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739055.335230:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739055.335258:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1abcbe lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094bcd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.335338:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1abcc5 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.335353:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1abcc5 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.335364:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1abcc5 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.335377:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1abccc lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.335388:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abc9b lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abc94 expref: 11 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.335396:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1abccc lrc: 4/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.335448:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abc9b lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1abc94 expref: 11 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739055.335463:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be7480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665566848:0@lo:104: 00000100:00100000:3.0:1787739055.335508:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.335563:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc780 going to sleep for 11 seconds 00000100:00100000:3.0:1787739055.335581:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665566848 00000100:00100000:3.0:1787739055.335585:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665566848 00000100:00100000:3.0:1787739055.335612:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1074 00000100:00100000:3.0:1787739055.335619:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:13793:x1874579665566848:12345-0@lo:104: 00010000:00010000:3.0:1787739055.335631:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1abc94 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1abc9b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.335681:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739055.335687:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:13793:x1874579665566848:12345-0@lo:104: Request processed in 69us (178us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.335693:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1074 00010000:00010000:3.0:1787739055.335836:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739055.335912:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1abc94 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1abc9b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.335921:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1da00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739055.335926:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1abc94 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1abc9b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739055.335934:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1abc94 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1abc9b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.335965:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1abc94 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abc9b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.335972:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739055.335991:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1abc94 lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abc9b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.335998:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1da00/0x7ee4453e4a1abc94 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abc9b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.336017:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739055.336018:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739055.336020:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739055.336028:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787739055.336038:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665566976:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739055.336067:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.336119:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be7480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665566848:0@lo:104: 00010000:00010000:3.0:1787739055.336138:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1abccc lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.336339:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665566976 00000100:00100000:0.0:1787739055.336345:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665566976 00010000:00010000:0.0:1787739055.336367:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abc9b lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1abc94 expref: 12 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.336397:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1406 00000100:00100000:0.0:1787739055.336404:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+12:3652:x1874579665566976:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739055.336432:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739055.342919:0:6502:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1abc2b lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abc32 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739055.342941:0:6502:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1abc2b lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abc32 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.342990:0:6502:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1abc2b lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abc32 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.342997:0:6502:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739055.343034:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1abc2b lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abc32 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787739055.343055:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983324 is committed 00000001:00080000:0.0:1787739055.343080:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983323 is committed 00000020:00080000:0.0:1787739055.343089:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 349 batchid updated 00000020:00080000:0.0:1787739055.343104:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07406576480 refcount 2 committed 0 result 0 batchid 350 00000020:00080000:0.0:1787739055.343108:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88f00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07307054000 00000020:00080000:0.0:1787739055.343111:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].347 00000020:00080000:0.0:1787739055.343113:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88c00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306963400 00000020:00080000:0.0:1787739055.343116:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].348 00000100:00100000:0.0:1787739055.343135:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787739055.343154:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665567104:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:0.0:1787739055.343219:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787739055.343268:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abbbb lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x14fc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.343278:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abbbb lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x14fc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.343284:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abbbb lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x14fc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.343291:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739055.343296:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abbbb lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x14fc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.343303:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abbbb lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x14fc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.343315:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14fc:0x0].0x0 (ffffa07307064000) 00010000:00010000:0.0:1787739055.343318:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739055.343322:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abbbb lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x14fc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.343328:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abbbb lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x14fc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.343344:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1abbc2 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.343350:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1abbc2 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.343356:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1abbc2 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.343361:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739055.343363:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1abbc2 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.343369:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1abbc2 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.343376:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb9:0x0].0x0 (ffffa07307064f00) 00010000:00010000:0.0:1787739055.343378:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739055.343381:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1abbc2 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.343387:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1abbc2 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:0.0:1787739055.343407:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 350 committed batchid 349 00000004:00080000:0.0:1787739055.343533:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730610e300 with version 665 00000020:00080000:0.0:1787739055.343544:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 349 cancel update log [0x240000401:0x3:0x0].346: rc = 0 00000020:00080000:0.0:1787739055.343586:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 349 cancel update log [0x200000400:0x3:0x0].347: rc = 0 00000020:00080000:0.0:1787739055.343593:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 349 00000100:00100000:0.0:1787739055.343608:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665567104 00000100:00100000:0.0:1787739055.343614:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665567104 00000100:00100000:0.0:1787739055.343661:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1207 00000100:00100000:0.0:1787739055.343670:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f79a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+7:3651:x1874579665567104:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:0.0:1787739055.343682:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739055.343698:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1abc32 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xcf4bce4d bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abc2b expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.343709:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xcf4bce4d (ffffa07305d56b00) 00010000:00010000:0.0:1787739055.343715:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739055.344517:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f79a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+6:3651:x1874579665567104:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 846us (1296us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.344528:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1207 00000004:00080000:0.0:1787739055.344557:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 665 rpc_version 665 00000020:00100000:0.0:1787739055.344581:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739055.344589:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07305f78f00 x1874579665567360/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739055.344610:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f78f00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665567360:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739055.344644:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.344731:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665567104:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:0.0:1787739055.346881:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e840 going to sleep for 11 seconds 00010000:00010000:0.0:1787739055.346963:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abc9b lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1abc94 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.346982:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x4dc682e0 (ffffa074419d5d00) 00010000:00010000:0.0:1787739055.346988:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1abccc lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.346997:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1abccc lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.347004:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1abccc lrc: 4/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.347044:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739055.347108:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+6:3652:x1874579665566976:12345-0@lo:103:ldlm_bl.0 Request processed in 10703us (11039us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.347118:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1406 00000100:00100000:0.0:1787739055.347180:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665566976:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739055.347743:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1abccc lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.347753:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1abccc lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.347877:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665567488:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.347921:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.347949:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355eda80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739055.348449:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665567360 00000100:00100000:1.0:1787739055.348458:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665567360 00000100:00100000:1.0:1787739055.348497:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665567488 00000100:00100000:1.0:1787739055.348499:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665567488 00000100:00100000:1.0:1787739055.348520:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2674 00000100:00100000:1.0:1787739055.348529:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665567360:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739055.348686:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665567360:12345-0@lo:1000:osp_up1-0.0 Request processed in 158us (4041us total) trans 4294968313 rc 0/0 00000100:00100000:1.0:1787739055.348693:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2674 00000100:00100000:1.0:1787739055.348708:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2675 00000100:00100000:1.0:1787739055.348726:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c380f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665567488:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739055.348774:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c380f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665567488:12345-0@lo:1000:mdt00_004.0 Request processed in 59us (852us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.348780:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2675 00000100:00100000:1.0:1787739055.348825:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f78f00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665567360:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739055.349011:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665567488:0@lo:1000:mdt00_004.0 00010000:00010000:2.0:1787739055.349059:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1abcd3 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x162:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.349081:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1abcd3 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x162:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.349092:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1abcd3 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x162:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739055.349114:0:13793:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1abcd3 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x162:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.349128:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1abcc5 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.349136:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1abcc5 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.349142:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1abcc5 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.349148:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1abcc5 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.349154:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739055.349158:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1abcc5 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.349166:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1abcc5 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.349177:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305d56500) 00010000:00010000:2.0:1787739055.349180:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739055.349184:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1abcc5 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.349190:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1abcc5 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.349203:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1abccc lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.349209:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1abccc lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.349214:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1abccc lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.349219:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739055.349221:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1abccc lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.349227:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1abccc lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.349234:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x4dc682e0 (ffffa074419d5d00) 00010000:00010000:2.0:1787739055.349237:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739055.349240:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1abccc lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.349249:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1abccc lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.349283:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1abcbe lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bcd expref: 15 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.349302:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1abcd3 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x162:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bcd expref: 14 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.349310:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x162:0x0].0x0 (ffffa074419d5600) 00010000:00010000:2.0:1787739055.349313:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739055.349315:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633cc00, rc 0) 00000100:00100000:2.0:1787739055.349368:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17464:x1874579652428032:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 14161us (14336us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739055.349376:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23227 00000100:00100000:1.0:1787739055.360813:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739055.360889:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652428160 00000100:00100000:3.0:1787739055.360897:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652428160 00000100:00100000:3.0:1787739055.360958:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23228 00000100:00100000:3.0:1787739055.360969:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c415680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17464:x1874579652428160:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739055.360993:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739055.361012:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1abcda lrc: 2/0,0 mode: --/CR res: [0x240000402:0x162:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094bd4 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.361082:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1abce1 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x162:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.361096:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1abce1 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x162:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.361108:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1abce1 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x162:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739055.361195:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x162:0x0]: returning size 4096 00000004:00200000:3.0:1787739055.361200:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739055.361235:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1abcda lrc: 0/0,0 mode: --/CR res: [0x240000402:0x162:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bd4 expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.361252:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1abce1 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x162:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bd4 expref: 10 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.361262:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x162:0x0].0x0 (ffffa07305d56600) 00010000:00010000:3.0:1787739055.361265:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739055.361269:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa000, rc 0) 00000100:00100000:3.0:1787739055.361335:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c415680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17464:x1874579652428160:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 368us (534us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.361343:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23228 00000100:00100000:1.0:1787739055.365653:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739055.365748:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652428288 00000100:00100000:3.0:1787739055.365757:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652428288 00000100:00100000:3.0:1787739055.365812:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23229 00000100:00100000:3.0:1787739055.365823:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c417480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17464:x1874579652428288:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739055.365854:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739055.365874:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1abce8 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094bdb expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.365928:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1abcef lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.365941:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1abcef lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.365952:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1abcef lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739055.365997:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xb7:0x0]: returning size 4096 00000004:00200000:3.0:1787739055.366003:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739055.366035:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1abce8 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bdb expref: 12 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.366053:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1abcef lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bdb expref: 11 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.366061:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb7:0x0].0x0 (ffffa07305d56800) 00010000:00010000:3.0:1787739055.366064:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739055.366067:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa600, rc 0) 00000100:00100000:3.0:1787739055.366133:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c417480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17464:x1874579652428288:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 312us (487us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.366140:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23229 00000100:00100000:1.0:1787739055.371066:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739055.371165:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652428416 00000100:00100000:2.0:1787739055.371175:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652428416 00000100:00100000:2.0:1787739055.371241:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23230 00000100:00100000:2.0:1787739055.371251:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c4161c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17464:x1874579652428416:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739055.371271:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739055.371293:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1abcf6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094be2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.371366:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1abcfd lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.371380:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abcb7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abcb0 expref: 5 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.371390:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1abcfd lrc: 4/1,0 mode: --/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.371461:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abcb7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1abcb0 expref: 5 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.371477:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665567616:0@lo:104: 00000100:00100000:2.0:1787739055.371525:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.371560:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739055.371634:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665567616 00000100:00100000:0.0:1787739055.371639:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665567616 00000100:00100000:0.0:1787739055.371681:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1075 00000100:00100000:0.0:1787739055.371689:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c383480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:13793:x1874579665567616:12345-0@lo:104: 00010000:00010000:0.0:1787739055.371702:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abcb0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1abcb7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.371773:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787739055.371779:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c383480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:13793:x1874579665567616:12345-0@lo:104: Request processed in 92us (254us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.371786:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1075 00010000:00010000:0.0:1787739055.371808:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739055.371814:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abcb0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1abcb7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.371823:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aae00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739055.371828:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abcb0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1abcb7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739055.371837:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abcb0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1abcb7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.371861:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abcb0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abcb7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.371869:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739055.371890:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abcb0 lrc: 1/0,0 mode: --/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abcb7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.371898:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abcb0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abcb7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.371916:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739055.371918:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739055.371920:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739055.371933:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f792c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665567744:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739055.371963:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.372054:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665567616:0@lo:104: 00010000:00010000:2.0:1787739055.372083:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1abcfd lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739055.372409:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665567744 00000100:00100000:1.0:1787739055.372444:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665567744 00010000:00010000:1.0:1787739055.372468:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abcb7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1abcb0 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739055.372495:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1407 00000100:00100000:1.0:1787739055.372502:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f78b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+6:3651:x1874579665567744:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739055.372513:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00080000:0.0:1787739055.375631:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 350 batchid updated 00000020:00080000:3.0:1787739055.375706:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 350 00000004:00080000:3.0:1787739055.375774:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07305fcc780 with version 666 00000020:00080000:3.0:1787739055.375792:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 350 cancel update log [0x240000401:0x3:0x0].347: rc = 0 00000020:00080000:3.0:1787739055.375873:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 350 cancel update log [0x200000400:0x3:0x0].348: rc = 0 00000020:00080000:3.0:1787739055.375881:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 350 00000004:00080000:1.0:1787739055.375923:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 666 rpc_version 666 00000020:00100000:1.0:1787739055.375956:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739055.375966:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743c416d00 x1874579665568000/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739055.375988:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c416d00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665568000:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739055.376052:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739055.376098:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054e40 going to sleep for 11 seconds 00010000:00010000:2.0:1787739055.376290:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abcb7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1abcb0 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.376307:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb7:0x0].0x0 (ffffa07433235f00) 00010000:00010000:2.0:1787739055.376312:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1abcfd lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.376321:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1abcfd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.376329:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1abcfd lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.376358:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739055.376407:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f78b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665567744:12345-0@lo:103:ldlm_bl.0 Request processed in 3906us (4444us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739055.376445:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1407 00000100:00100000:1.0:1787739055.376540:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f792c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665567744:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739055.376606:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665568000 00000100:00100000:3.0:1787739055.376612:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665568000 00000100:00100000:3.0:1787739055.376658:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2676 00000100:00100000:3.0:1787739055.376667:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c414b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665568000:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739055.376823:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c414b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665568000:12345-0@lo:1000:osp_up1-0.0 Request processed in 157us (771us total) trans 4294968314 rc 0/0 00000100:00100000:3.0:1787739055.376831:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2676 00000100:00100000:2.0:1787739055.377042:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c416d00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665568000:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787739055.377654:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1abcfd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.377665:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1abcfd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739055.377728:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb7:0x0]: returning size 4096 00000004:00200000:0.0:1787739055.377732:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739055.377777:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1abcf6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094be2 expref: 16 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.377795:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1abcfd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094be2 expref: 15 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.377802:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb7:0x0].0x0 (ffffa07433235f00) 00010000:00010000:0.0:1787739055.377805:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739055.377808:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c000, rc 0) 00000100:00100000:0.0:1787739055.377985:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c4161c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17464:x1874579652428416:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 6734us (6927us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.377994:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23230 00000100:00100000:1.0:1787739055.383781:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739055.383825:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739055.383916:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652428800 00000100:00100000:2.0:1787739055.383925:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652428800 00000100:00100000:2.0:1787739055.383981:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23231 00000100:00100000:2.0:1787739055.383990:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f52c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17464:x1874579652428800:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:2.0:1787739055.384043:0:7944:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739055.384068:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1abce1 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x162:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bd4 expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.384093:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x162:0x0].0x0 (ffffa07305d56600) 00010000:00010000:2.0:1787739055.384097:0:7944:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739055.384144:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1abd04 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.384153:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1abd04 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.384162:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1abd04 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.384183:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665568128:0@lo:101:mdt00_003.0 00000100:00100000:2.0:1787739055.384228:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.384261:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739055.384310:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652428544 00000100:00100000:0.0:1787739055.384315:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652428544 00000100:00100000:0.0:1787739055.384358:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1208 00000100:00100000:0.0:1787739055.384366:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415700780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2409:x1874579652428544:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:0.0:1787739055.384376:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739055.384389:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1abcd3 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x162:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bcd expref: 15 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.384401:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x162:0x0].0x0 (ffffa074419d5600) 00010000:00010000:0.0:1787739055.384434:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739055.384474:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415700780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652428544:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 108us (702us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.384481:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1208 00000100:00100000:1.0:1787739055.384777:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665568128 00000100:00100000:1.0:1787739055.384782:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665568128 00000100:00100000:1.0:1787739055.384817:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2677 00000100:00100000:1.0:1787739055.384824:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c380780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+5:7944:x1874579665568128:12345-0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739055.385046:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739055.385064:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abd0b lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abd04 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.385097:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abd0b lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abd04 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.385116:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abd0b lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abd04 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.385123:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235e00) 00010000:00010000:1.0:1787739055.385126:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739055.385128:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1cc00, rc 0) 00000100:00100000:1.0:1787739055.385176:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c380780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:7944:x1874579665568128:12345-0@lo:101:mdt00_003.0 Request processed in 353us (948us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.385183:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2677 00000100:00100000:0.0:1787739055.385380:0:7944:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983324 gen 2 00000004:00080000:0.0:1787739055.385387:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983318 committed transno 4294983324 00000020:00080000:0.0:1787739055.385392:0:7944:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07307054540 refcount 2 committed 0 result 0 batchid 341 00000020:00080000:0.0:1787739055.385396:0:7944:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c695a00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07307054240 00000020:00080000:0.0:1787739055.385399:0:7944:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].334 00000020:00080000:0.0:1787739055.385401:0:7944:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c695b00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073040e9e00 00000020:00080000:0.0:1787739055.385404:0:7944:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].334 00000004:00080000:0.0:1787739055.385465:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983320 committed transno 4294983324 00000004:00080000:0.0:1787739055.385476:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983321 committed transno 4294983324 00000004:00080000:0.0:1787739055.385485:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983322 committed transno 4294983324 00000004:00080000:0.0:1787739055.385504:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983324 committed transno 4294983324 00000100:00100000:0.0:1787739055.385536:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665568128:0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739055.385549:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1abd04 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abd0b expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.385560:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1abd04 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abd0b expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739055.385577:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739055.385603:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1abd12 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.385610:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1abd12 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.385622:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1abd12 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.385637:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7a580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665568256:0@lo:101:mdt00_003.0 00000100:00100000:0.0:1787739055.385665:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739055.385681:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4300 going to sleep for 11 seconds 00000100:00100000:3.0:1787739055.385744:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665568256 00000100:00100000:3.0:1787739055.385749:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665568256 00000100:00100000:3.0:1787739055.385784:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2678 00000100:00100000:3.0:1787739055.385791:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:7944:x1874579665568256:12345-0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739055.385802:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739055.385814:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1abd19 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abd12 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.385830:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1abd19 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abd12 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.385840:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1abd19 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abd12 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.385858:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x4dc682e0 (ffffa07305d56300) 00010000:00010000:3.0:1787739055.385861:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739055.385863:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739055.385909:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665568256:12345-0@lo:101:mdt00_003.0 Request processed in 119us (243us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.385915:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2678 00000100:00100000:2.0:1787739055.386080:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7a580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665568256:0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739055.386092:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1abd12 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abd19 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.386105:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1abd12 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abd19 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739055.386116:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:2.0:1787739055.386147:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c381a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665568384:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739055.386174:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.386196:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739055.386239:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665568384 00000100:00100000:0.0:1787739055.386243:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665568384 00000100:00100000:0.0:1787739055.386275:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2679 00000100:00100000:0.0:1787739055.386281:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c380000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665568384:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739055.386993:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c380000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665568384:12345-0@lo:1000:mdt00_003.0 Request processed in 709us (815us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.387003:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2679 00000100:00100000:2.0:1787739055.387144:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c381a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665568384:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739055.387192:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c381a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665568512:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739055.387223:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.387250:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739055.387681:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665568512 00000100:00100000:0.0:1787739055.387688:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665568512 00000100:00100000:0.0:1787739055.387735:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2680 00000100:00100000:0.0:1787739055.387743:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c381680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665568512:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739055.387835:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c381680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665568512:12345-0@lo:1000:mdt00_003.0 Request processed in 92us (610us total) trans 0 rc 1/1 00000100:00100000:0.0:1787739055.387853:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2680 00000020:00080000:1.0:1787739055.387982:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 341 committed batchid 340 00000020:00080000:1.0:1787739055.388035:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 340 00000100:00100000:2.0:1787739055.388300:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c381a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665568512:0@lo:1000:mdt00_003.0 00010000:00010000:2.0:1787739055.388352:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1abd20 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x162:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.388362:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1abd20 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x162:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.388371:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1abd20 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x162:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.388385:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c381a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665568640:0@lo:101:mdt00_003.0 00000100:00100000:2.0:1787739055.388438:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.388456:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068bac00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739055.388521:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665568640 00000100:00100000:0.0:1787739055.388525:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665568640 00000100:00100000:0.0:1787739055.388553:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2681 00000100:00100000:0.0:1787739055.388560:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c3821c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665568640:12345-0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739055.388569:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739055.388584:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abd27 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x162:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abd20 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.388607:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abd27 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x162:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abd20 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.388620:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abd27 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x162:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abd20 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.388628:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x162:0x0].0x0 (ffffa073068a9100) 00010000:00010000:0.0:1787739055.388631:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739055.388635:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b200, rc 0) 00000100:00100000:0.0:1787739055.388676:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c3821c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665568640:12345-0@lo:101:mdt00_003.0 Request processed in 116us (238us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.388682:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2681 00000100:00100000:2.0:1787739055.388944:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c381a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665568640:0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739055.388957:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1abd20 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x162:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abd27 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.388967:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1abd20 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x162:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abd27 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.388989:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1abd2e lrc: 3/0,1 mode: --/EX res: [0x240000402:0x162:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.388999:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1abd2e lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x162:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.389007:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1abd2e lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x162:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.389023:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1abd35 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.389035:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1abcef lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bdb expref: 10 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.389043:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1abd35 lrc: 4/0,1 mode: --/EX res: [0x240000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.389066:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1abcef lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bdb expref: 10 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.389092:0:7944:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1abcef lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bdb expref: 10 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.389103:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c381a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665568768:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739055.389162:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068bac00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.393314:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c381a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665568768:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739055.393345:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1abd35 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739055.395717:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739055.395816:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652428928 00000100:00100000:0.0:1787739055.395826:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652428928 00010000:00010000:0.0:1787739055.395878:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1abcef lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bdb expref: 11 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.395916:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1408 00000100:00100000:0.0:1787739055.395926:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba0c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2409:x1874579652428928:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739055.395941:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739055.395947:0:6510:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1abcef lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bdb expref: 11 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.395961:0:6510:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1abcef lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bdb expref: 11 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.395988:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1abcef lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bdb expref: 11 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.396004:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb7:0x0].0x0 (ffffa07305d56800) 00010000:00010000:0.0:1787739055.396010:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1abd35 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.396019:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1abd35 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.396028:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1abd35 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.396058:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739055.396119:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba0c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2409:x1874579652428928:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 194us (410us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.396127:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1408 00010000:00010000:2.0:1787739055.396319:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1abd35 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.396328:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1abd35 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.396373:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1abd3c lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.396380:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1abd3c lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.396390:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1abd3c lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.396448:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c381a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665568896:0@lo:101:mdt00_003.0 00000100:00100000:2.0:1787739055.396490:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.396517:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068bac00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739055.396863:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665568896 00000100:00100000:0.0:1787739055.396869:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665568896 00000100:00100000:0.0:1787739055.396906:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2682 00000100:00100000:0.0:1787739055.396912:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665568896:12345-0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739055.396926:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739055.396940:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abd43 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abd3c expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.396962:0:17834:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1abcfd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094be2 expref: 13 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.396976:0:17834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abd43 lrc: 4/0,0 mode: --/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abd3c expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.396997:0:17834:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1abcfd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094be2 expref: 13 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.397009:0:17834:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1abcfd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094be2 expref: 13 pid: 13793 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.397019:0:17834:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f79680 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:17834:1874579665569024:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739055.397081:0:17834:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739055.404482:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739055.404554:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652429056 00000100:00100000:2.0:1787739055.404562:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652429056 00010000:00010000:2.0:1787739055.404600:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1abcfd lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094be2 expref: 14 pid: 13793 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.404643:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1409 00000100:00100000:2.0:1787739055.404652:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2408:x1874579652429056:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739055.404666:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739055.404673:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1abcfd lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094be2 expref: 14 pid: 13793 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.404686:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1abcfd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094be2 expref: 14 pid: 13793 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.404709:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb7:0x0].0x0 (ffffa07433235f00) 00010000:00010000:2.0:1787739055.404716:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abd43 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abd3c expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.404739:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abd43 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1abd3c expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.404748:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abd43 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1abd3c expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.404780:0:6509:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abd43 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abd3c expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.404795:0:6509:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ec3c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665569152:0@lo:105: 00000100:00100000:2.0:1787739055.404841:0:6509:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.404881:0:6509:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739055.404944:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665569152 00000100:00100000:3.0:1787739055.404948:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665569152 00000100:00100000:3.0:1787739055.404988:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1076 00000100:00100000:3.0:1787739055.404996:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ec000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665569152:12345-0@lo:105: 00010000:00010000:3.0:1787739055.405007:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1abd3c lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abd43 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.405017:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1abd3c lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abd43 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.405026:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1abd3c lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abd43 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.405033:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1abd3c lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1abd43 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.405041:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1abd3c lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1abd43 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.405049:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1abd3c lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1abd43 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.405062:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633d400) 00000100:00100000:3.0:1787739055.405126:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ec000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665569152:12345-0@lo:105: Request processed in 131us (284us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.405137:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1076 00000100:00100000:3.0:1787739055.405175:0:6509:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ec3c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665569152:0@lo:105: 00010000:00010000:3.0:1787739055.405192:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739055.405226:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2408:x1874579652429056:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 575us (753us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.405233:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1409 00010000:00010000:0.0:1787739055.405376:0:17834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1abcfd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094be2 expref: 13 pid: 13793 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.405394:0:17834:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f79680 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:17834:1874579665569024:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739055.405458:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abd43 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abd3c expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.405468:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb7:0x0].0x0 (ffffa07433235f00) 00010000:00010000:0.0:1787739055.405471:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739055.405474:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a800, rc 0) 00000100:00100000:0.0:1787739055.405511:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665568896:12345-0@lo:101:mdt00_003.0 Request processed in 8599us (9020us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.405519:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2682 00000100:00100000:2.0:1787739055.405772:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c381a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665568896:0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739055.405786:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1abd3c lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abd43 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.406495:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ee1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665569280:0@lo:403:mdt00_003.0 00000100:00100000:2.0:1787739055.406540:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.406563:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068bac00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739055.406924:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665569280 00000100:00100000:0.0:1787739055.406931:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665569280 00000100:00100000:0.0:1787739055.406973:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2683 00000100:00100000:0.0:1787739055.406981:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ecf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665569280:12345-0@lo:403:mdt00_003.0 00000100:00100000:0.0:1787739055.408589:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ecf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665569280:12345-0@lo:403:mdt00_003.0 Request processed in 1604us (2043us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.408600:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2683 00000100:00100000:2.0:1787739055.409046:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ee1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665569280:0@lo:403:mdt00_003.0 00000040:00080000:2.0:1787739055.409903:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].336, 664 off511920 00000040:00080000:2.0:1787739055.410057:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].337, 744 off560168 00000004:00080000:2.0:1787739055.410063:0:7944:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa073068bac00 with version 641 00000004:00080000:0.0:1787739055.410724:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 641 rpc_version 641 00000020:00100000:0.0:1787739055.410755:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739055.410763:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07305f79680 x1874579665569536/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1808/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739055.410784:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f79680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665569536:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739055.410835:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739055.410869:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4540 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.410997:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665569536 00000100:00100000:2.0:1787739055.411003:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665569536 00000100:00100000:2.0:1787739055.411046:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2684 00000100:00100000:2.0:1787739055.411053:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f79a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665569536:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739055.413040:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f79a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665569536:12345-0@lo:1000:osp_up0-1.0 Request processed in 1980us (2197us total) trans 4294983325 rc 0/0 00000100:00100000:2.0:1787739055.413053:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2684 00000100:00100000:0.0:1787739055.414757:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f79680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665569536:0@lo:1000:osp_up0-1.0 00000004:00200000:2.0:1787739055.414850:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x162:0x0]: returning size 4096 00010000:00010000:2.0:1787739055.414877:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1abd35 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.414888:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1abd35 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.414896:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1abd35 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.414901:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739055.414906:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1abd35 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.414914:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1abd35 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.414930:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb7:0x0].0x0 (ffffa07305d56800) 00010000:00010000:2.0:1787739055.414933:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739055.414939:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1abd35 lrc: 2/0,0 mode: --/EX res: [0x240000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.414945:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1abd35 lrc: 0/0,0 mode: --/EX res: [0x240000400:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.414963:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1abd3c lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abd43 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.414972:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1abd3c lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abd43 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.414978:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1abd3c lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abd43 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.415007:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739055.415016:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xb7:0x0] 00010000:00010000:2.0:1787739055.415025:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1abd2e lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x162:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.415031:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1abd2e lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x162:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.415036:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1abd2e lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x162:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.415042:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739055.415045:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1abd2e lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x162:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.415051:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1abd2e lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x162:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.415059:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x162:0x0].0x0 (ffffa07305d56600) 00010000:00010000:2.0:1787739055.415062:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739055.415065:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1abd2e lrc: 2/0,0 mode: --/EX res: [0x240000402:0x162:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.415081:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1abd2e lrc: 0/0,0 mode: --/EX res: [0x240000402:0x162:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.415091:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1abd20 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x162:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abd27 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.415098:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1abd20 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x162:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abd27 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.415104:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1abd20 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x162:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abd27 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.415124:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:2.0:1787739055.415134:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739055.415139:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1abd04 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abd0b expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.415145:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1abd04 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abd0b expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.415151:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1abd04 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abd0b expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.415172:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:2.0:1787739055.415177:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739055.415181:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1abd12 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abd19 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.415188:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1abd12 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abd19 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.415271:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f52c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17464:x1874579652428800:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 31279us (31442us total) trans 4294968315 rc 0/0 00000100:00100000:2.0:1787739055.415279:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23231 00010000:00010000:1.0:1787739055.415431:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1787739055.415438:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1abd3c lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1abd43 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.415446:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739055.415450:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1abd3c lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abd43 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739055.415457:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1abd3c lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abd43 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.415489:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1abd3c lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abd43 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.415500:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739055.415522:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1abd3c lrc: 1/0,0 mode: --/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abd43 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.415530:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d400/0x7ee4453e4a1abd3c lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abd43 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.415544:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1787739055.415548:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1abd20 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x162:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1abd27 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.415554:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739055.415558:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1abd20 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x162:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abd27 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739055.415564:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1abd20 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x162:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abd27 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.415580:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1abd20 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x162:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abd27 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.415586:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739055.415592:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1abd20 lrc: 1/0,0 mode: --/EX res: [0x240000402:0x162:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abd27 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.415598:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c200/0x7ee4453e4a1abd20 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x162:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abd27 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.415614:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739055.415618:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1abd04 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1abd0b expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.415623:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633ce00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739055.415627:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1abd04 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abd0b expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739055.415633:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1abd04 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abd0b expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.415648:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1abd04 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abd0b expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.415654:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739055.415668:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1abd04 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abd0b expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.415675:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1abd04 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abd0b expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.415686:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739055.415688:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739055.415689:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739055.415699:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787739055.415713:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665569920:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739055.415762:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739055.415805:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665569792:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739055.415830:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739055.415850:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787739055.415855:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665569664:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739055.415876:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739055.415905:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665569920 00000100:00100000:1.0:1787739055.415911:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665569920 00000100:00100000:1.0:1787739055.415951:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665569792 00000100:00100000:1.0:1787739055.415955:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665569792 00000100:00100000:1.0:1787739055.415972:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665569664 00000100:00100000:1.0:1787739055.415975:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665569664 00000100:00100000:1.0:1787739055.415990:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1209 00000100:00100000:1.0:1787739055.415998:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+12:3651:x1874579665569920:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739055.416010:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739055.416027:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abd0b lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abd04 expref: 12 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.416039:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235e00) 00010000:00010000:1.0:1787739055.416046:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739055.416087:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+11:3651:x1874579665569920:12345-0@lo:103:ldlm_bl.0 Request processed in 90us (324us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.416094:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1209 00000100:00100000:1.0:1787739055.416108:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1210 00000100:00100000:1.0:1787739055.416114:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f7c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+10:3653:x1874579665569792:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739055.416122:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739055.416132:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abd27 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x162:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abd20 expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.416147:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x162:0x0].0x0 (ffffa073068a9100) 00010000:00010000:1.0:1787739055.416153:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739055.416180:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f7c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3653:x1874579665569792:12345-0@lo:103:ldlm_bl.0 Request processed in 66us (349us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.416186:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1210 00000100:00100000:1.0:1787739055.416198:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1211 00000100:00100000:1.0:1787739055.416204:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+8:3653:x1874579665569664:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739055.416212:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739055.416221:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abd43 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abd3c expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.416231:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb7:0x0].0x0 (ffffa07433235f00) 00010000:00010000:1.0:1787739055.416236:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739055.416257:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+7:3653:x1874579665569664:12345-0@lo:103:ldlm_bl.0 Request processed in 54us (380us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.416262:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1211 00000100:00100000:1.0:1787739055.416290:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665569664:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739055.416298:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665569792:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739055.416389:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665569920:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739055.416884:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739055.419369:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739055.436631:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739055.447186:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739055.447263:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652429184 00000100:00100000:0.0:1787739055.447273:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652429184 00000100:00100000:0.0:1787739055.447331:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23232 00000100:00100000:0.0:1787739055.447341:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17465:x1874579652429184:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739055.447359:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739055.447382:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abd4a lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094be9 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.447484:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abd51 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.447498:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abd51 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.447510:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abd51 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.447524:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1abd58 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.447532:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1abd58 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.447538:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1abd58 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.447575:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abd51 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.447583:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abd51 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.447589:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abd51 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.447596:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abd51 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.447602:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609bc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739055.447607:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abd51 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.447620:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abd51 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.447632:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9600) 00010000:00010000:0.0:1787739055.447635:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739055.447639:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abd51 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.447645:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abd51 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.447656:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1abd58 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.447661:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1abd58 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.447668:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1abd58 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.447673:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739055.447676:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1abd58 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.447681:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1abd58 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.447688:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xade5bda9 (ffffa073068a9d00) 00010000:00010000:0.0:1787739055.447691:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739055.447693:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1abd58 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.447699:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1abd58 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.447728:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abd4a lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094be9 expref: 14 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.447743:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9600) 00010000:00010000:0.0:1787739055.447745:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739055.447749:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abd4a lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094be9 expref: 14 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.447761:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b800, rc 0) 00000100:00100000:0.0:1787739055.447811:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17465:x1874579652429184:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 473us (634us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739055.447817:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23232 00000100:00100000:2.0:1787739055.452368:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739055.452495:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652429312 00000100:00100000:0.0:1787739055.452505:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652429312 00000100:00100000:0.0:1787739055.452559:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23233 00000100:00100000:0.0:1787739055.452569:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17465:x1874579652429312:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739055.452588:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739055.452610:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1abd5f lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094bf0 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.452672:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abd66 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.452685:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abd66 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.452695:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abd66 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.452708:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abd6d lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.452716:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abd6d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.452734:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abd6d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.452770:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abd66 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.452777:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abd66 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.452784:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abd66 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.452791:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abd66 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.452797:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739055.452802:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abd66 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.452810:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abd66 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.452822:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9900) 00010000:00010000:0.0:1787739055.452825:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739055.452830:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abd66 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.452835:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abd66 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.452860:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abd6d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.452865:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abd6d lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.452871:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abd6d lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.452876:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739055.452879:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abd6d lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.452885:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abd6d lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.452892:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xade5bda9 (ffffa073068a9c00) 00010000:00010000:0.0:1787739055.452894:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739055.452897:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abd6d lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.452902:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abd6d lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.452929:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1abd5f lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bf0 expref: 14 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.452939:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9900) 00010000:00010000:0.0:1787739055.452942:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739055.452945:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1abd5f lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bf0 expref: 14 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.452955:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609aa00, rc 0) 00000100:00100000:0.0:1787739055.453006:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17465:x1874579652429312:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 438us (645us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739055.453013:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23233 00000100:00100000:1.0:1787739055.458011:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739055.458095:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652429440 00000100:00100000:2.0:1787739055.458105:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652429440 00000100:00100000:2.0:1787739055.458165:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23234 00000100:00100000:2.0:1787739055.458175:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f6d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17465:x1874579652429440:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739055.458194:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739055.458217:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abd74 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094bf7 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.458272:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1abd7b lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.458282:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1abd7b lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.458289:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1abd7b lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739055.458339:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:2.0:1787739055.458343:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739055.458369:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abd74 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bf7 expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.458381:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1abd7b lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bf7 expref: 14 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.458387:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5100) 00010000:00010000:2.0:1787739055.458389:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739055.458391:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d000, rc 0) 00000100:00100000:2.0:1787739055.458553:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f6d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17465:x1874579652429440:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 380us (551us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739055.458558:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23234 00000100:00100000:1.0:1787739055.465160:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739055.465248:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652429696 00000100:00100000:3.0:1787739055.465260:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652429696 00000100:00100000:3.0:1787739055.465363:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23235 00000100:00100000:3.0:1787739055.465374:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f5a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17465:x1874579652429696:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:3.0:1787739055.465468:0:6523:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739055.465495:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1abd7b lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bf7 expref: 14 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.465515:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5100) 00010000:00010000:3.0:1787739055.465525:0:6523:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739055.465589:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1abd82 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.465603:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1abd82 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.465613:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1abd82 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.465625:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1abd89 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.465637:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1abd89 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.465644:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1abd89 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787739055.466463:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].350, 2168 off537784 00000040:00080000:3.0:1787739055.466539:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].349, 2368 off581496 00000004:00080000:3.0:1787739055.466545:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07305fcc240 with version 667 00000004:00080000:2.0:1787739055.467097:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 667 rpc_version 667 00000020:00100000:2.0:1787739055.467130:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739055.467140:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074156ed2c0 x1874579665570176/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 3976/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739055.467162:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ed2c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665570176:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739055.467225:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.467266:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba900 going to sleep for 11 seconds 00000100:00100000:1.0:1787739055.467338:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665570176 00000100:00100000:1.0:1787739055.467346:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665570176 00000100:00100000:1.0:1787739055.467392:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2685 00000100:00100000:1.0:1787739055.467401:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665570176:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739055.467810:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665570176:12345-0@lo:1000:osp_up1-0.0 Request processed in 409us (583us total) trans 4294968316 rc 0/0 00000100:00100000:1.0:1787739055.467818:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2685 00000100:00100000:2.0:1787739055.468802:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ed2c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665570176:0@lo:1000:osp_up1-0.0 00010000:00010000:1.0:1787739055.469961:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abd90 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x14fe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.469980:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abd90 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14fe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.469991:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abd90 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14fe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.470014:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1abd97 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xba:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.470021:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1abd97 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.470028:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1abd97 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.470065:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1abd9e lrc: 3/0,1 mode: --/PW res: [0x240000401:0xba:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.470082:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1abd9e lrc: 3/0,1 mode: --/PW res: [0x240000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.470091:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1abd9e lrc: 3/0,1 mode: --/PW res: [0x240000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739055.470116:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061ff0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665570304:0@lo:101:mdt00_000.0 00000100:00100000:1.0:1787739055.470167:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739055.470200:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739055.470282:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665570304 00000100:00100000:3.0:1787739055.470289:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665570304 00000100:00100000:3.0:1787739055.470604:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2686 00000100:00100000:3.0:1787739055.470615:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061ffc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6523:x1874579665570304:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739055.470635:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739055.470654:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1abda5 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xba:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abd9e expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.470675:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1abda5 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abd9e expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.470685:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1abda5 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abd9e expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.470693:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xba:0x0].0x0 (ffffa07305d56d00) 00010000:00010000:3.0:1787739055.470695:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739055.470699:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa600, rc 0) 00000100:00100000:3.0:1787739055.470758:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061ffc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6523:x1874579665570304:12345-0@lo:101:mdt00_000.0 Request processed in 146us (590us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.470768:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2686 00000100:00100000:1.0:1787739055.470853:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061ff0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665570304:0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739055.470867:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1abd9e lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abda5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.470876:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1abd9e lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abda5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739055.470897:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003e88bc09 reply state 00000000f2d43877 transno 4294983326 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1abd97 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.470908:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1abd97 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.470915:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xba:0x0].0x0 (ffffa07433235e00) 00010000:00010000:1.0:1787739055.470917:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739055.470920:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787739055.470926:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1abd9e lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abda5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.470934:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1abd9e lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abda5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739055.470945:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xba:0x0] 00000004:00010000:1.0:1787739055.470951:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003e88bc09 reply state 00000000f2d43877 transno 4294983326 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abd90 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14fe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.470964:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abd90 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x14fe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.470970:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14fe:0x0].0x0 (ffffa07433235f00) 00010000:00010000:1.0:1787739055.470972:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739055.470973:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:1.0:1787739055.470978:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14fe:0x0]: returning size 4096 00010000:00010000:1.0:1787739055.470988:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1abd82 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.470995:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1abd82 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.471001:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1abd82 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.471007:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1abd82 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.471013:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aaa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739055.471017:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1abd82 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.471026:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1abd82 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.471036:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305d56900) 00010000:00010000:1.0:1787739055.471038:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739055.471041:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1abd82 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.471051:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1abd82 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739055.471067:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003e88bc09 reply state 00000000f2d43877 transno 4294983326 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1abd89 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.471086:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1abd89 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.471092:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xade5bda9 (ffffa07305d56100) 00010000:00010000:1.0:1787739055.471094:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739055.471096:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 2 00000100:00100000:1.0:1787739055.471163:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f5a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17465:x1874579652429696:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 5789us (6009us total) trans 4294983326 rc 0/0 00000100:00100000:1.0:1787739055.471171:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23235 00000100:00100000:3.0:1787739055.496360:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739055.497777:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652429952 00000100:00100000:0.0:1787739055.497788:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652429952 00000100:00100000:2.0:1787739055.498208:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23236 00000100:00100000:2.0:1787739055.498220:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17466:x1874579652429952:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:2.0:1787739055.498781:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17466:x1874579652429952:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 500us (2367us total) trans 0 rc -17/-17 00000100:00100000:2.0:1787739055.498791:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23236 00000100:00100000:1.0:1787739055.517692:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739055.517832:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652430080 00000100:00100000:2.0:1787739055.517852:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652430080 00000100:00100000:2.0:1787739055.517909:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23237 00000100:00100000:2.0:1787739055.517919:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061ff0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17467:x1874579652430080:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739055.517938:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739055.518023:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abdac lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094bfe expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.518199:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abdb3 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.518213:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abdb3 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.518224:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abdb3 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.518234:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abdba lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.518242:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abdba lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.518248:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abdba lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.518314:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1abdc1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14fe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.518592:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1abdc1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14fe:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.518600:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1abdc1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14fe:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739055.518680:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14fe:0x0]: returning size 4096 00000004:00200000:2.0:1787739055.518685:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:2.0:1787739055.518706:0:7944:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1abdc1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14fe:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.518846:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abdb3 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.518854:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abdb3 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.518861:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abdb3 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.518867:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abdb3 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.518874:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739055.518878:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abdb3 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.518885:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abdb3 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.518897:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5b00) 00010000:00010000:2.0:1787739055.518901:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739055.518906:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abdb3 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.518912:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abdb3 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.518927:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abdba lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.518933:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abdba lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.518939:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abdba lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.518944:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739055.518947:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abdba lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.519158:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abdba lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.519170:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xade5bda9 (ffffa07305d56100) 00010000:00010000:2.0:1787739055.519172:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739055.519176:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abdba lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.519181:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abdba lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.519209:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abdac lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bfe expref: 17 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.519233:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1abdc1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14fe:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bfe expref: 16 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.519240:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14fe:0x0].0x0 (ffffa07433235f00) 00010000:00010000:2.0:1787739055.519242:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787739055.519245:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633de00, rc 0) 00000100:00100000:2.0:1787739055.519477:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061ff0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17467:x1874579652430080:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 1558us (1792us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739055.519485:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23237 00000100:00100000:3.0:1787739055.524502:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739055.524578:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652430208 00000100:00100000:0.0:1787739055.524587:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652430208 00000100:00100000:0.0:1787739055.524643:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23238 00000100:00100000:0.0:1787739055.524653:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17467:x1874579652430208:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739055.524673:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739055.524695:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1abdc8 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xba:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094c05 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.524761:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abdcf lrc: 3/1,0 mode: --/PR res: [0x200000400:0xba:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.524774:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abdcf lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.524787:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abdcf lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739055.524850:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xba:0x0]: returning size 4096 00000004:00200000:0.0:1787739055.524856:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739055.524891:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1abdc8 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xba:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c05 expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.524915:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abdcf lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xba:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c05 expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.524923:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xba:0x0].0x0 (ffffa07433235e00) 00010000:00010000:0.0:1787739055.524926:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739055.524930:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b200, rc 0) 00000100:00100000:0.0:1787739055.525872:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17467:x1874579652430208:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 1214us (1373us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.525883:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23238 00000100:00100000:3.0:1787739055.530244:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739055.532175:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652430336 00000100:00100000:0.0:1787739055.532190:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652430336 00000100:00100000:0.0:1787739055.532301:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23239 00000100:00100000:0.0:1787739055.532312:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17467:x1874579652430336:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739055.532336:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739055.532355:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1abdd6 lrc: 2/0,0 mode: --/CR res: [0x240000401:0xba:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094c0c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.532428:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1abddd lrc: 3/1,0 mode: --/PR res: [0x240000401:0xba:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.532443:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1abda5 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abd9e expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.532452:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1abddd lrc: 4/1,0 mode: --/PR res: [0x240000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.532484:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1abda5 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1abd9e expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.532500:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f78b40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665570432:0@lo:104: 00000100:00100000:0.0:1787739055.532561:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739055.532612:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739055.532743:0:6506:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787739055.532755:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665570432 00000100:00100000:1.0:1787739055.532760:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665570432 00000100:00100000:1.0:1787739055.532796:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1077 00000100:00100000:1.0:1787739055.532803:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07436dfd050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6524:x1874579665570432:12345-0@lo:104: 00010000:00010000:1.0:1787739055.532815:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1abd9e lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1abda5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.533221:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787739055.533228:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07436dfd050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6524:x1874579665570432:12345-0@lo:104: Request processed in 426us (668us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.533236:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1077 00010000:00010000:3.0:1787739055.533320:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739055.533328:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1abd9e lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1abda5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.533337:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739055.533342:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1abd9e lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xba:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1abda5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739055.533351:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1abd9e lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xba:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1abda5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.533378:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1abd9e lrc: 2/0,0 mode: --/PW res: [0x240000401:0xba:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abda5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.533384:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739055.533432:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1abd9e lrc: 1/0,0 mode: --/PW res: [0x240000401:0xba:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abda5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.533441:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1abd9e lrc: 0/0,0 mode: --/PW res: [0x240000401:0xba:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abda5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.533459:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739055.533461:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739055.533463:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739055.533472:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787739055.533483:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665570560:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739055.533514:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.533556:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665570560 00000100:00100000:3.0:1787739055.533560:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665570560 00010000:00010000:3.0:1787739055.533579:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1abda5 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1abd9e expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739055.533602:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1410 00000100:00100000:3.0:1787739055.533609:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be7c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+10:3650:x1874579665570560:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739055.533619:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787739055.535649:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f78b40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665570432:0@lo:104: 00010000:00010000:1.0:1787739055.535674:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1abddd lrc: 3/1,0 mode: --/PR res: [0x240000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.536820:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1abd12 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abd19 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739055.536836:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1abd12 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abd19 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.536941:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1abd12 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abd19 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.536948:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1787739055.537061:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fd680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665570688:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:0.0:1787739055.537239:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.537312:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665570688 00000100:00100000:2.0:1787739055.537318:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665570688 00000100:00100000:2.0:1787739055.537372:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1212 00000100:00100000:2.0:1787739055.537381:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f78780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+7:3652:x1874579665570688:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:2.0:1787739055.537393:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739055.537438:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1abd19 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abd12 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.537454:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x4dc682e0 (ffffa07305d56300) 00010000:00010000:2.0:1787739055.537464:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739055.537528:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a200/0x7ee4453e4a1abd12 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x4dc682e0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abd19 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787739055.537545:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968316 is committed 00000001:00080000:1.0:1787739055.537561:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968315 is committed 00000020:00080000:1.0:1787739055.537568:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 341 batchid updated 00000020:00080000:1.0:1787739055.537581:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07305fccf00 refcount 2 committed 0 result 0 batchid 342 00000020:00080000:1.0:1787739055.537584:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c695800 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07305fcc540 00000020:00080000:1.0:1787739055.537588:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].336 00000020:00080000:1.0:1787739055.537590:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c695900 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073040e8800 00000020:00080000:1.0:1787739055.537592:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].335 00010000:00010000:1.0:1787739055.537618:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abca2 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x162:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.537625:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abca2 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x162:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.537632:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abca2 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x162:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.537637:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739055.537641:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abca2 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x162:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.537647:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abca2 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x162:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.537656:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x162:0x0].0x0 (ffffa07305d56600) 00010000:00010000:1.0:1787739055.537658:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739055.537661:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abca2 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x162:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.537667:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1abca2 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x162:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.537679:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1abca9 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.537686:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1abca9 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.537691:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1abca9 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.537696:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739055.537698:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1abca9 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.537708:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1abca9 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.537725:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb7:0x0].0x0 (ffffa07305d56800) 00010000:00010000:1.0:1787739055.537728:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739055.537731:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1abca9 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.537737:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1abca9 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:1.0:1787739055.537757:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 342 committed batchid 341 00000004:00080000:1.0:1787739055.538155:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07307054180 with version 642 00000020:00080000:1.0:1787739055.538205:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 341 cancel update log [0x200000401:0x3:0x0].334: rc = 0 00000020:00080000:1.0:1787739055.538249:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 341 cancel update log [0x240000400:0x3:0x0].334: rc = 0 00000020:00080000:1.0:1787739055.538257:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 341 00010000:00010000:1.0:1787739055.538302:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1abda5 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1abd9e expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.538312:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xba:0x0].0x0 (ffffa07305d56d00) 00010000:00010000:1.0:1787739055.538316:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1abddd lrc: 3/1,0 mode: --/PR res: [0x240000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.538325:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1abddd lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.538332:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1abddd lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.538357:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739055.538402:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be7c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+5:3650:x1874579665570560:12345-0@lo:103:ldlm_bl.0 Request processed in 4793us (4886us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.538410:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1410 00010000:00010000:1.0:1787739055.538457:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1abddd lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.538465:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1abddd lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739055.538510:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xba:0x0]: returning size 4096 00000004:00200000:1.0:1787739055.538515:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739055.538557:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1abdd6 lrc: 0/0,0 mode: --/CR res: [0x240000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c0c expref: 9 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.538574:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1abddd lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c0c expref: 8 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.538581:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xba:0x0].0x0 (ffffa07305d56d00) 00010000:00010000:1.0:1787739055.538584:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739055.538586:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b800, rc 0) 00000100:00100000:1.0:1787739055.538628:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17467:x1874579652430336:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 6318us (8394us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.538636:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23239 00000004:00080000:3.0:1787739055.538980:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 642 rpc_version 642 00000020:00100000:3.0:1787739055.539006:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739055.539014:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306be6580 x1874579665570944/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739055.539033:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be6580 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665570944:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739055.539099:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.539136:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739055.539168:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665570560:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739055.539547:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665570944 00000100:00100000:1.0:1787739055.539553:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665570944 00000100:00100000:1.0:1787739055.539597:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2687 00000100:00100000:1.0:1787739055.539604:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07436d8e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665570944:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739055.539748:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07436d8e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665570944:12345-0@lo:1000:osp_up0-1.0 Request processed in 144us (648us total) trans 4294983327 rc 0/0 00000100:00100000:1.0:1787739055.539755:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2687 00000100:00100000:3.0:1787739055.540278:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be6580 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665570944:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739055.543566:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f78780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+7:3652:x1874579665570688:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 6175us (6316us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739055.543580:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1212 00000100:00100000:0.0:1787739055.546062:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fd680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665570688:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787739055.549470:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739055.549546:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652430592 00000100:00100000:2.0:1787739055.549555:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652430592 00000100:00100000:2.0:1787739055.549607:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23240 00000100:00100000:2.0:1787739055.549618:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061ffc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17467:x1874579652430592:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:2.0:1787739055.549673:0:7944:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739055.549699:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1abdc1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14fe:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bfe expref: 17 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.549727:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14fe:0x0].0x0 (ffffa07433235f00) 00010000:00010000:2.0:1787739055.549731:0:7944:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739055.549756:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abde4 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.549768:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abde4 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.549777:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abde4 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.549788:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abdeb lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.549796:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abdeb lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.549802:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abdeb lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.549884:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abdf2 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14fe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.549893:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abdf2 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14fe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.549900:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abdf2 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14fe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.549916:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abdf9 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xba:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.549925:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abdcf lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c05 expref: 16 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.549933:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abdf9 lrc: 4/0,1 mode: --/EX res: [0x200000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.549974:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abdcf lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c05 expref: 16 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.549989:0:7944:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abdcf lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c05 expref: 16 pid: 6523 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.550003:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665571072:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739055.550095:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba900 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.555360:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665571072:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739055.555389:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abdf9 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739055.558529:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739055.558617:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652430720 00000100:00100000:2.0:1787739055.558627:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652430720 00010000:00010000:2.0:1787739055.558667:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abdcf lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c05 expref: 17 pid: 6523 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.558707:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1411 00000100:00100000:2.0:1787739055.558722:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074301a0850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2408:x1874579652430720:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739055.558736:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739055.558742:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abdcf lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xba:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c05 expref: 17 pid: 6523 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.558756:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abdcf lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xba:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c05 expref: 17 pid: 6523 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.558778:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1abdcf lrc: 0/0,0 mode: --/PR res: [0x200000400:0xba:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c05 expref: 17 pid: 6523 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.558794:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xba:0x0].0x0 (ffffa07433235e00) 00010000:00010000:2.0:1787739055.558799:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abdf9 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.558808:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abdf9 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.558816:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abdf9 lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.558858:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739055.558910:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074301a0850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:2408:x1874579652430720:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 194us (390us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739055.558919:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1411 00010000:00010000:0.0:1787739055.559371:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abdf9 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.559380:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abdf9 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.559471:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609aa00/0x7ee4453e4a1abe00 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xba:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.559479:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609aa00/0x7ee4453e4a1abe00 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.559488:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609aa00/0x7ee4453e4a1abe00 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.559508:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665571200:0@lo:101:mdt00_003.0 00000100:00100000:0.0:1787739055.559549:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739055.559574:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.559648:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665571200 00000100:00100000:2.0:1787739055.559653:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665571200 00000100:00100000:2.0:1787739055.559690:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2688 00000100:00100000:2.0:1787739055.559697:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f78780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+5:7944:x1874579665571200:12345-0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739055.559711:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739055.559726:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1abe07 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xba:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abe00 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.559748:0:8448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1abddd lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c0c expref: 7 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.559757:0:8448:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1abe07 lrc: 4/0,0 mode: --/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abe00 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.559777:0:8448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1abddd lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c0c expref: 7 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.559788:0:8448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1abddd lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c0c expref: 7 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.559800:0:8448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8448:1874579665571328:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739055.559862:0:8448:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.566237:0:8448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8448:1874579665571328:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739055.566277:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1abe07 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abe00 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.566290:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xba:0x0].0x0 (ffffa07305d56d00) 00010000:00010000:2.0:1787739055.566293:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739055.566297:0:8448:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1abe07 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abe00 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.566313:0:8448:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1abe07 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abe00 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.566322:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633de00, rc 0) 00000100:00100000:2.0:1787739055.566400:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f78780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7944:x1874579665571200:12345-0@lo:101:mdt00_003.0 Request processed in 6702us (6848us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739055.566409:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2688 00000100:00100000:0.0:1787739055.566584:0:7944:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968316 gen 2 00000004:00080000:0.0:1787739055.566590:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968311 committed transno 4294968316 00000020:00080000:0.0:1787739055.566596:0:7944:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa0730610e9c0 refcount 2 committed 0 result 0 batchid 351 00000020:00080000:0.0:1787739055.566600:0:7944:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88d00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730610e3c0 00000020:00080000:0.0:1787739055.566603:0:7944:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].348 00000020:00080000:0.0:1787739055.566605:0:7944:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88e00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa0730558c600 00000020:00080000:0.0:1787739055.566608:0:7944:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].349 00000004:00080000:0.0:1787739055.566635:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968313 committed transno 4294968316 00000004:00080000:0.0:1787739055.566645:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968314 committed transno 4294968316 00000004:00080000:0.0:1787739055.566655:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968316 committed transno 4294968316 00000100:00100000:0.0:1787739055.566667:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665571200:0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739055.566684:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609aa00/0x7ee4453e4a1abe00 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abe07 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.566693:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609aa00/0x7ee4453e4a1abe00 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abe07 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1787739055.566800:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 351 committed batchid 350 00000020:00080000:3.0:1787739055.566869:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 350 00000100:00100000:1.0:1787739055.569611:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739055.570002:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652430848 00000100:00100000:3.0:1787739055.570011:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652430848 00010000:00010000:3.0:1787739055.570054:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1abddd lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c0c expref: 8 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739055.570226:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1412 00000100:00100000:3.0:1787739055.570237:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fe940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2409:x1874579652430848:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739055.570253:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739055.570260:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1abddd lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c0c expref: 8 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.570273:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1abddd lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c0c expref: 8 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.570295:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1abddd lrc: 0/0,0 mode: --/PR res: [0x240000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c0c expref: 8 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.570312:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xba:0x0].0x0 (ffffa07305d56d00) 00010000:00010000:3.0:1787739055.570319:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1abe07 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abe00 expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.570558:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1abe07 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1abe00 expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.570569:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1abe07 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1abe00 expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.570605:0:6509:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1abe07 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abe00 expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739055.570619:0:6509:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4b40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001_UUID:6509:1874579665571456:0@lo:105: 00000100:00100000:3.0:1787739055.570670:0:6509:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.570732:0:6509:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89900 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.571171:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665571456 00000100:00100000:2.0:1787739055.571177:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665571456 00000100:00100000:2.0:1787739055.571220:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1078 00000100:00100000:2.0:1787739055.571227:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6509:x1874579665571456:12345-0@lo:105: 00010000:00010000:2.0:1787739055.571239:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609aa00/0x7ee4453e4a1abe00 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abe07 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.571248:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609aa00/0x7ee4453e4a1abe00 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abe07 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.571257:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609aa00/0x7ee4453e4a1abe00 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abe07 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.571265:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609aa00/0x7ee4453e4a1abe00 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1abe07 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.571273:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609aa00/0x7ee4453e4a1abe00 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1abe07 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.571281:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609aa00/0x7ee4453e4a1abe00 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1abe07 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.571307:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609aa00) 00000100:00100000:2.0:1787739055.571363:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6509:x1874579665571456:12345-0@lo:105: Request processed in 136us (692us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739055.571371:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1078 00010000:00010000:0.0:1787739055.571481:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609aa00/0x7ee4453e4a1abe00 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abe07 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.571493:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609aa00/0x7ee4453e4a1abe00 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abe07 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.572307:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665571584:0@lo:403:mdt00_003.0 00000100:00100000:0.0:1787739055.572821:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739055.572865:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4c00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739055.573315:0:6509:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4b40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001_UUID:6509:1874579665571456:0@lo:105: 00010000:00010000:3.0:1787739055.573345:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739055.573389:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fe940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2409:x1874579652430848:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 3152us (3785us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.573399:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1412 00000100:00100000:2.0:1787739055.573488:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665571584 00000100:00100000:2.0:1787739055.573494:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665571584 00000100:00100000:2.0:1787739055.573537:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2689 00000100:00100000:2.0:1787739055.573545:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f78f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7944:x1874579665571584:12345-0@lo:403:mdt00_003.0 00000100:00100000:2.0:1787739055.579702:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f78f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7944:x1874579665571584:12345-0@lo:403:mdt00_003.0 Request processed in 6148us (6872us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739055.579716:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2689 00000100:00100000:0.0:1787739055.579986:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665571584:0@lo:403:mdt00_003.0 00000040:00080000:0.0:1787739055.580908:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].351, 952 off538736 00000040:00080000:0.0:1787739055.581219:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].350, 1032 off582528 00000004:00080000:0.0:1787739055.581225:0:7944:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa073068b4c00 with version 668 00000004:00080000:2.0:1787739055.581535:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 668 rpc_version 668 00000020:00100000:2.0:1787739055.581567:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739055.581575:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074156ef480 x1874579665571840/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1992/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739055.581596:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef480 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665571840:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739055.581654:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.581688:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739055.582202:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665571840 00000100:00100000:0.0:1787739055.582212:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665571840 00000100:00100000:0.0:1787739055.582255:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2690 00000100:00100000:0.0:1787739055.582264:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ee940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665571840:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739055.583254:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ee940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665571840:12345-0@lo:1000:osp_up1-0.0 Request processed in 987us (1595us total) trans 4294968317 rc 0/0 00000100:00100000:0.0:1787739055.583264:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2690 00000100:00100000:2.0:1787739055.584186:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef480 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665571840:0@lo:1000:osp_up1-0.0 00000004:00200000:0.0:1787739055.584238:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14fe:0x0]: returning size 4096 00010000:00010000:0.0:1787739055.584261:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abdf9 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584270:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abdf9 lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584278:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abdf9 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584285:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739055.584289:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abdf9 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584296:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abdf9 lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584311:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xba:0x0].0x0 (ffffa07433235e00) 00010000:00010000:0.0:1787739055.584315:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739055.584319:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abdf9 lrc: 2/0,0 mode: --/EX res: [0x200000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584326:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1abdf9 lrc: 0/0,0 mode: --/EX res: [0x200000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584347:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609aa00/0x7ee4453e4a1abe00 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abe07 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584355:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609aa00/0x7ee4453e4a1abe00 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abe07 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584362:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609aa00/0x7ee4453e4a1abe00 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abe07 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584460:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739055.584469:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xba:0x0] 00010000:00010000:0.0:1787739055.584476:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abdf2 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14fe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584483:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abdf2 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x14fe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584489:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abdf2 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14fe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584493:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739055.584496:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abdf2 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14fe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584502:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abdf2 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x14fe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584510:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14fe:0x0].0x0 (ffffa07433235f00) 00010000:00010000:0.0:1787739055.584512:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739055.584516:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abdf2 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14fe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584521:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1abdf2 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14fe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584537:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abde4 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584542:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abde4 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584548:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abde4 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584554:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abde4 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584559:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739055.584562:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abde4 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584568:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abde4 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584574:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5b00) 00010000:00010000:0.0:1787739055.584576:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739055.584578:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abde4 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584583:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abde4 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739055.584599:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000475fe979 reply state 00000000c3e691fe transno 4294983328 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abdeb lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584613:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abdeb lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.584620:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xade5bda9 (ffffa07305d56100) 00010000:00010000:0.0:1787739055.584623:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739055.584625:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787739055.584687:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061ffc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17467:x1874579652430592:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 35070us (35224us total) trans 4294983328 rc 0/0 00000100:00100000:0.0:1787739055.584695:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23240 00010000:00010000:2.0:1787739055.588795:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739055.588810:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609aa00/0x7ee4453e4a1abe00 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1abe07 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.588822:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609aa00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739055.588828:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609aa00/0x7ee4453e4a1abe00 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abe07 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739055.588845:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609aa00/0x7ee4453e4a1abe00 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abe07 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.588888:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609aa00/0x7ee4453e4a1abe00 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abe07 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.588895:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739055.588919:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609aa00/0x7ee4453e4a1abe00 lrc: 1/0,0 mode: --/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abe07 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.588927:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609aa00/0x7ee4453e4a1abe00 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abe07 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.588948:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739055.588949:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739055.588958:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739055.589883:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665571968:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739055.589962:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.590980:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665571968 00000100:00100000:2.0:1787739055.590989:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665571968 00000100:00100000:2.0:1787739055.591047:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1213 00000100:00100000:2.0:1787739055.591055:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+7:3653:x1874579665571968:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739055.591080:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739055.591102:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1abe07 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abe00 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.591121:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xba:0x0].0x0 (ffffa07305d56d00) 00010000:00010000:2.0:1787739055.591130:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739055.591180:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+6:3653:x1874579665571968:12345-0@lo:103:ldlm_bl.0 Request processed in 125us (1226us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739055.591188:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1213 00000100:00100000:1.0:1787739055.591470:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665571968:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739055.609178:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739055.609315:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652430976 00000100:00100000:0.0:1787739055.609335:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652430976 00000100:00100000:0.0:1787739055.609397:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23241 00000100:00100000:0.0:1787739055.609407:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fe1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17468:x1874579652430976:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739055.609459:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739055.609481:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1abe0e lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094c13 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.609545:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1abe15 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.609559:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1abe15 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.609571:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1abe15 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.609584:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abe1c lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.609595:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abe1c lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.609601:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abe1c lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.609643:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1abe15 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.609651:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1abe15 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.609657:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1abe15 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.609663:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1abe15 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.609669:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739055.609673:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1abe15 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.609679:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1abe15 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.609691:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07307064000) 00010000:00010000:0.0:1787739055.609693:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739055.609697:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1abe15 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.609703:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1abe15 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.609728:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abe1c lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.609736:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abe1c lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.609742:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abe1c lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.609747:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609be00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739055.609750:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abe1c lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.609756:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abe1c lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.609762:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x367edb60 (ffffa07307064f00) 00010000:00010000:0.0:1787739055.609765:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739055.609767:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abe1c lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.609773:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abe1c lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.609801:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1abe0e lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c13 expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.609811:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07307064000) 00010000:00010000:0.0:1787739055.609813:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739055.609817:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1abe0e lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c13 expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.609828:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609ac00, rc 0) 00000100:00100000:0.0:1787739055.609901:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fe1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17468:x1874579652430976:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 495us (732us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739055.609908:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23241 00000100:00100000:1.0:1787739055.615706:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739055.616241:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652431104 00000100:00100000:2.0:1787739055.616250:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652431104 00000100:00100000:2.0:1787739055.616308:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23242 00000100:00100000:2.0:1787739055.616319:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fde00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17468:x1874579652431104:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739055.616347:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739055.616371:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1abe23 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094c1a expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.616459:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1abe2a lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.616472:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1abe2a lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.616483:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1abe2a lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.616497:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abe31 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.616504:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abe31 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.616511:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abe31 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.616546:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1abe2a lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.616553:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1abe2a lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.616561:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1abe2a lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.616567:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1abe2a lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.616573:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739055.616577:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1abe2a lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.616585:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1abe2a lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.616596:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5400) 00010000:00010000:2.0:1787739055.616599:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739055.616603:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1abe2a lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.616609:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1abe2a lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.616624:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abe31 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.616630:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abe31 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.616635:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abe31 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.616640:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739055.616643:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abe31 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.616649:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abe31 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.616655:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x367edb60 (ffffa074419d5800) 00010000:00010000:2.0:1787739055.616658:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739055.616661:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abe31 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.616666:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1abe31 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.616693:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1abe23 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c1a expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.616703:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5400) 00010000:00010000:2.0:1787739055.616705:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739055.616709:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1abe23 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c1a expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.616733:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633de00, rc 0) 00000100:00100000:2.0:1787739055.616795:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fde00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17468:x1874579652431104:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 477us (1097us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739055.616804:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23242 00000100:00100000:3.0:1787739055.623306:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739055.624787:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652431232 00000100:00100000:0.0:1787739055.624800:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652431232 00000100:00100000:1.0:1787739055.625212:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23243 00000100:00100000:1.0:1787739055.625225:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be7c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17468:x1874579652431232:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739055.625245:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739055.625269:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abe38 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094c21 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.625336:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abe3f lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.625350:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abe3f lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.625361:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abe3f lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739055.625408:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:1.0:1787739055.625521:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739055.625559:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abe38 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c21 expref: 19 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.625583:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abe3f lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c21 expref: 18 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.625591:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235500) 00010000:00010000:1.0:1787739055.625595:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739055.625598:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c200, rc 0) 00000100:00100000:1.0:1787739055.625694:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be7c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17468:x1874579652431232:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 472us (2399us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.625702:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23243 00000100:00100000:1.0:1787739055.636272:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739055.636369:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652431488 00000100:00100000:2.0:1787739055.636379:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652431488 00000100:00100000:2.0:1787739055.636628:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23244 00000100:00100000:2.0:1787739055.636639:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17468:x1874579652431488:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:2.0:1787739055.636765:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665572096:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.636817:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.636870:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068bae40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739055.637187:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665572096 00000100:00100000:3.0:1787739055.637193:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665572096 00000100:00100000:3.0:1787739055.637234:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2691 00000100:00100000:3.0:1787739055.637242:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665572096:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739055.637341:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665572096:12345-0@lo:1000:mdt00_004.0 Request processed in 100us (523us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.637349:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2691 00000100:00100000:0.0:1787739055.637892:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665572096:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739055.637953:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7b480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665572224:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739055.637991:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739055.638066:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739055.638562:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665572224 00000100:00100000:3.0:1787739055.638569:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665572224 00000100:00100000:3.0:1787739055.638602:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2692 00000100:00100000:3.0:1787739055.638609:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f783c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665572224:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739055.638695:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f783c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665572224:12345-0@lo:1000:mdt00_004.0 Request processed in 86us (702us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739055.638703:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2692 00000100:00100000:0.0:1787739055.639026:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7b480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665572224:0@lo:1000:mdt00_004.0 00010000:00010000:0.0:1787739055.639211:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abe46 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.639223:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abe46 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.639233:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abe46 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.639246:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7b480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665572352:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739055.639279:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739055.639312:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4b40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739055.639456:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665572352 00000100:00100000:3.0:1787739055.639461:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665572352 00000100:00100000:3.0:1787739055.639488:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2693 00000100:00100000:3.0:1787739055.639494:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665572352:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739055.639505:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739055.639519:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1abe4d lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abe46 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.639542:0:18535:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abe3f lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c21 expref: 17 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.639552:0:18535:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1abe4d lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abe46 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.639573:0:18535:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abe3f lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c21 expref: 17 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.639587:0:18535:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abe3f lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c21 expref: 17 pid: 7944 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739055.639598:0:18535:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be5e00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:18535:1874579665572480:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739055.639740:0:18535:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89900 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.644673:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739055.644777:0:18535:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5e00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:18535:1874579665572480:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739055.644813:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1abe4d lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abe46 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.644825:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235500) 00010000:00010000:3.0:1787739055.644828:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739055.644832:0:18535:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1abe4d lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abe46 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.644855:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa000, rc 0) 00000100:00100000:3.0:1787739055.644918:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665572352:12345-0@lo:101:mdt00_004.0 Request processed in 5423us (5638us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.644926:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2693 00000100:00100000:2.0:1787739055.645252:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7b480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665572352:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739055.645271:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abe46 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abe4d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.645283:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abe46 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abe4d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.645494:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652431616 00000100:00100000:0.0:1787739055.645501:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652431616 00010000:00010000:0.0:1787739055.645532:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abe3f lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c21 expref: 18 pid: 7944 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.645565:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1413 00000100:00100000:0.0:1787739055.645575:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2409:x1874579652431616:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739055.645590:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739055.645596:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abe3f lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c21 expref: 18 pid: 7944 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.645608:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abe3f lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c21 expref: 18 pid: 7944 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.645629:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abe3f lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c21 expref: 18 pid: 7944 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.645643:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235500) 00010000:00010000:0.0:1787739055.645648:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1abe4d lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abe46 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.645658:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1abe4d lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1abe46 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.645667:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1abe4d lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1abe46 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.645696:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1abe4d lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abe46 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.645711:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7a940 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665572608:0@lo:105: 00000100:00100000:0.0:1787739055.645750:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739055.645788:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4540 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.645869:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665572608 00000100:00100000:2.0:1787739055.645873:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665572608 00000100:00100000:2.0:1787739055.645909:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1079 00000100:00100000:2.0:1787739055.645916:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f79a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665572608:12345-0@lo:105: 00010000:00010000:2.0:1787739055.645925:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abe46 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abe4d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.645934:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abe46 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abe4d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.645942:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abe46 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abe4d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.645949:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abe46 lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1abe4d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.645959:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abe46 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1abe4d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.645965:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abe46 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1abe4d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.645989:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609ac00) 00000100:00100000:2.0:1787739055.646022:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f79a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665572608:12345-0@lo:105: Request processed in 107us (271us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739055.646029:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1079 00010000:00010000:3.0:1787739055.646204:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abe46 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abe4d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.646213:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abe46 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abe4d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739055.646233:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739055.646265:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abe54 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.646272:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abe54 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.646281:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abe54 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739055.646294:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be5e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665572736:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787739055.646321:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.646403:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89900 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.646985:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665572736 00000100:00100000:2.0:1787739055.646993:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665572736 00000100:00100000:2.0:1787739055.647055:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2694 00000100:00100000:2.0:1787739055.647064:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665572736:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739055.647096:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739055.647114:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1abe5b lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abe54 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.647137:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1abe5b lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abe54 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.647150:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1abe5b lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abe54 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.647158:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x367edb60 (ffffa074419d5400) 00010000:00010000:2.0:1787739055.647161:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739055.647164:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633de00, rc 0) 00000100:00100000:2.0:1787739055.647215:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665572736:12345-0@lo:101:mdt00_004.0 Request processed in 152us (893us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739055.647223:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2694 00000100:00100000:2.0:1787739055.647270:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665572736:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739055.647287:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abe54 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abe5b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.647296:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abe54 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abe5b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739055.647316:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:2.0:1787739055.647366:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665572864:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.647399:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.647446:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068bae40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.647464:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665572864 00000100:00100000:2.0:1787739055.647468:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665572864 00000100:00100000:2.0:1787739055.647499:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2695 00000100:00100000:2.0:1787739055.647506:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665572864:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.647565:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665572864:12345-0@lo:1000:mdt00_004.0 Request processed in 59us (164us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739055.647571:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2695 00000100:00100000:2.0:1787739055.647597:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665572864:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.647635:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665572992:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.647657:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.647675:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665572992 00000100:00100000:2.0:1787739055.647679:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665572992 00000100:00100000:2.0:1787739055.647698:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2696 00000100:00100000:2.0:1787739055.647704:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665572992:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.647774:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665572992:12345-0@lo:1000:mdt00_004.0 Request processed in 69us (116us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739055.647780:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2696 00000100:00100000:2.0:1787739055.647796:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068bae40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.647808:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665572992:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.647877:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665573120:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.647901:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.647919:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665573120 00000100:00100000:2.0:1787739055.647923:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665573120 00000100:00100000:2.0:1787739055.647942:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2697 00000100:00100000:2.0:1787739055.647948:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665573120:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.647989:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665573120:12345-0@lo:1000:mdt00_004.0 Request processed in 41us (87us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739055.647995:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2697 00000100:00100000:2.0:1787739055.648011:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068bae40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.648022:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665573120:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.648062:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665573248:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.648096:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.648115:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665573248 00000100:00100000:2.0:1787739055.648118:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665573248 00000100:00100000:2.0:1787739055.648139:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2698 00000100:00100000:2.0:1787739055.648144:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156efc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665573248:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.648186:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156efc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665573248:12345-0@lo:1000:mdt00_004.0 Request processed in 41us (89us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739055.648193:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2698 00000100:00100000:2.0:1787739055.648208:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068bae40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.648220:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665573248:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.648256:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665573376:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.648281:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.648299:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665573376 00000100:00100000:2.0:1787739055.648302:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665573376 00000100:00100000:2.0:1787739055.648322:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2699 00000100:00100000:2.0:1787739055.648326:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665573376:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.648372:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665573376:12345-0@lo:1000:mdt00_004.0 Request processed in 45us (91us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739055.648379:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2699 00000100:00100000:2.0:1787739055.648394:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068bae40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.648405:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ecf00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665573376:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.648460:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665573504:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.648481:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.648501:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665573504 00000100:00100000:2.0:1787739055.648504:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665573504 00000100:00100000:2.0:1787739055.648523:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2700 00000100:00100000:2.0:1787739055.648529:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f7c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665573504:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.648570:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f7c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665573504:12345-0@lo:1000:mdt00_004.0 Request processed in 41us (87us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739055.648576:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2700 00000100:00100000:2.0:1787739055.648590:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068bae40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.648601:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665573504:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.648632:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665573632:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.648653:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.648670:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665573632 00000100:00100000:2.0:1787739055.648673:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665573632 00000100:00100000:2.0:1787739055.648691:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2701 00000100:00100000:2.0:1787739055.648697:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665573632:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.648736:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665573632:12345-0@lo:1000:mdt00_004.0 Request processed in 39us (82us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739055.648741:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2701 00000100:00100000:2.0:1787739055.648756:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068bae40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.648767:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665573632:0@lo:1000:mdt00_004.0 00000040:00080000:2.0:1787739055.649533:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].337, 2168 off514088 00000040:00080000:2.0:1787739055.649616:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].338, 2368 off562536 00000004:00080000:2.0:1787739055.649623:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa073068ba480 with version 643 00000004:00080000:3.0:1787739055.650239:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 643 rpc_version 643 00000020:00100000:3.0:1787739055.650276:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739055.650286:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306be5a40 x1874579665573888/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739055.650308:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665573888:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739055.650384:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.650436:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739055.650461:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665573888 00000100:00100000:3.0:1787739055.650467:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665573888 00000100:00100000:3.0:1787739055.650511:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2702 00000100:00100000:3.0:1787739055.650518:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be6940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665573888:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739055.651376:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be6940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665573888:12345-0@lo:1000:osp_up0-1.0 Request processed in 856us (990us total) trans 4294983329 rc 0/0 00000100:00100000:3.0:1787739055.651386:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2702 00000100:00100000:3.0:1787739055.651479:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665573888:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739055.651915:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7a940 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665572608:0@lo:105: 00010000:00010000:0.0:1787739055.651933:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739055.651988:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652431616:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 6413us (7323us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.651996:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1413 00010000:00010000:2.0:1787739055.654244:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1abe62 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x163:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.654263:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1abe62 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x163:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.654273:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1abe62 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x163:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.654295:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abe69 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.654302:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abe69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.654313:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abe69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.654347:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abe70 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.654354:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abe70 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.654364:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abe70 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.654385:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665574016:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1787739055.654596:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.654666:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355ed780 going to sleep for 11 seconds 00000100:00100000:0.0:1787739055.655389:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665574016 00000100:00100000:0.0:1787739055.655397:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665574016 00000100:00100000:0.0:1787739055.655513:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2703 00000100:00100000:0.0:1787739055.655524:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ecf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665574016:12345-0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739055.655544:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739055.655562:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abe77 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abe70 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.655586:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abe77 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abe70 expref: 11 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.655599:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abe77 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abe70 expref: 11 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.655607:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffffa07307064f00) 00010000:00010000:0.0:1787739055.655610:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739055.655612:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609be00, rc 0) 00000100:00100000:0.0:1787739055.655667:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ecf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665574016:12345-0@lo:101:mdt00_004.0 Request processed in 144us (1071us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.655673:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2703 00000100:00100000:2.0:1787739055.655752:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665574016:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739055.655768:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abe70 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abe77 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.655779:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abe70 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abe77 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739055.655802:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000bc7dab9f reply state 00000000e7bc79fb transno 4294968318 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abe69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.655814:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abe69 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.655821:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb8:0x0].0x0 (ffffa074419d5500) 00010000:00010000:2.0:1787739055.655824:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739055.655828:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787739055.655833:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abe70 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abe77 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.655851:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abe70 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abe77 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739055.655862:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xb8:0x0] 00000004:00010000:2.0:1787739055.655869:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000bc7dab9f reply state 00000000e7bc79fb transno 4294968318 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1abe62 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x163:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.655877:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1abe62 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x163:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.655883:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x163:0x0].0x0 (ffffa074419d5800) 00010000:00010000:2.0:1787739055.655885:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739055.655888:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:2.0:1787739055.655892:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x163:0x0]: returning size 4096 00000004:00080000:2.0:1787739055.655900:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739055.655909:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abe46 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abe4d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.655917:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abe46 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abe4d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.655924:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abe46 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abe4d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.655952:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739055.655956:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739055.655961:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abe54 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abe5b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.655969:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abe54 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abe5b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.656050:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17468:x1874579652431488:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 19400us (19775us total) trans 4294968318 rc 0/0 00000100:00100000:2.0:1787739055.656058:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23244 00010000:00010000:1.0:1787739055.656342:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739055.656350:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abe46 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1abe4d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.656358:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ac00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739055.656362:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abe46 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abe4d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739055.656370:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abe46 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abe4d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.656399:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abe46 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abe4d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.656406:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739055.656450:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abe46 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abe4d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.656463:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abe46 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abe4d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.656480:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739055.656482:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739055.656484:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739055.656491:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787739055.656503:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fe580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665574144:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739055.656540:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.659789:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665574144 00000100:00100000:3.0:1787739055.659803:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665574144 00000100:00100000:3.0:1787739055.659881:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1214 00000100:00100000:3.0:1787739055.659891:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+11:3651:x1874579665574144:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739055.659908:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739055.659932:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1abe4d lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abe46 expref: 11 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.659950:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235500) 00010000:00010000:3.0:1787739055.659958:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739055.660207:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fe580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665574144:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739055.660304:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+10:3651:x1874579665574144:12345-0@lo:103:ldlm_bl.0 Request processed in 414us (3762us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.660311:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1214 00000100:00100000:2.0:1787739055.676800:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739055.676978:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652431872 00000100:00100000:0.0:1787739055.676989:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652431872 00000100:00100000:0.0:1787739055.677061:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23245 00000100:00100000:0.0:1787739055.677082:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17469:x1874579652431872:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:0.0:1787739055.677235:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17469:x1874579652431872:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 165us (442us total) trans 0 rc -17/-17 00000100:00100000:0.0:1787739055.677243:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23245 00000100:00100000:2.0:1787739055.693171:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739055.693315:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652432000 00000100:00100000:1.0:1787739055.693335:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652432000 00000100:00100000:1.0:1787739055.693394:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23246 00000100:00100000:1.0:1787739055.693404:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c383840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17470:x1874579652432000:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739055.693458:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739055.693481:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1abe7e lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094c28 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.693545:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abe85 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.693558:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abe85 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.693568:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abe85 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.693582:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1abe8c lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.693593:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1abe5b lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abe54 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.693602:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1abe8c lrc: 4/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.693633:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1abe5b lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1abe54 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739055.693648:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fe580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665574272:0@lo:104: 00000100:00100000:1.0:1787739055.693694:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739055.693744:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739055.693896:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665574272 00000100:00100000:3.0:1787739055.693901:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665574272 00000100:00100000:3.0:1787739055.693938:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1080 00000100:00100000:3.0:1787739055.693946:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061ff480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665574272:12345-0@lo:104: 00010000:00010000:3.0:1787739055.693958:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abe54 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1abe5b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.694027:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739055.694033:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061ff480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665574272:12345-0@lo:104: Request processed in 89us (338us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.694039:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1080 00000100:00100000:0.0:1787739055.694176:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fe580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665574272:0@lo:104: 00010000:00010000:0.0:1787739055.694195:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1abe8c lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.694614:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739055.694622:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abe54 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1abe5b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.694632:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aae00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739055.694637:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abe54 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1abe5b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739055.694645:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abe54 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1abe5b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.694675:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abe54 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abe5b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.694682:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739055.694720:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abe54 lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abe5b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.694734:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aae00/0x7ee4453e4a1abe54 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abe5b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.694752:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739055.694754:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739055.694755:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739055.694856:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c3812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665574400:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739055.694898:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.694982:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665574400 00000100:00100000:2.0:1787739055.694988:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665574400 00010000:00010000:2.0:1787739055.695010:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1abe5b lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1abe54 expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.695049:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1414 00000100:00100000:2.0:1787739055.695057:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7a940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+10:3651:x1874579665574400:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739055.695079:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:2.0:1787739055.705801:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983329 is committed 00000001:00080000:2.0:1787739055.705870:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983328 is committed 00000020:00080000:2.0:1787739055.705880:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 351 batchid updated 00000020:00080000:2.0:1787739055.705900:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07305fcc540 refcount 2 committed 0 result 0 batchid 352 00000020:00080000:2.0:1787739055.705905:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c695f00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07305fcc240 00000020:00080000:2.0:1787739055.705909:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].349 00000020:00080000:2.0:1787739055.705912:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c695000 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073069be800 00000020:00080000:2.0:1787739055.705915:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].350 00000020:00080000:3.0:1787739055.706066:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 352 committed batchid 351 00000004:00080000:3.0:1787739055.706193:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07302e89a80 with version 669 00000020:00080000:3.0:1787739055.706208:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 351 cancel update log [0x240000401:0x3:0x0].348: rc = 0 00000020:00080000:3.0:1787739055.706249:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 351 cancel update log [0x200000400:0x3:0x0].349: rc = 0 00000020:00080000:3.0:1787739055.706257:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 351 00010000:00010000:1.0:1787739055.706362:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1abe5b lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1abe54 expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.706381:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x367edb60 (ffffa074419d5400) 00010000:00010000:1.0:1787739055.706386:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1abe8c lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.706396:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1abe8c lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.706403:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1abe8c lrc: 4/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.706462:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739055.706537:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7a940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+6:3651:x1874579665574400:12345-0@lo:103:ldlm_bl.0 Request processed in 11479us (11637us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.706546:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1414 00000100:00100000:1.0:1787739055.706603:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c3812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665574400:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739055.706670:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1abe8c lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.706678:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1abe8c lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739055.706779:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be5e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665574528:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739055.706822:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.706869:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89900 going to sleep for 11 seconds 00000004:00080000:2.0:1787739055.707032:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 669 rpc_version 669 00000020:00100000:2.0:1787739055.707050:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739055.707058:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743c382940 x1874579665574784/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739055.707090:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c382940 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665574784:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739055.707118:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.707133:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355eda80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739055.707177:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665574528 00000100:00100000:1.0:1787739055.707183:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665574528 00000100:00100000:1.0:1787739055.707223:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665574784 00000100:00100000:1.0:1787739055.707227:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665574784 00000100:00100000:1.0:1787739055.707248:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2704 00000100:00100000:1.0:1787739055.707255:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f71680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665574528:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739055.707326:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f71680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665574528:12345-0@lo:1000:mdt00_003.0 Request processed in 71us (504us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.707332:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2704 00000100:00100000:1.0:1787739055.707345:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2705 00000100:00100000:1.0:1787739055.707352:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c380f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665574784:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739055.707468:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c380f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665574784:12345-0@lo:1000:osp_up1-0.0 Request processed in 115us (348us total) trans 4294968319 rc 0/0 00000100:00100000:1.0:1787739055.707474:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2705 00000100:00100000:1.0:1787739055.707508:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c382940 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665574784:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739055.707673:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665574528:0@lo:1000:mdt00_003.0 00010000:00010000:3.0:1787739055.707727:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1abe93 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x163:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.707740:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1abe93 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x163:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.707754:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1abe93 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x163:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739055.707774:0:7944:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1abe93 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x163:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.707787:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abe85 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.707794:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abe85 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.707801:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abe85 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.707808:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abe85 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.707813:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739055.707817:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abe85 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.707824:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abe85 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.707834:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235100) 00010000:00010000:3.0:1787739055.707846:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739055.707850:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abe85 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.707856:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1abe85 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.707868:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1abe8c lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.707875:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1abe8c lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.707880:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1abe8c lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.707886:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739055.707889:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1abe8c lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.707895:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1abe8c lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.707902:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x367edb60 (ffffa074419d5400) 00010000:00010000:3.0:1787739055.707904:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739055.707907:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1abe8c lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.707912:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1abe8c lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.707947:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1abe7e lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c28 expref: 17 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.707964:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1abe93 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x163:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c28 expref: 16 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.707977:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x163:0x0].0x0 (ffffa07305d56e00) 00010000:00010000:3.0:1787739055.707979:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739055.707982:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa000, rc 0) 00000100:00100000:3.0:1787739055.708030:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c383840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17470:x1874579652432000:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 14627us (14867us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.708037:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23246 00010000:00010000:0.0:1787739055.710977:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1abd89 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.710994:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1abd89 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711002:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1abd89 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711008:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739055.711013:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1abd89 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711021:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1abd89 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711053:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xade5bda9 (ffffa07305d56100) 00010000:00010000:0.0:1787739055.711056:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739055.711061:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1abd89 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711078:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1abd89 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711096:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abd90 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x14fe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711101:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abd90 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x14fe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711107:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abd90 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x14fe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711111:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739055.711114:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abd90 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x14fe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711120:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abd90 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x14fe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711128:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14fe:0x0].0x0 (ffffa07433235f00) 00010000:00010000:0.0:1787739055.711131:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739055.711134:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abd90 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x14fe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711139:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abd90 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x14fe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711158:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1abd97 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711166:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1abd97 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711171:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1abd97 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711176:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739055.711179:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1abd97 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711185:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1abd97 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711192:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xba:0x0].0x0 (ffffa07433235e00) 00010000:00010000:0.0:1787739055.711195:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739055.711198:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1abd97 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711203:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1abd97 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711221:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abdeb lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711227:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abdeb lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711233:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abdeb lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711238:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739055.711241:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abdeb lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711247:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abdeb lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711254:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xade5bda9 (ffffa07305d56100) 00010000:00010000:0.0:1787739055.711256:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739055.711259:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abdeb lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.711264:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1abdeb lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xade5bda9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739055.713137:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739055.713283:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652432128 00000100:00100000:2.0:1787739055.713294:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652432128 00000100:00100000:2.0:1787739055.713352:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23247 00000100:00100000:2.0:1787739055.713362:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f71a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17470:x1874579652432128:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739055.713387:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739055.713406:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1abe9a lrc: 2/0,0 mode: --/CR res: [0x240000402:0x163:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094c2f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.713488:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1abea1 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x163:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.713500:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1abea1 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x163:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.713510:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1abea1 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x163:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739055.713601:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x163:0x0]: returning size 4096 00000004:00200000:2.0:1787739055.713606:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739055.713639:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1abe9a lrc: 0/0,0 mode: --/CR res: [0x240000402:0x163:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c2f expref: 11 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.713656:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1abea1 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x163:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c2f expref: 10 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.713668:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x163:0x0].0x0 (ffffa074419d5800) 00010000:00010000:2.0:1787739055.713671:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739055.713674:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d400, rc 0) 00000100:00100000:2.0:1787739055.714210:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f71a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17470:x1874579652432128:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 846us (1078us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739055.714218:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23247 00000100:00100000:2.0:1787739055.719270:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739055.719343:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652432256 00000100:00100000:1.0:1787739055.719353:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652432256 00000100:00100000:1.0:1787739055.719442:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23248 00000100:00100000:1.0:1787739055.719453:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c380780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17470:x1874579652432256:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739055.719473:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739055.719493:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1abea8 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094c36 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.719557:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abeaf lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.719572:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abeaf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.719583:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abeaf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739055.719632:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xb8:0x0]: returning size 4096 00000004:00200000:1.0:1787739055.719637:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739055.719669:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1abea8 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c36 expref: 12 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.719687:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abeaf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c36 expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.719696:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb8:0x0].0x0 (ffffa074419d5500) 00010000:00010000:1.0:1787739055.719699:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739055.719703:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c200, rc 0) 00000100:00100000:1.0:1787739055.719766:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c380780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17470:x1874579652432256:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 315us (505us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.719772:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23248 00000100:00100000:3.0:1787739055.724459:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739055.724559:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652432384 00000100:00100000:0.0:1787739055.724568:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652432384 00000100:00100000:0.0:1787739055.724620:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23249 00000100:00100000:0.0:1787739055.724631:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f721c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17470:x1874579652432384:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739055.724650:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739055.724669:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1abeb6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094c3d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.724738:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1abebd lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.724753:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abe77 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abe70 expref: 5 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.724766:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1abebd lrc: 4/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.724798:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abe77 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1abe70 expref: 5 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.724813:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f78b40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665574912:0@lo:104: 00000100:00100000:0.0:1787739055.724856:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739055.724888:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739055.725244:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665574912 00000100:00100000:3.0:1787739055.725249:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665574912 00000100:00100000:3.0:1787739055.725287:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1081 00000100:00100000:3.0:1787739055.725294:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7a580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665574912:12345-0@lo:104: 00010000:00010000:3.0:1787739055.725306:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abe70 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1abe77 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.725481:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739055.725488:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7a580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665574912:12345-0@lo:104: Request processed in 195us (631us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.725496:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1081 00000100:00100000:0.0:1787739055.725765:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f78b40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665574912:0@lo:104: 00010000:00010000:0.0:1787739055.725783:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1abebd lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.726372:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739055.726381:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abe70 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1abe77 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.726390:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739055.726396:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abe70 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1abe77 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739055.726406:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abe70 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1abe77 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.726465:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abe70 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abe77 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.726472:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1787739055.726527:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00010000:00010000:2.0:1787739055.726547:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abe70 lrc: 1/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abe77 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.726555:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abe70 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abe77 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.726578:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739055.726580:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739055.726582:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739055.726687:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665575040:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739055.726799:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739055.726957:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665575040 00000100:00100000:0.0:1787739055.726963:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665575040 00010000:00010000:0.0:1787739055.726988:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abe77 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1abe70 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.727170:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1415 00000100:00100000:0.0:1787739055.727179:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fd2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+6:3653:x1874579665575040:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739055.727191:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00080000:2.0:1787739055.729267:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 352 batchid updated 00000020:00080000:0.0:1787739055.729308:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 352 00000004:00080000:0.0:1787739055.729371:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa073068b4540 with version 670 00000020:00080000:0.0:1787739055.729387:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 352 cancel update log [0x240000401:0x3:0x0].349: rc = 0 00000020:00080000:0.0:1787739055.729476:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 352 cancel update log [0x200000400:0x3:0x0].350: rc = 0 00000020:00080000:0.0:1787739055.729487:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 352 00000004:00080000:1.0:1787739055.729535:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 670 rpc_version 670 00000020:00100000:1.0:1787739055.729566:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739055.729576:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073061ff840 x1874579665575296/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739055.729599:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061ff840 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665575296:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739055.729663:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739055.729699:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073070540c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739055.729757:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665575296 00000100:00100000:0.0:1787739055.729763:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665575296 00000100:00100000:0.0:1787739055.729815:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2706 00000100:00100000:0.0:1787739055.729824:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fcb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665575296:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739055.730020:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fcb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665575296:12345-0@lo:1000:osp_up1-0.0 Request processed in 197us (356us total) trans 4294968320 rc 0/0 00000100:00100000:0.0:1787739055.730027:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2706 00000100:00100000:1.0:1787739055.730318:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061ff840 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665575296:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787739055.730432:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1abe77 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1abe70 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.730449:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffffa07307064f00) 00010000:00010000:0.0:1787739055.730454:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1abebd lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.730463:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1abebd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.730471:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1abebd lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.730540:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739055.730580:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fd2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+5:3653:x1874579665575040:12345-0@lo:103:ldlm_bl.0 Request processed in 3403us (3781us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.730588:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1415 00000100:00100000:2.0:1787739055.730659:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665575040:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739055.730958:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1abebd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.730966:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1abebd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739055.731012:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb8:0x0]: returning size 4096 00000004:00200000:1.0:1787739055.731017:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739055.731192:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1abeb6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c3d expref: 16 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.731210:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1abebd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c3d expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.731218:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffffa07307064f00) 00010000:00010000:1.0:1787739055.731220:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739055.731223:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609ba00, rc 0) 00000100:00100000:1.0:1787739055.731307:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f721c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17470:x1874579652432384:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 6678us (6899us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.731315:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23249 00000100:00100000:3.0:1787739055.736276:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739055.736310:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739055.736521:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652432512 00000100:00100000:1.0:1787739055.736529:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652432512 00000100:00100000:1.0:1787739055.736585:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1215 00000100:00100000:1.0:1787739055.736595:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074301a2850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2409:x1874579652432512:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:1.0:1787739055.736612:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739055.736636:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1abe93 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x163:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c28 expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.736653:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x163:0x0].0x0 (ffffa07305d56e00) 00010000:00010000:1.0:1787739055.736661:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739055.736707:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074301a2850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652432512:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 114us (396us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.736715:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1215 00000100:00100000:2.0:1787739055.737639:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652432768 00000100:00100000:2.0:1787739055.737647:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652432768 00000100:00100000:2.0:1787739055.737690:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23250 00000100:00100000:2.0:1787739055.737698:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f730c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17470:x1874579652432768:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:2.0:1787739055.737756:0:13793:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739055.737774:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1abea1 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x163:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c2f expref: 11 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.737790:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x163:0x0].0x0 (ffffa074419d5800) 00010000:00010000:2.0:1787739055.737794:0:13793:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739055.737902:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abec4 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.737911:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abec4 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.737921:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abec4 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.737941:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665575424:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1787739055.737987:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.738017:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355eda80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739055.738163:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665575424 00000100:00100000:3.0:1787739055.738168:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665575424 00000100:00100000:3.0:1787739055.738228:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2707 00000100:00100000:3.0:1787739055.738236:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c381680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+5:13793:x1874579665575424:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739055.738252:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739055.738268:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1abecb lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abec4 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.738289:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1abecb lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abec4 expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.738300:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1abecb lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abec4 expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.738308:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305d56100) 00010000:00010000:3.0:1787739055.738310:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739055.738313:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab600, rc 0) 00000100:00100000:3.0:1787739055.738367:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c381680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:13793:x1874579665575424:12345-0@lo:101:mdt00_004.0 Request processed in 132us (378us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.738374:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2707 00000100:00100000:2.0:1787739055.738459:0:13793:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983329 gen 2 00000004:00080000:2.0:1787739055.738465:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983325 committed transno 4294983329 00000020:00080000:2.0:1787739055.738470:0:13793:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa073068ba780 refcount 2 committed 0 result 0 batchid 343 00000020:00080000:2.0:1787739055.738474:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730415f700 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa073068bac00 00000020:00080000:2.0:1787739055.738477:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].337 00000020:00080000:2.0:1787739055.738480:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730415fc00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304011600 00000020:00080000:2.0:1787739055.738482:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].336 00000004:00080000:2.0:1787739055.738503:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983327 committed transno 4294983329 00000004:00080000:2.0:1787739055.738514:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983329 committed transno 4294983329 00000100:00100000:2.0:1787739055.738530:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665575424:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739055.738543:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abec4 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abecb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.738553:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abec4 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abecb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739055.738567:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739055.738585:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1abed2 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.738591:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1abed2 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.738599:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1abed2 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.738609:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665575552:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1787739055.738632:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000020:00080000:1.0:1787739055.738681:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 343 committed batchid 341 00000020:00080000:1.0:1787739055.738725:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 341 00000100:00100000:2.0:1787739055.741281:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355eda80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739055.741391:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665575552 00000100:00100000:0.0:1787739055.741399:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665575552 00000100:00100000:0.0:1787739055.741619:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2708 00000100:00100000:0.0:1787739055.741629:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c3821c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:13793:x1874579665575552:12345-0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739055.741648:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739055.741670:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1abed9 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abed2 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.741699:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1abed9 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abed2 expref: 7 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.741723:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1abed9 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abed2 expref: 7 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.741732:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x367edb60 (ffffa073068a9b00) 00010000:00010000:0.0:1787739055.741734:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739055.741738:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a600, rc 0) 00000100:00100000:0.0:1787739055.741797:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c3821c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665575552:12345-0@lo:101:mdt00_004.0 Request processed in 170us (3164us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.741804:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2708 00000100:00100000:2.0:1787739055.742016:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665575552:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739055.742033:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1abed2 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abed9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.742223:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1abed2 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abed9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739055.742247:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:2.0:1787739055.742300:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665575680:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.742353:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.742436:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355ed780 going to sleep for 11 seconds 00000100:00100000:0.0:1787739055.742621:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665575680 00000100:00100000:0.0:1787739055.742627:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665575680 00000100:00100000:0.0:1787739055.742667:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2709 00000100:00100000:0.0:1787739055.742675:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c3830c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665575680:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739055.742989:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c3830c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665575680:12345-0@lo:1000:mdt00_004.0 Request processed in 315us (636us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.742997:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2709 00000100:00100000:2.0:1787739055.743267:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665575680:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.743316:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665575808:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.745972:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.746187:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba600 going to sleep for 11 seconds 00000100:00100000:3.0:1787739055.752608:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665575808 00000100:00100000:3.0:1787739055.752623:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665575808 00000100:00100000:3.0:1787739055.753321:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2710 00000100:00100000:3.0:1787739055.753335:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c383480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665575808:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739055.753514:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c383480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665575808:12345-0@lo:1000:mdt00_004.0 Request processed in 182us (7549us total) trans 0 rc 1/1 00000100:00100000:3.0:1787739055.753521:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2710 00000100:00100000:0.0:1787739055.753613:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665575808:0@lo:1000:mdt00_004.0 00010000:00010000:0.0:1787739055.753687:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1abee0 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x163:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.753698:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1abee0 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x163:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.753712:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1abee0 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x163:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.753742:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f78b40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665575936:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739055.753778:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739055.754349:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.754587:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665575936 00000100:00100000:2.0:1787739055.754594:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665575936 00000100:00100000:2.0:1787739055.754638:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2711 00000100:00100000:2.0:1787739055.754646:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f792c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665575936:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739055.754661:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739055.754677:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1abee7 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x163:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abee0 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.754702:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1abee7 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x163:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abee0 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.755029:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1abee7 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x163:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abee0 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.755160:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x163:0x0].0x0 (ffffa074419d5300) 00010000:00010000:2.0:1787739055.755163:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739055.755167:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d400, rc 0) 00000100:00100000:2.0:1787739055.755234:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f792c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665575936:12345-0@lo:101:mdt00_004.0 Request processed in 588us (1454us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739055.755241:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2711 00000100:00100000:0.0:1787739055.755324:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f78b40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665575936:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739055.755342:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1abee0 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x163:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abee7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.755353:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1abee0 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x163:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abee7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.755381:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1abeee lrc: 3/0,1 mode: --/EX res: [0x240000402:0x163:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.755390:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1abeee lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x163:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.755398:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1abeee lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x163:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.755512:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abef5 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.755526:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abeaf lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c36 expref: 10 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.755534:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abef5 lrc: 4/0,1 mode: --/EX res: [0x240000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.755561:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abeaf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c36 expref: 10 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.755575:0:13793:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abeaf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c36 expref: 10 pid: 6523 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.755587:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f78b40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665576064:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739055.755647:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739055.760060:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f78b40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665576064:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739055.760210:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abef5 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739055.762754:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739055.763508:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652432896 00000100:00100000:1.0:1787739055.763518:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652432896 00010000:00010000:1.0:1787739055.763557:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abeaf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c36 expref: 11 pid: 6523 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739055.763595:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1416 00000100:00100000:1.0:1787739055.763604:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f73c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2408:x1874579652432896:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739055.763619:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739055.763625:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abeaf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c36 expref: 11 pid: 6523 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.763639:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abeaf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c36 expref: 11 pid: 6523 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.763665:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abeaf lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c36 expref: 11 pid: 6523 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.763680:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb8:0x0].0x0 (ffffa074419d5500) 00010000:00010000:1.0:1787739055.763685:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abef5 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.763695:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abef5 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.763702:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abef5 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.763946:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739055.764006:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f73c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652432896:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 402us (1257us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.764015:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1416 00010000:00010000:0.0:1787739055.766628:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abef5 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.766645:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abef5 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.766707:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abefc lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.766725:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abefc lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.766738:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abefc lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.766763:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f78b40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665576192:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739055.767158:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739055.767201:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.767282:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665576192 00000100:00100000:2.0:1787739055.767290:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665576192 00000100:00100000:2.0:1787739055.767351:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2712 00000100:00100000:2.0:1787739055.767359:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665576192:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739055.767373:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739055.767390:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1abf03 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abefc expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.767648:0:18535:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1abebd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c3d expref: 13 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.767660:0:18535:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1abf03 lrc: 4/0,0 mode: --/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abefc expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.767685:0:18535:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1abebd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c3d expref: 13 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.767699:0:18535:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1abebd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c3d expref: 13 pid: 7944 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.767919:0:18535:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c381a40 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:18535:1874579665576320:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739055.768028:0:18535:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.776539:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739055.776609:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652433024 00000100:00100000:0.0:1787739055.776618:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652433024 00010000:00010000:0.0:1787739055.776659:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1abebd lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c3d expref: 14 pid: 7944 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.776700:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1417 00000100:00100000:0.0:1787739055.776711:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c380000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652433024:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739055.776736:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739055.776743:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1abebd lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c3d expref: 14 pid: 7944 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.776755:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1abebd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c3d expref: 14 pid: 7944 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.776777:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffffa07307064f00) 00010000:00010000:0.0:1787739055.776784:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1abf03 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abefc expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.776804:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1abf03 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1abefc expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.776814:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1abf03 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1abefc expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.776856:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1abf03 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abefc expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.776876:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665576448:0@lo:105: 00000100:00100000:0.0:1787739055.776925:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739055.776973:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739055.777052:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665576448 00000100:00100000:3.0:1787739055.777057:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665576448 00000100:00100000:3.0:1787739055.777108:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1082 00000100:00100000:3.0:1787739055.777116:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f79680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665576448:12345-0@lo:105: 00010000:00010000:3.0:1787739055.777127:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abefc lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abf03 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.777138:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abefc lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abf03 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.777146:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abefc lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abf03 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.777153:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abefc lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1abf03 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.777161:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abefc lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1abf03 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.777168:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abefc lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1abf03 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.777182:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609ac00) 00000100:00100000:3.0:1787739055.777237:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f79680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665576448:12345-0@lo:105: Request processed in 122us (311us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.777244:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1082 00000100:00100000:3.0:1787739055.777282:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665576448:0@lo:105: 00010000:00010000:3.0:1787739055.777298:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739055.777349:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c380000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652433024:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 639us (817us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.777356:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1417 00010000:00010000:1.0:1787739055.781266:0:18535:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1abebd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c3d expref: 13 pid: 7944 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739055.781290:0:18535:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c381a40 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:18535:1874579665576320:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739055.781318:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1abf03 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abefc expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.781328:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffffa07307064f00) 00010000:00010000:1.0:1787739055.781331:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739055.781334:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d600, rc 0) 00000100:00100000:0.0:1787739055.781625:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665576192:12345-0@lo:101:mdt00_004.0 Request processed in 14264us (14466us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.781633:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2712 00000100:00100000:2.0:1787739055.781693:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f78b40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665576192:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739055.781711:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abefc lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abf03 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.782915:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7a940 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665576576:0@lo:403:mdt00_004.0 00000100:00100000:0.0:1787739055.782971:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739055.783028:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4dcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739055.783186:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665576576 00000100:00100000:1.0:1787739055.783194:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665576576 00000100:00100000:1.0:1787739055.783278:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2713 00000100:00100000:1.0:1787739055.783288:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665576576:12345-0@lo:403:mdt00_004.0 00000100:00100000:1.0:1787739055.786267:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665576576:12345-0@lo:403:mdt00_004.0 Request processed in 2971us (3285us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.786282:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2713 00000100:00100000:2.0:1787739055.786694:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7a940 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665576576:0@lo:403:mdt00_004.0 00000040:00080000:2.0:1787739055.788351:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].338, 664 off514752 00000040:00080000:2.0:1787739055.788778:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].339, 744 off563280 00000004:00080000:2.0:1787739055.788788:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa073068ba3c0 with version 644 00000004:00080000:3.0:1787739055.789014:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 644 rpc_version 644 00000020:00100000:3.0:1787739055.789173:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739055.789183:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303f712c0 x1874579665576832/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1808/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739055.789206:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f712c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665576832:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739055.789267:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.789353:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89900 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.789859:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665576832 00000100:00100000:2.0:1787739055.789867:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665576832 00000100:00100000:2.0:1787739055.789922:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2714 00000100:00100000:2.0:1787739055.789932:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f72580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665576832:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739055.790727:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f72580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665576832:12345-0@lo:1000:osp_up0-1.0 Request processed in 793us (1456us total) trans 4294983330 rc 0/0 00000100:00100000:2.0:1787739055.790735:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2714 00000100:00100000:1.0:1787739055.791270:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f712c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665576832:0@lo:1000:osp_up0-1.0 00000004:00200000:2.0:1787739055.791326:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x163:0x0]: returning size 4096 00010000:00010000:2.0:1787739055.791347:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abef5 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.791357:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abef5 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.791364:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abef5 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.791371:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739055.791376:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abef5 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.791384:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abef5 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.791400:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb8:0x0].0x0 (ffffa074419d5500) 00010000:00010000:2.0:1787739055.791532:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739055.791539:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abef5 lrc: 2/0,0 mode: --/EX res: [0x240000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.791545:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1abef5 lrc: 0/0,0 mode: --/EX res: [0x240000400:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.791561:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abefc lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abf03 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.791568:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abefc lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abf03 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.791576:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abefc lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abf03 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.791632:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739055.791641:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xb8:0x0] 00010000:00010000:2.0:1787739055.791648:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1abeee lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x163:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.791654:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1abeee lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x163:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.791661:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1abeee lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x163:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.791666:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739055.791669:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1abeee lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x163:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.791674:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1abeee lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x163:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.791683:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x163:0x0].0x0 (ffffa074419d5800) 00010000:00010000:2.0:1787739055.791686:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739055.791689:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1abeee lrc: 2/0,0 mode: --/EX res: [0x240000402:0x163:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.791695:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1abeee lrc: 0/0,0 mode: --/EX res: [0x240000402:0x163:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.791704:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1abee0 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x163:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abee7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.791721:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1abee0 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x163:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abee7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.791729:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1abee0 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x163:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abee7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.791811:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787739055.791819:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abefc lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1abf03 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.791827:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ac00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739055.791831:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abefc lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abf03 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739055.791956:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abefc lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abf03 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.791991:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abefc lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abf03 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.791998:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1787739055.792077:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665576960:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739055.792131:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.792212:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665576960 00000100:00100000:3.0:1787739055.792219:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665576960 00000100:00100000:3.0:1787739055.792265:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1216 00000100:00100000:3.0:1787739055.792274:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fe940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+10:3653:x1874579665576960:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739055.792286:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739055.792305:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1abf03 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abefc expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.792318:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb8:0x0].0x0 (ffffa07307064f00) 00010000:00010000:3.0:1787739055.792327:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739055.792451:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abefc lrc: 1/0,0 mode: --/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abf03 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.792465:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1abefc lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abf03 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.792481:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739055.792485:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1abee0 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x163:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1abee7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.792492:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609be00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739055.792496:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1abee0 lrc: 5/0,0 mode: EX/EX res: [0x240000402:0x163:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abee7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739055.792503:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1abee0 lrc: 5/0,0 mode: EX/EX res: [0x240000402:0x163:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abee7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.792520:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1abee0 lrc: 4/0,0 mode: --/EX res: [0x240000402:0x163:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abee7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.792527:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739055.792542:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1abee0 lrc: 3/0,0 mode: --/EX res: [0x240000402:0x163:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abee7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.792550:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739055.792552:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739055.792554:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739055.792733:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:2.0:1787739055.792740:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1abee0 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x163:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abee7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739055.792764:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739055.792768:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abec4 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abecb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.792775:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abec4 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abecb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.792783:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abec4 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abecb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.792806:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739055.792811:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739055.792818:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739055.792822:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abec4 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1abecb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.792828:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739055.792831:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abec4 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abecb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739055.792847:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abec4 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abecb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.792866:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abec4 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abecb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.792872:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739055.792878:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abec4 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abecb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.792884:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1abec4 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abecb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.792896:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739055.792898:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739055.792899:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739055.792909:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1abed2 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abed9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.792918:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1abed2 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abed9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.792982:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f730c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17470:x1874579652432768:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 55282us (56712us total) trans 4294968321 rc 0/0 00000100:00100000:2.0:1787739055.792990:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23250 00010000:00010000:0.0:1787739055.794595:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739055.794879:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fe940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3653:x1874579665576960:12345-0@lo:103:ldlm_bl.0 Request processed in 2601us (2742us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.794891:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1216 00000100:00100000:3.0:1787739055.794927:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f78b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665577088:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739055.794977:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.795014:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787739055.795021:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c3812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665577216:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739055.795053:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739055.795557:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665577088 00000100:00100000:0.0:1787739055.795566:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665577088 00000100:00100000:0.0:1787739055.795608:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665577216 00000100:00100000:0.0:1787739055.795613:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665577216 00000100:00100000:0.0:1787739055.795636:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1217 00000100:00100000:0.0:1787739055.795645:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f70b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+10:3650:x1874579665577088:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739055.795658:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739055.795680:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1abee7 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x163:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abee0 expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.795699:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x163:0x0].0x0 (ffffa074419d5300) 00010000:00010000:0.0:1787739055.795709:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739055.795744:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f70b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+9:3650:x1874579665577088:12345-0@lo:103:ldlm_bl.0 Request processed in 100us (767us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.795749:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1217 00000100:00100000:0.0:1787739055.795763:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1218 00000100:00100000:0.0:1787739055.795769:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3650:x1874579665577216:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739055.795777:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739055.795788:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1abecb lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abec4 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.795797:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305d56100) 00010000:00010000:0.0:1787739055.795804:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739055.795825:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+7:3650:x1874579665577216:12345-0@lo:103:ldlm_bl.0 Request processed in 57us (772us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739055.795831:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1218 00000100:00100000:3.0:1787739055.796042:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c3812c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665577216:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739055.796058:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f78b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665577088:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739055.799342:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665576960:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739055.825908:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739055.826732:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652433152 00000100:00100000:1.0:1787739055.826744:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652433152 00000100:00100000:1.0:1787739055.826812:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23251 00000100:00100000:1.0:1787739055.826823:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c3812c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17471:x1874579652433152:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739055.826853:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739055.826874:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abf0a lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094c44 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.826944:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1abf11 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.826958:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1abf11 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.826970:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1abf11 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.826990:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abf18 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.826997:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abf18 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.827003:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abf18 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.827053:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1abf11 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.827060:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1abf11 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.827578:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1abf11 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.827588:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1abf11 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.827594:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ce00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739055.827598:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1abf11 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.827606:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1abf11 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.827618:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064300) 00010000:00010000:1.0:1787739055.827621:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739055.827626:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1abf11 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.827632:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1abf11 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.827647:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abf18 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.827653:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abf18 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.827659:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abf18 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.827665:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1cc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739055.827667:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abf18 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.827674:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abf18 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.827681:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xe09b0941 (ffffa07433235e00) 00010000:00010000:1.0:1787739055.827683:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739055.827685:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abf18 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.827691:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abf18 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.827734:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abf0a lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c44 expref: 11 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.827749:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064300) 00010000:00010000:1.0:1787739055.827752:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739055.827755:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abf0a lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c44 expref: 11 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.827765:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c200, rc 0) 00000100:00100000:3.0:1787739055.829552:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c3812c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17471:x1874579652433152:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 2722us (3643us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739055.829565:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23251 00000100:00100000:2.0:1787739055.837617:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739055.837774:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652433280 00000100:00100000:1.0:1787739055.837783:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652433280 00000100:00100000:1.0:1787739055.837891:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23252 00000100:00100000:1.0:1787739055.837902:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c381a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17471:x1874579652433280:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739055.837920:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739055.837941:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abf1f lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094c4b expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.837999:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1abf26 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.838012:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1abf26 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.838024:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1abf26 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739055.838235:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:1.0:1787739055.838242:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739055.838290:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abf1f lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c4b expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.838314:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1abf26 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c4b expref: 14 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.838322:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235f00) 00010000:00010000:1.0:1787739055.838325:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739055.838328:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c600, rc 0) 00000100:00100000:1.0:1787739055.838390:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c381a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17471:x1874579652433280:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 490us (1141us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.838398:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23252 00000100:00100000:2.0:1787739055.851833:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739055.852156:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652433536 00000100:00100000:0.0:1787739055.852166:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652433536 00000100:00100000:0.0:1787739055.852230:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23253 00000100:00100000:0.0:1787739055.852240:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d86940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17471:x1874579652433536:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:0.0:1787739055.852371:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665577344:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739055.852465:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739055.852514:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d000 going to sleep for 11 seconds 00000100:00100000:1.0:1787739055.852730:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665577344 00000100:00100000:1.0:1787739055.852735:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665577344 00000100:00100000:1.0:1787739055.852774:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2715 00000100:00100000:1.0:1787739055.852783:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665577344:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739055.852877:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665577344:12345-0@lo:1000:mdt00_004.0 Request processed in 94us (409us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.852883:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2715 00000100:00100000:0.0:1787739055.853213:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665577344:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739055.853267:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665577472:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739055.853292:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739055.853317:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d000 going to sleep for 11 seconds 00000100:00100000:1.0:1787739055.853669:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665577472 00000100:00100000:1.0:1787739055.853674:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665577472 00000100:00100000:1.0:1787739055.853723:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2716 00000100:00100000:1.0:1787739055.853731:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f78b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665577472:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739055.853816:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f78b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665577472:12345-0@lo:1000:mdt00_004.0 Request processed in 86us (522us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739055.853823:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2716 00000100:00100000:0.0:1787739055.854450:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665577472:0@lo:1000:mdt00_004.0 00010000:00010000:0.0:1787739055.854509:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1abf2d lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.854520:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1abf2d lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.854529:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1abf2d lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.854543:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665577600:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739055.854579:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739055.854613:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d840 going to sleep for 11 seconds 00000100:00100000:1.0:1787739055.855242:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665577600 00000100:00100000:1.0:1787739055.855248:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665577600 00000100:00100000:1.0:1787739055.855304:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2717 00000100:00100000:1.0:1787739055.855311:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f711e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665577600:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739055.855333:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739055.855350:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abf34 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abf2d expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.855378:0:8447:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1abc7f lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bc6 expref: 9 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.855387:0:8447:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abf34 lrc: 4/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abf2d expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.855683:0:8447:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1abc7f lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bc6 expref: 9 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.855698:0:8447:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1abc7f lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bc6 expref: 9 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739055.855709:0:8447:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fc3c0 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8447:1874579665577728:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739055.856137:0:8447:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.879675:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739055.879726:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652433664 00000100:00100000:2.0:1787739055.879736:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652433664 00010000:00010000:2.0:1787739055.879777:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1abc7f lrc: 5/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bc6 expref: 10 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.879825:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1418 00000100:00100000:2.0:1787739055.879845:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652433664:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739055.879864:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739055.879871:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1abc7f lrc: 5/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bc6 expref: 10 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.879884:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1abc7f lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bc6 expref: 10 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.879906:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064300) 00010000:00010000:2.0:1787739055.879913:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abf34 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abf2d expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.879923:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abf34 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1abf2d expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.879932:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abf34 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1abf2d expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.879963:0:6508:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abf34 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abf2d expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739055.879977:0:6508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d85680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001_UUID:6508:1874579665577856:0@lo:105: 00000100:00100000:2.0:1787739055.880021:0:6508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.880058:0:6508:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355ed780 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.880090:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665577856 00000100:00100000:2.0:1787739055.880094:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665577856 00000100:00100000:2.0:1787739055.880123:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1083 00000100:00100000:2.0:1787739055.880130:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d852c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6508:x1874579665577856:12345-0@lo:105: 00010000:00010000:2.0:1787739055.880140:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1abf2d lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abf34 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.880150:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1abf2d lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abf34 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.880159:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1abf2d lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abf34 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.880166:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1abf2d lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1abf34 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.880173:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1abf2d lrc: 5/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1abf34 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.880180:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1abf2d lrc: 5/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1abf34 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.880194:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609ac00) 00000100:00100000:2.0:1787739055.880228:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d852c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6508:x1874579665577856:12345-0@lo:105: Request processed in 99us (206us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739055.880235:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1083 00000100:00100000:2.0:1787739055.880285:0:6508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d85680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001_UUID:6508:1874579665577856:0@lo:105: 00010000:00010000:2.0:1787739055.880301:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739055.880343:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652433664:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 509us (678us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739055.880350:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1418 00010000:00010000:1.0:1787739055.886063:0:8447:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1abc7f lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094bc6 expref: 9 pid: 6524 timeout: 742 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739055.886105:0:8447:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fc3c0 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8447:1874579665577728:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739055.886134:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abf34 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abf2d expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.886150:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064300) 00010000:00010000:1.0:1787739055.886154:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739055.887218:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1da00, rc 0) 00000100:00100000:1.0:1787739055.887301:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f711e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:13793:x1874579665577600:12345-0@lo:101:mdt00_004.0 Request processed in 31988us (32719us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.887310:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2717 00000100:00100000:1.0:1787739055.887367:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665577600:0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739055.887387:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1abf2d lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abf34 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739055.887407:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739055.887710:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1abf3b lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.887779:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1abf3b lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.887790:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1abf3b lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739055.887809:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fc3c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665577984:0@lo:101:mdt00_004.0 00000100:00100000:1.0:1787739055.887845:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739055.887874:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665577984 00000100:00100000:1.0:1787739055.887880:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665577984 00000100:00100000:1.0:1787739055.887918:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2718 00000100:00100000:1.0:1787739055.887926:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fd2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:13793:x1874579665577984:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739055.887941:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739055.887957:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abf42 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abf3b expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.887980:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abf42 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abf3b expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.887993:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abf42 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abf3b expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.888000:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xe09b0941 (ffffa07433235100) 00010000:00010000:1.0:1787739055.888003:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739055.888006:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1cc00, rc 0) 00000100:00100000:1.0:1787739055.888029:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fd2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665577984:12345-0@lo:101:mdt00_004.0 Request processed in 104us (184us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739055.888035:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2718 00000100:00100000:1.0:1787739055.888058:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739055.888327:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fc3c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665577984:0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739055.888340:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1abf3b lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abf42 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.888350:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1abf3b lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abf42 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739055.888363:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:1.0:1787739055.888396:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fc3c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665578112:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739055.888455:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739055.888470:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739055.891019:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665578112 00000100:00100000:3.0:1787739055.891032:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665578112 00000100:00100000:3.0:1787739055.891173:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2719 00000100:00100000:3.0:1787739055.891184:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665578112:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739055.891312:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665578112:12345-0@lo:1000:mdt00_004.0 Request processed in 129us (2856us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.891319:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2719 00000100:00100000:2.0:1787739055.894281:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fc3c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665578112:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.894359:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d85680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665578240:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.894552:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.896958:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355edcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739055.897075:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665578240 00000100:00100000:0.0:1787739055.897086:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665578240 00000100:00100000:0.0:1787739055.897141:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2720 00000100:00100000:0.0:1787739055.897150:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665578240:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739055.897282:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665578240:12345-0@lo:1000:mdt00_004.0 Request processed in 133us (2730us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739055.897290:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2720 00000100:00100000:2.0:1787739055.898060:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d85680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665578240:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.898174:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d85680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665578368:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.898217:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.898245:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355edcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739055.898433:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665578368 00000100:00100000:0.0:1787739055.898439:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665578368 00000100:00100000:0.0:1787739055.898474:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2721 00000100:00100000:0.0:1787739055.898481:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d87840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665578368:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739055.898561:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d87840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665578368:12345-0@lo:1000:mdt00_004.0 Request processed in 79us (343us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739055.898568:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2721 00000100:00100000:2.0:1787739055.898732:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d85680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665578368:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.898780:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d85680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665578496:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.898989:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.899016:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355edcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739055.899064:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665578496 00000100:00100000:3.0:1787739055.899081:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665578496 00000100:00100000:3.0:1787739055.899114:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2722 00000100:00100000:3.0:1787739055.899122:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665578496:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739055.899186:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665578496:12345-0@lo:1000:mdt00_004.0 Request processed in 64us (196us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.899193:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2722 00000100:00100000:2.0:1787739055.899368:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d85680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665578496:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.899454:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d85680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665578624:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.899479:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.899492:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355edcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739055.899616:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665578624 00000100:00100000:3.0:1787739055.899620:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665578624 00000100:00100000:3.0:1787739055.899643:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2723 00000100:00100000:3.0:1787739055.899649:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d843c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665578624:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739055.899709:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d843c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665578624:12345-0@lo:1000:mdt00_004.0 Request processed in 60us (229us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.899728:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2723 00000100:00100000:2.0:1787739055.900050:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d85680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665578624:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.900103:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d85680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665578752:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.900127:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739055.900150:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355edcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739055.900246:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665578752 00000100:00100000:3.0:1787739055.900251:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665578752 00000100:00100000:3.0:1787739055.900291:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2724 00000100:00100000:3.0:1787739055.900298:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d870c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665578752:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739055.900355:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d870c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665578752:12345-0@lo:1000:mdt00_004.0 Request processed in 58us (227us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.900362:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2724 00000100:00100000:2.0:1787739055.900636:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d85680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665578752:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.900669:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d85680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665578880:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.900694:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.900789:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665578880 00000100:00100000:3.0:1787739055.900802:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665578880 00000100:00100000:3.0:1787739055.900834:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2725 00000100:00100000:3.0:1787739055.901040:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305619680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665578880:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739055.901107:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355edcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739055.901645:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305619680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665578880:12345-0@lo:1000:mdt00_004.0 Request processed in 606us (948us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739055.901657:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2725 00000100:00100000:0.0:1787739055.901777:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d85680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665578880:0@lo:1000:mdt00_004.0 00000040:00080000:0.0:1787739055.904206:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].352, 2168 off540904 00000040:00080000:0.0:1787739055.904386:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].351, 2368 off584896 00000004:00080000:0.0:1787739055.904393:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07410b4d000 with version 671 00000004:00080000:1.0:1787739055.904897:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 671 rpc_version 671 00000020:00100000:1.0:1787739055.904941:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739055.904951:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073061fed00 x1874579665579136/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739055.904975:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fed00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665579136:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739055.905042:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739055.905173:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739055.905655:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665579136 00000100:00100000:0.0:1787739055.905662:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665579136 00000100:00100000:0.0:1787739055.905935:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2726 00000100:00100000:0.0:1787739055.905946:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fcf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665579136:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739055.908232:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fcf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:7740:x1874579665579136:12345-0@lo:1000:osp_up1-0.0 Request processed in 2284us (3184us total) trans 4294968322 rc 0/0 00000100:00100000:0.0:1787739055.908243:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2726 00000100:00100000:1.0:1787739055.908445:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fed00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665579136:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787739055.908627:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abf49 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1501:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.908642:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abf49 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1501:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.908653:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abf49 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1501:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.908672:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abf50 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xbb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.908679:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abf50 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.908685:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abf50 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.908920:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1abf57 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xbb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.908929:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1abf57 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.908939:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1abf57 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.908959:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f7112c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665579264:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739055.908998:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739055.909083:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062346c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739055.912803:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665579264 00000100:00100000:3.0:1787739055.912818:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665579264 00000100:00100000:2.0:1787739055.913282:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2727 00000100:00100000:2.0:1787739055.913292:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:13793:x1874579665579264:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739055.913316:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739055.913340:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1abf5e lrc: 2/0,0 mode: --/PW res: [0x240000401:0xbb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abf57 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.913370:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1abf5e lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abf57 expref: 12 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.913384:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1abf5e lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abf57 expref: 12 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739055.913392:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xbb:0x0].0x0 (ffffa074419d5e00) 00010000:00010000:2.0:1787739055.913395:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739055.913399:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633ce00, rc 0) 00000100:00100000:2.0:1787739055.913502:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:13793:x1874579665579264:12345-0@lo:101:mdt00_004.0 Request processed in 211us (4503us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739055.913508:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2727 00000100:00100000:0.0:1787739055.913589:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f7112c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665579264:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739055.913608:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1abf57 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abf5e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.913619:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1abf57 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abf5e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739055.913646:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000acb0f277 reply state 0000000076f33916 transno 4294983331 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abf50 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.913658:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abf50 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.913666:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xbb:0x0].0x0 (ffffa073068a9a00) 00010000:00010000:0.0:1787739055.913669:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739055.913672:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739055.913678:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1abf57 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abf5e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.913686:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1abf57 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abf5e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739055.913698:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xbb:0x0] 00000004:00010000:0.0:1787739055.913704:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000acb0f277 reply state 0000000076f33916 transno 4294983331 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abf49 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1501:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.913722:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abf49 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1501:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.913729:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1501:0x0].0x0 (ffffa073068a9e00) 00010000:00010000:0.0:1787739055.913731:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739055.913733:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787739055.913737:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1501:0x0]: returning size 4096 00000004:00080000:0.0:1787739055.913746:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739055.913755:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1abf2d lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abf34 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.913762:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1abf2d lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abf34 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.913769:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1abf2d lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abf34 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.913804:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739055.913808:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739055.913812:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1abf3b lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abf42 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739055.913820:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1abf3b lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abf42 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739055.913894:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d86940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17471:x1874579652433536:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 61654us (62069us total) trans 4294983331 rc 0/0 00000100:00100000:0.0:1787739055.913901:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23253 00010000:00010000:1.0:1787739055.915550:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739055.915562:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1abf2d lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1abf34 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.915661:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ac00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739055.915667:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1abf2d lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abf34 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739055.915676:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1abf2d lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abf34 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.915716:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1abf2d lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abf34 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.915723:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739055.915747:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1abf2d lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abf34 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.915755:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1abf2d lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abf34 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.915781:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739055.915783:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739055.915785:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739055.915801:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fc780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665579392:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739055.915860:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.920531:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665579392 00000100:00100000:3.0:1787739055.920545:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665579392 00000100:00100000:3.0:1787739055.920674:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1219 00000100:00100000:3.0:1787739055.920684:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074301a0050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+12:3652:x1874579665579392:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739055.920699:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739055.920722:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abf34 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abf2d expref: 12 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.920739:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064300) 00010000:00010000:3.0:1787739055.920751:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739055.920835:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074301a0050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+11:3652:x1874579665579392:12345-0@lo:103:ldlm_bl.0 Request processed in 153us (4975us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.920843:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1219 00000100:00100000:3.0:1787739055.920917:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fc780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665579392:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739055.933227:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739055.933608:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652433920 00000100:00100000:3.0:1787739055.933618:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652433920 00000100:00100000:3.0:1787739055.933677:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23254 00000100:00100000:3.0:1787739055.933688:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17472:x1874579652433920:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:3.0:1787739055.933836:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17472:x1874579652433920:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 151us (617us total) trans 0 rc -17/-17 00000100:00100000:3.0:1787739055.933842:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23254 00000100:00100000:0.0:1787739055.967710:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739055.970632:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652434048 00000100:00100000:3.0:1787739055.970645:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652434048 00000100:00100000:3.0:1787739055.970705:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23255 00000100:00100000:3.0:1787739055.970715:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7112c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17473:x1874579652434048:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739055.970735:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739055.970757:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1abf65 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094c52 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.970827:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1abf6c lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.970841:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1abf6c lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.970852:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1abf6c lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.970862:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1abf73 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.970872:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abf42 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abf3b expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.970880:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1abf73 lrc: 4/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.970917:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abf42 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1abf3b expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739055.970946:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665579520:0@lo:104: 00000100:00100000:3.0:1787739055.971015:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.971084:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739055.971110:0:6507:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:3.0:1787739055.971120:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665579520 00000100:00100000:3.0:1787739055.971123:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665579520 00000100:00100000:3.0:1787739055.971153:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1084 00000100:00100000:3.0:1787739055.971160:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306b98450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665579520:12345-0@lo:104: 00010000:00010000:3.0:1787739055.971172:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1abf3b lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1abf42 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.971222:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739055.971228:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306b98450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665579520:12345-0@lo:104: Request processed in 69us (213us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739055.971235:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1084 00010000:00010000:3.0:1787739055.971279:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739055.971285:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1abf3b lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1abf42 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.971293:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739055.971298:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1abf3b lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1abf42 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739055.971306:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1abf3b lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1abf42 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.971330:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1abf3b lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abf42 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.971336:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739055.971359:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1abf3b lrc: 1/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abf42 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.971365:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1abf3b lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abf42 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739055.971381:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739055.971383:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739055.971385:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739055.971392:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787739055.971401:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665579648:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739055.971467:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739055.971508:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665579520:0@lo:104: 00010000:00010000:3.0:1787739055.971524:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1abf73 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739055.971544:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665579648 00000100:00100000:3.0:1787739055.971547:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665579648 00010000:00010000:3.0:1787739055.971563:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abf42 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1abf3b expref: 11 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739055.971586:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1419 00000100:00100000:3.0:1787739055.971593:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be6580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+11:3652:x1874579665579648:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739055.971604:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739055.999765:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1abed2 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abed9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739055.999786:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1abed2 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abed9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.999830:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1abed2 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abed9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739055.999848:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739055.999880:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c400/0x7ee4453e4a1abed2 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abed9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787739055.999900:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968322 is committed 00000001:00080000:1.0:1787739055.999916:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968321 is committed 00000100:00100000:3.0:1787739056.000254:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061ff480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665579776:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:3.0:1787739056.000321:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739056.000390:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665579776 00000100:00100000:3.0:1787739056.000396:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665579776 00000100:00100000:3.0:1787739056.000471:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1220 00000100:00100000:3.0:1787739056.000479:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+7:3650:x1874579665579776:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:3.0:1787739056.000491:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739056.000509:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1abed9 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x367edb60 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abed2 expref: 7 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.000523:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x367edb60 (ffffa073068a9b00) 00010000:00010000:3.0:1787739056.000531:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739056.000566:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+6:3650:x1874579665579776:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 87us (245us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.000573:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1220 00000100:00100000:3.0:1787739056.000613:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061ff480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665579776:0@lo:103:jbd2/dm-1-8.0 00000020:00080000:1.0:1787739056.001132:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 343 batchid updated 00000020:00080000:1.0:1787739056.001147:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa073068bae40 refcount 2 committed 0 result 0 batchid 344 00000020:00080000:1.0:1787739056.001151:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730415f100 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa073068ba480 00000020:00080000:1.0:1787739056.001154:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].338 00000020:00080000:1.0:1787739056.001156:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730415f900 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304011200 00000020:00080000:1.0:1787739056.001158:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].337 00000020:00080000:1.0:1787739056.001180:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 342 batchid updated 00010000:00010000:1.0:1787739056.001204:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1abe62 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x163:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.001212:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1abe62 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x163:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.001218:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1abe62 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x163:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.001224:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739056.001228:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1abe62 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x163:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.001236:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1abe62 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x163:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.001246:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x163:0x0].0x0 (ffffa074419d5800) 00010000:00010000:1.0:1787739056.001249:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739056.001252:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1abe62 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x163:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.001258:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1abe62 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x163:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.001274:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abe69 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.001280:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abe69 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.001286:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abe69 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.001296:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739056.001299:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abe69 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.001304:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abe69 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.001311:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb8:0x0].0x0 (ffffa074419d5500) 00010000:00010000:1.0:1787739056.001313:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739056.001316:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abe69 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.001322:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abe69 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xb8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1787739056.002444:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 344 committed batchid 343 00000004:00080000:3.0:1787739056.002555:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07305fcc540 with version 645 00000020:00080000:3.0:1787739056.002566:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 342 cancel update log [0x200000401:0x3:0x0].336: rc = 0 00000020:00080000:3.0:1787739056.002605:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 342 cancel update log [0x240000400:0x3:0x0].335: rc = 0 00000004:00080000:3.0:1787739056.002628:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07305fccf00 with version 646 00000020:00080000:3.0:1787739056.002631:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 343 cancel update log [0x200000401:0x3:0x0].337: rc = 0 00000020:00080000:3.0:1787739056.002648:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 343 cancel update log [0x240000400:0x3:0x0].336: rc = 0 00000020:00080000:3.0:1787739056.002654:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 343 00010000:00010000:3.0:1787739056.002705:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abf42 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1abf3b expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.002732:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xe09b0941 (ffffa07433235100) 00010000:00010000:3.0:1787739056.002737:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1abf73 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.002746:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1abf73 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.002753:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1abf73 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.002779:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739056.002858:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be6580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+6:3652:x1874579665579648:12345-0@lo:103:ldlm_bl.0 Request processed in 31264us (31389us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.002867:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1419 00000100:00100000:3.0:1787739056.002947:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665579648:0@lo:103:ldlm_bl.0 00000004:00080000:1.0:1787739056.003179:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 645 rpc_version 645 00000020:00100000:1.0:1787739056.003210:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739056.003218:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073061fc780 x1874579665580032/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739056.003237:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fc780 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665580032:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739056.003277:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.003306:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739056.006358:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665580032 00000100:00100000:3.0:1787739056.006373:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665580032 00000100:00100000:3.0:1787739056.006462:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2728 00000100:00100000:3.0:1787739056.006472:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fe940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:7467:x1874579665580032:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739056.006676:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fe940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665580032:12345-0@lo:1000:osp_up0-1.0 Request processed in 205us (3397us total) trans 4294983332 rc 0/0 00000100:00100000:3.0:1787739056.006683:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2728 00010000:00010000:0.0:1787739056.008160:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1abf73 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.008178:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1abf73 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739056.008300:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f712580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665580160:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739056.008359:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739056.008391:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739056.008430:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665580160 00000100:00100000:0.0:1787739056.008437:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665580160 00000100:00100000:0.0:1787739056.008478:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2729 00000100:00100000:0.0:1787739056.008486:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665580160:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739056.008560:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665580160:12345-0@lo:1000:mdt00_003.0 Request processed in 74us (200us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.008568:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2729 00000100:00100000:0.0:1787739056.008607:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f712580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665580160:0@lo:1000:mdt00_003.0 00010000:00010000:0.0:1787739056.008657:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1abf7a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1501:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.008673:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1abf7a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1501:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.008683:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1abf7a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1501:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739056.008702:0:7944:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1abf7a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1501:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.008715:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1abf6c lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.008720:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1abf6c lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.008726:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1abf6c lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.008731:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1abf6c lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.008737:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739056.008740:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1abf6c lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.008747:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1abf6c lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.008757:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305d56a00) 00010000:00010000:0.0:1787739056.008759:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739056.008764:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1abf6c lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.008769:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1abf6c lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.008779:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1abf73 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.008786:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1abf73 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.008792:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1abf73 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.008797:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739056.008799:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1abf73 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.008805:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1abf73 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.008814:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xe09b0941 (ffffa07433235100) 00010000:00010000:0.0:1787739056.008816:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739056.008819:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1abf73 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.008825:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1abf73 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.008865:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1abf65 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c52 expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.008879:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1abf7a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1501:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c52 expref: 7 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.008885:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1501:0x0].0x0 (ffffa07307064c00) 00010000:00010000:0.0:1787739056.008887:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739056.008889:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a200, rc 0) 00000100:00100000:0.0:1787739056.008942:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7112c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17473:x1874579652434048:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 38228us (41240us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.008948:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23255 00000100:00100000:1.0:1787739056.010672:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fc780 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665580032:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787739056.010688:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 646 rpc_version 646 00000020:00100000:1.0:1787739056.010711:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739056.010719:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073061fc3c0 x1874579665580416/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739056.010738:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fc3c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665580416:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739056.010795:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.010832:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739056.012162:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665580416 00000100:00100000:2.0:1787739056.012173:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665580416 00000100:00100000:2.0:1787739056.012234:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2730 00000100:00100000:2.0:1787739056.012244:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fe580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665580416:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739056.012858:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fe580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665580416:12345-0@lo:1000:osp_up0-1.0 Request processed in 614us (2060us total) trans 4294983333 rc 0/0 00000100:00100000:2.0:1787739056.012871:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2730 00000100:00100000:1.0:1787739056.019629:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fc3c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665580416:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739056.035727:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739056.036196:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652434176 00000100:00100000:0.0:1787739056.036209:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652434176 00000100:00100000:0.0:1787739056.036281:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23256 00000100:00100000:0.0:1787739056.036290:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fd680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17473:x1874579652434176:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739056.036317:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739056.036340:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1abf81 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1501:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094c59 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.036661:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abf88 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1501:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.036677:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abf88 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1501:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.036691:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abf88 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1501:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739056.036783:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1501:0x0]: returning size 4096 00000004:00200000:0.0:1787739056.036789:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739056.036824:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1abf81 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1501:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c59 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.037005:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abf88 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1501:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c59 expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.037015:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1501:0x0].0x0 (ffffa073068a9e00) 00010000:00010000:0.0:1787739056.037018:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739056.037021:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a800, rc 0) 00000100:00100000:0.0:1787739056.037501:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fd680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17473:x1874579652434176:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 1209us (1777us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.037510:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23256 00000100:00100000:0.0:1787739056.042455:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739056.042496:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652434304 00000100:00100000:0.0:1787739056.042505:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652434304 00000100:00100000:0.0:1787739056.042557:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23257 00000100:00100000:0.0:1787739056.042586:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f712580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17473:x1874579652434304:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739056.042609:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739056.042630:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1abf8f lrc: 2/0,0 mode: --/CR res: [0x200000400:0xbb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094c60 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.042688:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1abf96 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xbb:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.042701:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1abf96 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.042714:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1abf96 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739056.042762:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xbb:0x0]: returning size 4096 00000004:00200000:0.0:1787739056.042767:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739056.042803:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1abf8f lrc: 0/0,0 mode: --/CR res: [0x200000400:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c60 expref: 19 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.042819:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1abf96 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c60 expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.042827:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xbb:0x0].0x0 (ffffa073068a9a00) 00010000:00010000:0.0:1787739056.042829:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739056.042843:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b000, rc 0) 00000100:00100000:0.0:1787739056.042895:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f712580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17473:x1874579652434304:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 324us (450us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.042903:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23257 00000100:00100000:0.0:1787739056.073581:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739056.073660:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652434432 00000100:00100000:2.0:1787739056.073670:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652434432 00000100:00100000:2.0:1787739056.073731:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23258 00000100:00100000:2.0:1787739056.073742:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7130c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17473:x1874579652434432:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739056.073761:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739056.073783:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1abf9d lrc: 2/0,0 mode: --/CR res: [0x240000401:0xbb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094c67 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.073838:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abfa4 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xbb:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.073853:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1abf5e lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abf57 expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.073862:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abfa4 lrc: 4/1,0 mode: --/PR res: [0x240000401:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.073894:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1abf5e lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1abf57 expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739056.073910:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561a580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665580544:0@lo:104: 00000100:00100000:2.0:1787739056.073955:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739056.074000:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355ed780 going to sleep for 11 seconds 00000100:00100000:2.0:1787739056.074017:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665580544 00000100:00100000:2.0:1787739056.074021:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665580544 00000100:00100000:2.0:1787739056.074063:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1085 00000100:00100000:2.0:1787739056.074079:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561b840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665580544:12345-0@lo:104: 00010000:00010000:2.0:1787739056.074091:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1abf57 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1abf5e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.074135:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739056.074141:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561b840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665580544:12345-0@lo:104: Request processed in 64us (187us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.074147:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1085 00010000:00010000:2.0:1787739056.074166:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739056.074171:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1abf57 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1abf5e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.074178:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739056.074183:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1abf57 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xbb:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1abf5e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739056.074191:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1abf57 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xbb:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1abf5e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.074214:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1abf57 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xbb:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abf5e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.074221:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739056.074238:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1abf57 lrc: 1/0,0 mode: --/PW res: [0x240000401:0xbb:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abf5e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.074245:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b200/0x7ee4453e4a1abf57 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xbb:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1abf5e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.074260:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739056.074262:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739056.074264:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739056.074286:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561a580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665580544:0@lo:104: 00010000:00010000:2.0:1787739056.074302:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abfa4 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739056.074357:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665580672:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739056.074386:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739056.076588:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665580672 00000100:00100000:2.0:1787739056.076600:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665580672 00010000:00010000:2.0:1787739056.076635:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1abf5e lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1abf57 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739056.076676:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1420 00000100:00100000:2.0:1787739056.076684:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f712d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+6:3651:x1874579665580672:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739056.076697:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00080000:0.0:1787739056.087235:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 344 batchid updated 00010000:00010000:1.0:1787739056.087370:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1abf5e lrc: 0/0,0 mode: --/PW res: [0x240000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1abf57 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.087390:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xbb:0x0].0x0 (ffffa074419d5e00) 00010000:00010000:1.0:1787739056.087405:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abfa4 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.087436:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abfa4 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.087445:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abfa4 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.087480:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739056.087520:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abfa4 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.087529:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abfa4 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.087559:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f712d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665580672:12345-0@lo:103:ldlm_bl.0 Request processed in 10875us (13171us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.087569:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1420 00000004:00200000:0.0:1787739056.087595:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xbb:0x0]: returning size 4096 00000004:00200000:0.0:1787739056.087600:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000100:00100000:1.0:1787739056.087622:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665580672:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739056.087641:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1abf9d lrc: 0/0,0 mode: --/CR res: [0x240000401:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c67 expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.087657:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abfa4 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c67 expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.087665:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xbb:0x0].0x0 (ffffa074419d5e00) 00010000:00010000:0.0:1787739056.087668:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739056.087670:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633da00, rc 0) 00000020:00080000:3.0:1787739056.087684:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 344 00000100:00100000:0.0:1787739056.087732:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7130c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17473:x1874579652434432:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 13992us (14162us total) trans 0 rc 0/0 00000004:00080000:3.0:1787739056.087737:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07305fcce40 with version 647 00000100:00100000:0.0:1787739056.087739:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23258 00000020:00080000:3.0:1787739056.087745:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 344 cancel update log [0x200000401:0x3:0x0].338: rc = 0 00000020:00080000:3.0:1787739056.087810:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 344 cancel update log [0x240000400:0x3:0x0].337: rc = 0 00000020:00080000:3.0:1787739056.087817:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 344 00000004:00080000:1.0:1787739056.087952:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 647 rpc_version 647 00000020:00100000:1.0:1787739056.087980:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739056.087987:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073061ff480 x1874579665580928/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739056.088007:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061ff480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665580928:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739056.088059:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.088097:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054540 going to sleep for 11 seconds 00000100:00100000:0.0:1787739056.088772:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665580928 00000100:00100000:0.0:1787739056.088780:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665580928 00000100:00100000:0.0:1787739056.088842:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2731 00000100:00100000:0.0:1787739056.088851:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f79a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665580928:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739056.089019:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f79a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665580928:12345-0@lo:1000:osp_up0-1.0 Request processed in 168us (958us total) trans 4294983334 rc 0/0 00000100:00100000:0.0:1787739056.089025:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2731 00000100:00100000:1.0:1787739056.089153:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061ff480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665580928:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739056.111139:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739056.111318:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652434560 00000100:00100000:3.0:1787739056.111328:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652434560 00000100:00100000:3.0:1787739056.111390:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1221 00000100:00100000:3.0:1787739056.111409:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7a580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2406:x1874579652434560:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:3.0:1787739056.111472:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739056.111499:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1abf7a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1501:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c52 expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.111514:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1501:0x0].0x0 (ffffa07307064c00) 00010000:00010000:3.0:1787739056.111524:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739056.111918:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7a580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2406:x1874579652434560:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 517us (786us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.111927:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1221 00000100:00100000:1.0:1787739056.129720:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739056.131248:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652434816 00000100:00100000:0.0:1787739056.131261:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652434816 00000100:00100000:0.0:1787739056.131374:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23259 00000100:00100000:0.0:1787739056.131385:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f79680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17473:x1874579652434816:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:0.0:1787739056.132053:0:13793:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739056.132091:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1abf88 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1501:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c59 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.132115:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1501:0x0].0x0 (ffffa073068a9e00) 00010000:00010000:0.0:1787739056.132120:0:13793:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739056.132167:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1abfab lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.132176:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1abfab lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.132186:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1abfab lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739056.132208:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665581056:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739056.132269:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739056.132346:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739056.145149:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665581056 00000100:00100000:1.0:1787739056.145164:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665581056 00000100:00100000:1.0:1787739056.145230:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2732 00000100:00100000:1.0:1787739056.145239:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+5:13793:x1874579665581056:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739056.145258:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739056.145283:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1abfb2 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abfab expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.145317:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1abfb2 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abfab expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.145332:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1abfb2 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abfab expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.145341:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07433235100) 00010000:00010000:1.0:1787739056.145344:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739056.145348:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d600, rc 0) 00000100:00100000:1.0:1787739056.145445:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:13793:x1874579665581056:12345-0@lo:101:mdt00_004.0 Request processed in 206us (13176us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.145452:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2732 00000100:00100000:1.0:1787739056.145585:0:13793:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968322 gen 2 00000004:00080000:1.0:1787739056.145592:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968317 committed transno 4294968322 00000020:00080000:1.0:1787739056.145599:0:13793:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa073068b4180 refcount 2 committed 0 result 0 batchid 353 00000020:00080000:1.0:1787739056.145603:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88c00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa073068b4c00 00000020:00080000:1.0:1787739056.145607:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].350 00000020:00080000:1.0:1787739056.145609:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88900 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306063600 00000020:00080000:1.0:1787739056.145612:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].351 00000004:00080000:1.0:1787739056.145636:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968319 committed transno 4294968322 00000004:00080000:1.0:1787739056.145645:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968320 committed transno 4294968322 00000004:00080000:1.0:1787739056.145654:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968322 committed transno 4294968322 00000100:00100000:1.0:1787739056.145671:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665581056:0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739056.145688:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1abfab lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abfb2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.145699:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1abfab lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abfb2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739056.145717:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739056.145748:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1abfb9 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.145755:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1abfb9 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.145764:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1abfb9 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.145783:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665581184:0@lo:101:mdt00_004.0 00000100:00100000:1.0:1787739056.145816:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.145847:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054540 going to sleep for 11 seconds 00000020:00080000:1.0:1787739056.145860:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 353 committed batchid 352 00000020:00080000:1.0:1787739056.145915:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 352 00000100:00100000:1.0:1787739056.145928:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665581184 00000100:00100000:1.0:1787739056.145932:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665581184 00000100:00100000:1.0:1787739056.145957:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2733 00000100:00100000:1.0:1787739056.145964:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:13793:x1874579665581184:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739056.145972:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739056.145984:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1abfc0 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abfb9 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.145998:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1abfc0 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abfb9 expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.146008:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1abfc0 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abfb9 expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.146014:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xe09b0941 (ffffa07433235700) 00010000:00010000:1.0:1787739056.146017:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739056.146019:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1de00, rc 0) 00000100:00100000:1.0:1787739056.146048:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665581184:12345-0@lo:101:mdt00_004.0 Request processed in 85us (232us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.146054:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2733 00000100:00100000:1.0:1787739056.146094:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665581184:0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739056.146105:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1abfb9 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abfc0 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.146113:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1abfb9 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abfc0 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739056.146124:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:1.0:1787739056.146152:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061ff840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665581312:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739056.146178:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.146191:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073070549c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739056.146204:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665581312 00000100:00100000:1.0:1787739056.146207:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665581312 00000100:00100000:1.0:1787739056.146226:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2734 00000100:00100000:1.0:1787739056.146231:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7bc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665581312:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739056.146292:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7bc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665581312:12345-0@lo:1000:mdt00_004.0 Request processed in 60us (112us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.146298:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2734 00000100:00100000:1.0:1787739056.146768:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061ff840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665581312:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739056.146809:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061ff840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665581440:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739056.146835:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.146850:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073070549c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739056.146863:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665581440 00000100:00100000:1.0:1787739056.146866:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665581440 00000100:00100000:1.0:1787739056.146889:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2735 00000100:00100000:1.0:1787739056.146896:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665581440:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739056.146975:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665581440:12345-0@lo:1000:mdt00_004.0 Request processed in 80us (140us total) trans 0 rc 1/1 00000100:00100000:1.0:1787739056.146982:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2735 00000100:00100000:1.0:1787739056.147010:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061ff840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665581440:0@lo:1000:mdt00_004.0 00010000:00010000:1.0:1787739056.147049:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1abfc7 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.147057:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1abfc7 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.147080:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1abfc7 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.147092:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665581568:0@lo:101:mdt00_004.0 00000100:00100000:1.0:1787739056.147114:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.147128:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739056.147139:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665581568 00000100:00100000:1.0:1787739056.147142:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665581568 00000100:00100000:1.0:1787739056.147162:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2736 00000100:00100000:1.0:1787739056.147167:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061ff840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665581568:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739056.147175:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739056.147188:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abfce lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abfc7 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.147204:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abfce lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abfc7 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.147214:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abfce lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abfc7 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.147221:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1501:0x0].0x0 (ffffa07433235000) 00010000:00010000:1.0:1787739056.147224:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739056.147226:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1cc00, rc 0) 00000100:00100000:1.0:1787739056.147253:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061ff840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665581568:12345-0@lo:101:mdt00_004.0 Request processed in 86us (138us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.147258:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2736 00000100:00100000:1.0:1787739056.147283:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665581568:0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739056.147293:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1abfc7 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abfce expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.147301:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1abfc7 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abfce expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.147320:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abfd5 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.147329:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abfd5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.147337:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abfd5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.147352:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abfdc lrc: 3/0,1 mode: --/EX res: [0x200000400:0xbb:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.147362:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1abf96 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c60 expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.147370:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abfdc lrc: 4/0,1 mode: --/EX res: [0x200000400:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.147392:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1abf96 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c60 expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.149237:0:13793:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1abf96 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c60 expref: 17 pid: 6523 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.149264:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665581696:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739056.149385:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739056.172182:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665581696:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739056.172211:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abfdc lrc: 3/0,1 mode: --/EX res: [0x200000400:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739056.172384:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739056.172588:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652434944 00000100:00100000:3.0:1787739056.172595:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652434944 00010000:00010000:3.0:1787739056.172629:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1abf96 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c60 expref: 18 pid: 6523 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739056.172668:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1421 00000100:00100000:3.0:1787739056.172676:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561a580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2409:x1874579652434944:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739056.172690:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739056.172696:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1abf96 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c60 expref: 18 pid: 6523 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.172707:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1abf96 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c60 expref: 18 pid: 6523 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.172734:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1abf96 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c60 expref: 18 pid: 6523 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.172750:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xbb:0x0].0x0 (ffffa073068a9a00) 00010000:00010000:3.0:1787739056.172756:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abfdc lrc: 3/0,1 mode: --/EX res: [0x200000400:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.172765:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abfdc lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.172772:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abfdc lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.172798:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739056.172839:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561a580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652434944:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 164us (457us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.172846:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1421 00010000:00010000:2.0:1787739056.172916:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abfdc lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.172923:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abfdc lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.172966:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1abfe3 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.172974:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1abfe3 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.172983:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1abfe3 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739056.173000:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073056192c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665581824:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1787739056.173038:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739056.173078:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074062f3b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739056.173183:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665581824 00000100:00100000:1.0:1787739056.173188:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665581824 00000100:00100000:1.0:1787739056.173223:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2737 00000100:00100000:1.0:1787739056.173231:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305619e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665581824:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739056.173244:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739056.173257:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1abfea lrc: 2/0,0 mode: --/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1abfe3 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.173276:0:17834:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abfa4 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c67 expref: 6 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.173285:0:17834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1abfea lrc: 4/0,0 mode: --/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1abfe3 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.173304:0:17834:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abfa4 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c67 expref: 6 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.173315:0:17834:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abfa4 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c67 expref: 6 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.173330:0:17834:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:17834:1874579665581952:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739056.173380:0:17834:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739056.179474:0:17834:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:17834:1874579665581952:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739056.179511:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1abfea lrc: 3/0,0 mode: --/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abfe3 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.179525:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xbb:0x0].0x0 (ffffa074419d5e00) 00010000:00010000:3.0:1787739056.179528:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739056.179532:0:17834:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1abfea lrc: 3/0,0 mode: --/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abfe3 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.179542:0:17834:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1abfea lrc: 3/0,0 mode: --/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abfe3 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.179550:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c000, rc 0) 00000100:00100000:3.0:1787739056.179619:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305619e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:13793:x1874579665581824:12345-0@lo:101:mdt00_004.0 Request processed in 6388us (6579us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.179629:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2737 00000100:00100000:2.0:1787739056.179729:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073056192c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665581824:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739056.179747:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1abfe3 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abfea expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.179758:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1abfe3 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abfea expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739056.183134:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739056.185159:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652435072 00000100:00100000:1.0:1787739056.185172:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652435072 00010000:00010000:1.0:1787739056.185209:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abfa4 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c67 expref: 7 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.185290:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1422 00000100:00100000:1.0:1787739056.185300:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652435072:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739056.185314:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739056.185321:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abfa4 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c67 expref: 7 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.185334:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abfa4 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c67 expref: 7 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.185356:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1abfa4 lrc: 0/0,0 mode: --/PR res: [0x240000401:0xbb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c67 expref: 7 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.185370:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xbb:0x0].0x0 (ffffa074419d5e00) 00010000:00010000:1.0:1787739056.185377:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1abfea lrc: 2/0,0 mode: --/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abfe3 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.185387:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1abfea lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1abfe3 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.185516:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1abfea lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1abfe3 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.185553:0:10340:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1abfea lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1abfe3 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.185567:0:10340:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665582080:0@lo:105: 00000100:00100000:1.0:1787739056.185624:0:10340:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.185663:0:10340:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073070549c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739056.185734:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665582080 00000100:00100000:3.0:1787739056.185739:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665582080 00000100:00100000:3.0:1787739056.185804:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1086 00000100:00100000:3.0:1787739056.185812:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c416d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665582080:12345-0@lo:105: 00010000:00010000:3.0:1787739056.185822:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1abfe3 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abfea expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.185831:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1abfe3 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abfea expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.185848:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1abfe3 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abfea expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.185855:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1abfe3 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1abfea expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.185861:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1abfe3 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1abfea expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.185868:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1abfe3 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1abfea expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.185897:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633c400) 00000100:00100000:3.0:1787739056.185961:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c416d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665582080:12345-0@lo:105: Request processed in 151us (336us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.185969:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1086 00010000:00010000:1.0:1787739056.186125:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1abfe3 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abfea expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.186135:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1abfe3 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abfea expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.187104:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c416580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665582208:0@lo:403:mdt00_004.0 00000100:00100000:1.0:1787739056.187153:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.187177:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739056.187199:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665582208 00000100:00100000:1.0:1787739056.187204:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665582208 00000100:00100000:1.0:1787739056.187236:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2738 00000100:00100000:1.0:1787739056.187242:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c416940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:13793:x1874579665582208:12345-0@lo:403:mdt00_004.0 00000100:00100000:0.0:1787739056.188885:0:10340:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665582080:0@lo:105: 00010000:00010000:0.0:1787739056.188908:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739056.190146:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2408:x1874579652435072:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 4840us (7014us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.190160:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1422 00000100:00100000:1.0:1787739056.193865:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c416940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:13793:x1874579665582208:12345-0@lo:403:mdt00_004.0 Request processed in 6614us (6702us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.193879:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2738 00000100:00100000:2.0:1787739056.194648:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c416580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665582208:0@lo:403:mdt00_004.0 00000040:00080000:2.0:1787739056.197755:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].353, 664 off541568 00000040:00080000:2.0:1787739056.197934:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].352, 744 off585640 00000004:00080000:2.0:1787739056.197952:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa074062f3b40 with version 672 00000004:00080000:1.0:1787739056.198591:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 672 rpc_version 672 00000020:00100000:1.0:1787739056.198628:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739056.198638:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743c415e00 x1874579665582464/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2064/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739056.198660:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c415e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665582464:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739056.198736:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.198772:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739056.198951:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665582464 00000100:00100000:2.0:1787739056.198958:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665582464 00000100:00100000:2.0:1787739056.199010:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2739 00000100:00100000:2.0:1787739056.199020:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c414b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665582464:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739056.199399:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c414b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665582464:12345-0@lo:1000:osp_up1-0.0 Request processed in 379us (661us total) trans 4294968323 rc 0/0 00000100:00100000:2.0:1787739056.199723:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2739 00000100:00100000:1.0:1787739056.199990:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c415e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665582464:0@lo:1000:osp_up1-0.0 00000004:00200000:2.0:1787739056.200131:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1501:0x0]: returning size 4096 00010000:00010000:2.0:1787739056.200155:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abfdc lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.200164:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abfdc lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.200171:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abfdc lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.200178:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739056.200182:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abfdc lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.200189:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abfdc lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.200203:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xbb:0x0].0x0 (ffffa073068a9a00) 00010000:00010000:2.0:1787739056.200206:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739056.200211:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abfdc lrc: 2/0,0 mode: --/EX res: [0x200000400:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.200217:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1abfdc lrc: 0/0,0 mode: --/EX res: [0x200000400:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.200230:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1abfe3 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abfea expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.200237:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1abfe3 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abfea expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.200244:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1abfe3 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abfea expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.200357:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739056.200367:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xbb:0x0] 00010000:00010000:2.0:1787739056.200377:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abfd5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.200383:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abfd5 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.200390:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abfd5 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.200396:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739056.200505:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739056.200513:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1abfe3 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1abfea expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.200520:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739056.200525:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1abfe3 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abfea expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739056.200538:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1abfe3 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abfea expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.200571:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1abfe3 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abfea expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.200578:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739056.200607:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1abfe3 lrc: 1/0,0 mode: --/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abfea expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.200614:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1abfe3 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abfea expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.200633:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739056.200635:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739056.200637:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739056.200692:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665582592:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739056.200759:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739056.201015:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665582592 00000100:00100000:0.0:1787739056.201021:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665582592 00000100:00100000:0.0:1787739056.201091:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1222 00000100:00100000:0.0:1787739056.201099:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074301a1c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+10:3652:x1874579665582592:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739056.201111:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739056.201126:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1abfea lrc: 0/0,0 mode: --/EX res: [0x240000401:0xbb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abfe3 expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.201137:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xbb:0x0].0x0 (ffffa074419d5e00) 00010000:00010000:0.0:1787739056.201143:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739056.201184:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074301a1c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+9:3652:x1874579665582592:12345-0@lo:103:ldlm_bl.0 Request processed in 86us (425us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.201191:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1222 00000100:00100000:1.0:1787739056.201284:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665582592:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739056.201747:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abfd5 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.201761:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abfd5 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.201771:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1501:0x0].0x0 (ffffa073068a9e00) 00010000:00010000:2.0:1787739056.201774:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739056.201778:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abfd5 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.201784:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1abfd5 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.201796:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1abfc7 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abfce expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.201803:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1abfc7 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abfce expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.201810:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1abfc7 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abfce expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.201824:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1787739056.201921:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739056.201926:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1abfc7 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1abfce expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.201934:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ce00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739056.201947:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1abfc7 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abfce expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739056.201955:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1abfc7 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abfce expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.201979:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1abfc7 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abfce expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.201985:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739056.202010:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1abfc7 lrc: 1/0,0 mode: --/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abfce expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.202018:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1abfc7 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abfce expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.202050:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739056.202052:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739056.202054:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739056.202100:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787739056.202112:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c4152c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665582720:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739056.202147:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.202447:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665582720 00000100:00100000:1.0:1787739056.202453:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665582720 00000100:00100000:1.0:1787739056.202490:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1223 00000100:00100000:1.0:1787739056.202497:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+9:3652:x1874579665582720:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739056.202507:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739056.202521:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abfce lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1501:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abfc7 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.202531:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1501:0x0].0x0 (ffffa07433235000) 00010000:00010000:1.0:1787739056.202538:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:2.0:1787739056.202634:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:2.0:1787739056.202641:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1abfab lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1abfb2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.202649:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1abfab lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abfb2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.202657:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1abfab lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1abfb2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.202671:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739056.202675:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:2.0:1787739056.202680:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1abfb9 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abfc0 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.202686:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1abfb9 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1abfc0 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739056.202748:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f79680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17473:x1874579652434816:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 71363us (73036us total) trans 4294983335 rc 0/0 00000100:00100000:2.0:1787739056.202755:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23259 00000100:00100000:1.0:1787739056.203407:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+8:3652:x1874579665582720:12345-0@lo:103:ldlm_bl.0 Request processed in 903us (1252us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.203436:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1223 00010000:00010000:2.0:1787739056.205171:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739056.205184:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1abfab lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1abfb2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.205198:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739056.205205:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1abfab lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abfb2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739056.205215:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1abfab lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abfb2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.205271:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1abfab lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abfb2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.205279:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739056.205313:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1abfab lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abfb2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.205321:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1abfab lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abfb2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.205349:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739056.205351:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739056.205353:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739056.205400:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305618b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665582848:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739056.205493:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739056.205950:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665582848 00000100:00100000:2.0:1787739056.205958:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665582848 00000100:00100000:2.0:1787739056.206015:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1224 00000100:00100000:2.0:1787739056.206023:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f712d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+8:3650:x1874579665582848:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739056.206045:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739056.206072:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1abfb2 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abfab expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.206087:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07433235100) 00010000:00010000:2.0:1787739056.206095:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739056.206143:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f712d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+7:3650:x1874579665582848:12345-0@lo:103:ldlm_bl.0 Request processed in 120us (649us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.206150:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1224 00000100:00100000:1.0:1787739056.206219:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305618b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665582848:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739056.208932:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c4152c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665582720:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739056.255864:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739056.255949:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652435200 00000100:00100000:1.0:1787739056.255958:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652435200 00000100:00100000:1.0:1787739056.256061:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23260 00000100:00100000:1.0:1787739056.256083:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17474:x1874579652435200:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739056.256107:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739056.256128:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abff1 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094c6e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.256192:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1abff8 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.256207:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1abff8 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.256218:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1abff8 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.256230:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abfff lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.256237:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abfff lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.256243:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abfff lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.256294:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1abff8 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.256301:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1abff8 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.256307:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1abff8 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.256313:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1abff8 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.256615:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ce00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739056.256620:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1abff8 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.256630:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1abff8 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.256642:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235f00) 00010000:00010000:1.0:1787739056.256644:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739056.256649:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1abff8 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.256667:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1abff8 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.256677:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abfff lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.256683:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abfff lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.256688:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abfff lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.256692:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739056.256694:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abfff lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.256700:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abfff lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.256707:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd431a204 (ffffa07433235000) 00010000:00010000:1.0:1787739056.256718:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739056.256721:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abfff lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.256727:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1abfff lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.256761:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abff1 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c6e expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.256771:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235f00) 00010000:00010000:1.0:1787739056.256774:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739056.256777:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1abff1 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c6e expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.256786:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1cc00, rc 0) 00000100:00100000:1.0:1787739056.256848:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17474:x1874579652435200:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 767us (992us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739056.256856:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23260 00000100:00100000:2.0:1787739056.272763:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739056.272956:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652435328 00000100:00100000:1.0:1787739056.272969:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652435328 00000100:00100000:1.0:1787739056.273043:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23261 00000100:00100000:1.0:1787739056.273053:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073056183c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17474:x1874579652435328:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739056.273083:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739056.273106:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac006 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094c75 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.273167:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac00d lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.273182:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac00d lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.273194:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac00d lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739056.273241:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x5:0x0]: returning size 4096 00000004:00200000:1.0:1787739056.273246:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739056.273284:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac006 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c75 expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.273301:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac00d lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c75 expref: 7 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.273309:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07433235400) 00010000:00010000:1.0:1787739056.273312:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739056.273328:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d000, rc 0) 00000100:00100000:1.0:1787739056.273433:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073056183c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17474:x1874579652435328:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 351us (646us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.273441:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23261 00000100:00100000:3.0:1787739056.310147:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739056.312337:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652435584 00000100:00100000:2.0:1787739056.312355:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652435584 00000100:00100000:2.0:1787739056.312445:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23262 00000100:00100000:2.0:1787739056.312457:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17474:x1874579652435584:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:2.0:1787739056.312573:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305618f00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665582976:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739056.312639:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739056.312684:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d600 going to sleep for 11 seconds 00000100:00100000:1.0:1787739056.315093:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665582976 00000100:00100000:1.0:1787739056.315105:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665582976 00000100:00100000:1.0:1787739056.315154:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2740 00000100:00100000:1.0:1787739056.315164:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665582976:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739056.315273:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665582976:12345-0@lo:1000:mdt00_004.0 Request processed in 110us (2633us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.315280:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2740 00000100:00100000:1.0:1787739056.315334:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305618f00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665582976:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739056.315444:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c4152c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665583104:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739056.315482:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.315511:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739056.315525:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665583104 00000100:00100000:1.0:1787739056.315528:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665583104 00000100:00100000:1.0:1787739056.315551:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2741 00000100:00100000:1.0:1787739056.315557:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c416580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665583104:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739056.315636:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c416580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665583104:12345-0@lo:1000:mdt00_004.0 Request processed in 78us (153us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739056.315643:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2741 00000100:00100000:1.0:1787739056.315670:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c4152c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665583104:0@lo:1000:mdt00_004.0 00010000:00010000:1.0:1787739056.315716:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac014 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.315725:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac014 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.315735:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac014 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.315745:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c4152c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665583232:0@lo:101:mdt00_004.0 00000100:00100000:1.0:1787739056.315769:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.315782:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054180 going to sleep for 11 seconds 00000100:00100000:1.0:1787739056.315793:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665583232 00000100:00100000:1.0:1787739056.315795:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665583232 00000100:00100000:1.0:1787739056.315815:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2742 00000100:00100000:1.0:1787739056.315821:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665583232:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739056.315829:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739056.315843:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac01b lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac014 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.315869:0:8448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1abf26 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c4b expref: 16 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.315878:0:8448:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac01b lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac014 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.315917:0:8448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1abf26 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c4b expref: 16 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.315930:0:8448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1abf26 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c4b expref: 16 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.315941:0:8448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004780 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8448:1874579665583360:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739056.316005:0:8448:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739056.320109:0:8448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004780 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8448:1874579665583360:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739056.320145:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac01b lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac014 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.320158:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235f00) 00010000:00010000:3.0:1787739056.320162:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739056.320166:0:8448:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac01b lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac014 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.320175:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c800, rc 0) 00000100:00100000:3.0:1787739056.320236:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665583232:12345-0@lo:101:mdt00_004.0 Request processed in 4414us (4465us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.320244:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2742 00000100:00100000:3.0:1787739056.320303:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c4152c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665583232:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739056.320318:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac014 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac01b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.320329:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac014 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac01b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739056.323826:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739056.323927:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652435712 00000100:00100000:0.0:1787739056.323937:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652435712 00010000:00010000:0.0:1787739056.323978:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1abf26 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c4b expref: 17 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739056.324020:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1423 00000100:00100000:0.0:1787739056.324039:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652435712:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739056.324056:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739056.324071:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1abf26 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c4b expref: 17 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.324084:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1abf26 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c4b expref: 17 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.324107:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1abf26 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c4b expref: 17 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.324123:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235f00) 00010000:00010000:0.0:1787739056.324129:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac01b lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac014 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.324140:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac01b lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac014 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.324148:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac01b lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac014 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.324178:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac01b lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac014 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739056.324192:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710780 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665583488:0@lo:105: 00000100:00100000:0.0:1787739056.324234:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739056.324602:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234840 going to sleep for 11 seconds 00000100:00100000:0.0:1787739056.324620:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665583488 00000100:00100000:0.0:1787739056.324624:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665583488 00000100:00100000:0.0:1787739056.325517:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1087 00000100:00100000:0.0:1787739056.325529:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665583488:12345-0@lo:105: 00010000:00010000:0.0:1787739056.325542:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac014 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac01b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.325552:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac014 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac01b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.325561:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac014 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac01b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.325569:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac014 lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac01b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.325577:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac014 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac01b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.325586:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac014 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ac01b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.325609:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07441a1cc00) 00000100:00100000:0.0:1787739056.325663:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665583488:12345-0@lo:105: Request processed in 137us (1428us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.325671:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1087 00010000:00010000:3.0:1787739056.329239:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac014 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac01b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.329271:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac014 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac01b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739056.329300:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739056.329353:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ac022 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.329361:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ac022 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.329381:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ac022 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739056.329426:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be6580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665583616:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787739056.329491:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739056.329533:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fccc00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739056.329567:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665583616 00000100:00100000:3.0:1787739056.329573:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665583616 00000100:00100000:3.0:1787739056.329619:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2743 00000100:00100000:3.0:1787739056.329628:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07436d8fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665583616:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739056.329644:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739056.329660:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac029 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac022 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.329685:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac029 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac022 expref: 11 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.329698:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac029 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac022 expref: 11 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.329706:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd431a204 (ffffa07305d56300) 00010000:00010000:3.0:1787739056.330304:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739056.330309:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab200, rc 0) 00000100:00100000:3.0:1787739056.330376:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07436d8fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665583616:12345-0@lo:101:mdt00_004.0 Request processed in 747us (883us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.330384:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2743 00000100:00100000:3.0:1787739056.330617:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be6580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665583616:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739056.330636:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ac022 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac029 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.330646:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ac022 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac029 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739056.330661:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:3.0:1787739056.330699:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be6580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665583744:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739056.330924:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739056.330959:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739056.330978:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665583744 00000100:00100000:3.0:1787739056.330983:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665583744 00000100:00100000:3.0:1787739056.331015:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2744 00000100:00100000:3.0:1787739056.331022:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665583744:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739056.331198:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665583744:12345-0@lo:1000:mdt00_004.0 Request processed in 176us (274us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.331205:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2744 00000100:00100000:3.0:1787739056.331234:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be6580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665583744:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739056.331392:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be6580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665583872:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739056.331500:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739056.331520:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739056.331544:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665583872 00000100:00100000:3.0:1787739056.331548:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665583872 00000100:00100000:3.0:1787739056.331571:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2745 00000100:00100000:3.0:1787739056.331578:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be6940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665583872:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739056.331647:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be6940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665583872:12345-0@lo:1000:mdt00_004.0 Request processed in 69us (147us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739056.331654:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2745 00000100:00100000:3.0:1787739056.331681:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be6580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665583872:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739056.331933:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be6580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665584000:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739056.332007:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739056.332025:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739056.334872:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665584000 00000100:00100000:3.0:1787739056.334887:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665584000 00000100:00100000:3.0:1787739056.334957:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2746 00000100:00100000:3.0:1787739056.334968:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665584000:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739056.335827:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665584000:12345-0@lo:1000:mdt00_004.0 Request processed in 858us (3817us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739056.335837:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2746 00000100:00100000:3.0:1787739056.335892:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be6580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665584000:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739056.335973:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665584128:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739056.336009:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739056.336811:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739056.336838:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665584128 00000100:00100000:3.0:1787739056.336844:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665584128 00000100:00100000:3.0:1787739056.336885:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2747 00000100:00100000:3.0:1787739056.336893:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665584128:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739056.337219:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665584128:12345-0@lo:1000:mdt00_004.0 Request processed in 325us (1209us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.337227:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2747 00000100:00100000:3.0:1787739056.337281:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665584128:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739056.337348:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665584256:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739056.337378:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739056.337847:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739056.337864:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665584256 00000100:00100000:3.0:1787739056.337869:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665584256 00000100:00100000:3.0:1787739056.337898:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2748 00000100:00100000:3.0:1787739056.337905:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be6580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665584256:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739056.339215:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710780 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665583488:0@lo:105: 00010000:00010000:0.0:1787739056.339238:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739056.339300:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:2409:x1874579652435712:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 15270us (15482us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.339309:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1423 00000100:00100000:3.0:1787739056.340158:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be6580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665584256:12345-0@lo:1000:mdt00_004.0 Request processed in 2250us (2776us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.340171:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2748 00000100:00100000:0.0:1787739056.342264:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665584256:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739056.342854:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665584384:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739056.342914:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739056.342951:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062340c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739056.343057:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665584384 00000100:00100000:3.0:1787739056.343072:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665584384 00000100:00100000:3.0:1787739056.343127:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2749 00000100:00100000:3.0:1787739056.343136:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f711e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665584384:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739056.343219:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f711e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665584384:12345-0@lo:1000:mdt00_004.0 Request processed in 84us (305us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.343226:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2749 00000100:00100000:0.0:1787739056.345683:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665584384:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739056.345767:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665584512:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739056.345822:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739056.345938:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062340c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739056.347898:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665584512 00000100:00100000:3.0:1787739056.347911:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665584512 00000100:00100000:3.0:1787739056.348060:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2750 00000100:00100000:3.0:1787739056.348081:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665584512:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739056.348201:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665584512:12345-0@lo:1000:mdt00_004.0 Request processed in 122us (2377us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.348209:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2750 00000100:00100000:1.0:1787739056.348288:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665584512:0@lo:1000:mdt00_004.0 00000040:00080000:1.0:1787739056.349165:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].339, 2168 off516920 00000040:00080000:1.0:1787739056.349243:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].340, 2368 off565648 00000004:00080000:1.0:1787739056.349250:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07307054480 with version 648 00000004:00080000:1.0:1787739056.349275:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 648 rpc_version 648 00000020:00100000:1.0:1787739056.349309:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739056.349317:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073060061c0 x1874579665584768/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739056.349337:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060061c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665584768:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739056.349389:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.350248:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4540 going to sleep for 11 seconds 00000100:00100000:0.0:1787739056.350607:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665584768 00000100:00100000:0.0:1787739056.350615:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665584768 00000100:00100000:1.0:1787739056.350926:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2751 00000100:00100000:1.0:1787739056.350935:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060070c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:7467:x1874579665584768:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739056.352482:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060070c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:7467:x1874579665584768:12345-0@lo:1000:osp_up0-1.0 Request processed in 1544us (3088us total) trans 4294983336 rc 0/0 00000100:00100000:1.0:1787739056.352494:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2751 00010000:00010000:0.0:1787739056.354920:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac030 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x165:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.355056:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac030 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x165:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.355078:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac030 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x165:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.355100:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac037 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.355108:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac037 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.355114:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac037 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.355147:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac03e lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.355154:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac03e lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.355163:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac03e lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739056.355183:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710f00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665584896:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739056.355242:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739056.355352:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739056.355596:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665584896 00000100:00100000:0.0:1787739056.355603:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665584896 00000100:00100000:0.0:1787739056.355648:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2752 00000100:00100000:0.0:1787739056.355657:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:13793:x1874579665584896:12345-0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739056.355674:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739056.355690:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac045 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac03e expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.355778:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac045 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac03e expref: 13 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.355790:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac045 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac03e expref: 13 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.355799:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb9:0x0].0x0 (ffffa07307064e00) 00010000:00010000:0.0:1787739056.355801:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739056.355804:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a000, rc 0) 00000100:00100000:3.0:1787739056.355915:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060061c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665584768:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739056.356025:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+13:13793:x1874579665584896:12345-0@lo:101:mdt00_004.0 Request processed in 369us (782us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.356032:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2752 00000100:00100000:0.0:1787739056.356077:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710f00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665584896:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739056.356090:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac03e lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac045 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.356100:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac03e lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac045 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739056.356119:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002bb00bfc reply state 00000000763aec46 transno 4294968324 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac037 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.356131:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac037 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.356141:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb9:0x0].0x0 (ffffa07307064200) 00010000:00010000:0.0:1787739056.356143:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739056.356145:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739056.356150:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac03e lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac045 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.356158:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac03e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac045 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739056.356168:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xb9:0x0] 00000004:00010000:0.0:1787739056.356174:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002bb00bfc reply state 00000000763aec46 transno 4294968324 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac030 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x165:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.356180:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac030 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x165:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.356186:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x165:0x0].0x0 (ffffa07307064700) 00010000:00010000:0.0:1787739056.356188:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739056.356190:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787739056.356196:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x165:0x0]: returning size 4096 00000004:00080000:0.0:1787739056.356204:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739056.356215:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac014 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac01b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.356222:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac014 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac01b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.356228:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac014 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac01b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.356928:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739056.356995:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739056.358134:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739056.358146:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac014 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac01b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.358156:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1cc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739056.358162:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac014 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac01b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739056.358171:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac014 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac01b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.358227:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac014 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac01b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.358234:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739056.358552:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac014 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac01b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.358562:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac014 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac01b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.358582:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739056.358584:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739056.358586:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739056.358598:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787739056.358616:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fcb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665585024:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739056.358674:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787739056.358816:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ac022 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac029 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.358827:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ac022 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac029 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.358927:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17474:x1874579652435584:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 46471us (48788us total) trans 4294968324 rc 0/0 00000100:00100000:1.0:1787739056.358936:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23262 00000100:00100000:3.0:1787739056.360111:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665585024 00000100:00100000:3.0:1787739056.360122:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665585024 00000100:00100000:3.0:1787739056.360181:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1225 00000100:00100000:3.0:1787739056.360190:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fd2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+13:3650:x1874579665585024:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739056.360205:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739056.360228:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac01b lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac014 expref: 13 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.360244:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235f00) 00010000:00010000:3.0:1787739056.362766:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739056.363198:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fd2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+12:3650:x1874579665585024:12345-0@lo:103:ldlm_bl.0 Request processed in 3005us (4518us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.363211:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1225 00000100:00100000:3.0:1787739056.364155:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fcb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665585024:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739056.435180:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739056.435218:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652435968 00000100:00100000:0.0:1787739056.435226:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652435968 00000100:00100000:0.0:1787739056.435278:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23263 00000100:00100000:0.0:1787739056.435288:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17475:x1874579652435968:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:0.0:1787739056.435520:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17475:x1874579652435968:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 233us (348us total) trans 0 rc -17/-17 00000100:00100000:0.0:1787739056.435527:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23263 00010000:00010000:0.0:1787739056.450875:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739056.464348:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739056.465591:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652436096 00000100:00100000:0.0:1787739056.465604:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652436096 00000100:00100000:0.0:1787739056.465713:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23264 00000100:00100000:0.0:1787739056.465724:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17476:x1874579652436096:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739056.466335:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739056.466367:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac04c lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094c7c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.466480:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac053 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.466495:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac053 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.466506:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac053 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.466522:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac05a lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.466532:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac029 lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac022 expref: 11 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.466540:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac05a lrc: 4/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.466573:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac029 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ac022 expref: 11 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739056.466589:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665585152:0@lo:104: 00000100:00100000:0.0:1787739056.466670:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739056.466727:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739056.469320:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665585152 00000100:00100000:3.0:1787739056.469333:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665585152 00000100:00100000:3.0:1787739056.469404:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1088 00000100:00100000:3.0:1787739056.469441:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f7c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6523:x1874579665585152:12345-0@lo:104: 00010000:00010000:3.0:1787739056.469459:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ac022 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac029 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.469555:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739056.469562:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f7c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6523:x1874579665585152:12345-0@lo:104: Request processed in 150us (2892us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.469569:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1088 00010000:00010000:3.0:1787739056.469592:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739056.469598:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ac022 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac029 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.469619:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739056.469624:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ac022 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ac029 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739056.469633:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ac022 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ac029 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739056.469785:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665585152:0@lo:104: 00010000:00010000:0.0:1787739056.469807:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac05a lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.470255:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ac022 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac029 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.470268:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739056.470300:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ac022 lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac029 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.470307:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ac022 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac029 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.470325:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739056.470327:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739056.470329:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739056.470336:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787739056.470349:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fcb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665585280:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739056.470388:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739056.471396:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665585280 00000100:00100000:0.0:1787739056.471405:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665585280 00010000:00010000:0.0:1787739056.471459:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac029 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ac022 expref: 12 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739056.471490:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1424 00000100:00100000:0.0:1787739056.471498:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fda40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+12:3653:x1874579665585280:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739056.471511:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739056.480367:0:6502:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1abfb9 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1abfc0 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739056.480388:0:6502:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1abfb9 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1abfc0 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.480445:0:6502:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1abfb9 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abfc0 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.480453:0:6502:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739056.480475:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1abfb9 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1abfc0 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787739056.480494:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983336 is committed 00000001:00080000:3.0:1787739056.480514:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983335 is committed 00000020:00080000:3.0:1787739056.480524:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 353 batchid updated 00000020:00080000:3.0:1787739056.480544:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07410b4d900 refcount 2 committed 0 result 0 batchid 354 00000020:00080000:1.0:1787739056.480544:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 353 00000020:00080000:3.0:1787739056.480551:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88600 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07410b4d000 00000020:00080000:3.0:1787739056.480555:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].351 00000020:00080000:3.0:1787739056.480557:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88f00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304011e00 00000020:00080000:3.0:1787739056.480560:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].352 00000100:00100000:3.0:1787739056.480590:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fcf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665585408:0@lo:103:jbd2/dm-0-8.0 00000004:00080000:1.0:1787739056.480608:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730610ef00 with version 673 00000020:00080000:1.0:1787739056.480619:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 353 cancel update log [0x240000401:0x3:0x0].350: rc = 0 00000100:00100000:3.0:1787739056.480679:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000020:00080000:1.0:1787739056.480695:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 353 cancel update log [0x200000400:0x3:0x0].351: rc = 0 00000020:00080000:1.0:1787739056.480707:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 354 committed batchid 353 00000020:00080000:1.0:1787739056.480727:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 353 00000004:00080000:1.0:1787739056.480737:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 673 rpc_version 673 00000020:00100000:1.0:1787739056.480755:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739056.480764:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306007c00 x1874579665585664/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739056.480779:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306007c00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665585664:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739056.480816:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.480839:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739056.481680:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665585408 00000100:00100000:1.0:1787739056.481690:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665585408 00000100:00100000:1.0:1787739056.481742:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1226 00000100:00100000:1.0:1787739056.481752:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fe940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+7:3653:x1874579665585408:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:1.0:1787739056.481767:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739056.481789:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1abfc0 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xe09b0941 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1abfb9 expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.481806:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xe09b0941 (ffffa07433235700) 00010000:00010000:1.0:1787739056.481815:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739056.481851:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665585664 00000100:00100000:0.0:1787739056.481858:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665585664 00000100:00100000:1.0:1787739056.481866:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fe940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+7:3653:x1874579665585408:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 114us (1186us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.481874:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1226 00000100:00100000:0.0:1787739056.481898:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2753 00010000:00010000:1.0:1787739056.481903:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abf49 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1501:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739056.481909:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665585664:12345-0@lo:1000:osp_up1-0.0 00010000:00010000:1.0:1787739056.481921:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abf49 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1501:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.481931:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abf49 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1501:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.481937:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739056.481943:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abf49 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1501:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.481950:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abf49 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1501:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.481961:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1501:0x0].0x0 (ffffa073068a9e00) 00010000:00010000:1.0:1787739056.481964:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739056.481969:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abf49 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1501:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.481975:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1abf49 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1501:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.481991:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abf50 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.481998:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abf50 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.482005:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abf50 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.482011:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609bc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739056.482013:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abf50 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.482019:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abf50 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.482026:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xbb:0x0].0x0 (ffffa073068a9a00) 00010000:00010000:1.0:1787739056.482029:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739056.482032:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abf50 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.482037:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1abf50 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xbb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739056.482083:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665585664:12345-0@lo:1000:osp_up1-0.0 Request processed in 176us (1264us total) trans 4294968325 rc 0/0 00000100:00100000:0.0:1787739056.482091:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2753 00000100:00100000:1.0:1787739056.482111:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306007c00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665585664:0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1787739056.482276:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac029 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ac022 expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.482291:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd431a204 (ffffa07305d56300) 00010000:00010000:2.0:1787739056.482294:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac05a lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.482303:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac05a lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.482310:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac05a lrc: 4/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.482341:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739056.482372:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fda40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+6:3653:x1874579665585280:12345-0@lo:103:ldlm_bl.0 Request processed in 10876us (11984us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.482379:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1424 00000100:00100000:3.0:1787739056.483153:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fcf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665585408:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:3.0:1787739056.483170:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fcb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665585280:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739056.483524:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac05a lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.483534:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac05a lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.483643:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665585792:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739056.483693:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.483727:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ecc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739056.483873:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665585792 00000100:00100000:3.0:1787739056.483880:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665585792 00000100:00100000:3.0:1787739056.483920:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2754 00000100:00100000:3.0:1787739056.483928:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306006940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665585792:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739056.484013:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306006940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665585792:12345-0@lo:1000:mdt00_000.0 Request processed in 85us (319us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.484020:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2754 00000100:00100000:2.0:1787739056.491130:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665585792:0@lo:1000:mdt00_000.0 00010000:00010000:2.0:1787739056.491202:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac061 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x165:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.491224:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac061 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x165:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.491236:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac061 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x165:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739056.491262:0:6523:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac061 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x165:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.491280:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac053 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.491287:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac053 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.491294:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac053 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.491301:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac053 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.491308:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739056.491313:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac053 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.491322:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac053 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.491335:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07307064d00) 00010000:00010000:2.0:1787739056.491338:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739056.491342:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac053 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.491349:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac053 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.491363:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac05a lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.491369:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac05a lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.491376:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac05a lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.491382:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739056.491385:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac05a lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.491391:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac05a lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.491399:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd431a204 (ffffa07305d56300) 00010000:00010000:2.0:1787739056.491402:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739056.491405:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac05a lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.491427:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac05a lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.491466:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac04c lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c7c expref: 15 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.491485:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac061 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x165:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c7c expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.491493:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x165:0x0].0x0 (ffffa074419d5700) 00010000:00010000:2.0:1787739056.491496:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739056.491498:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d600, rc 0) 00000100:00100000:2.0:1787739056.491559:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17476:x1874579652436096:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 25837us (27220us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.491569:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23264 00000100:00100000:0.0:1787739056.511481:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739056.513090:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652436224 00000100:00100000:1.0:1787739056.513103:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652436224 00000100:00100000:1.0:1787739056.513182:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23265 00000100:00100000:1.0:1787739056.513193:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17476:x1874579652436224:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739056.513219:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739056.513240:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac068 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x165:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094c83 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.513295:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac06f lrc: 3/1,0 mode: --/PR res: [0x240000402:0x165:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.513309:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac06f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x165:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.513320:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac06f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x165:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739056.513431:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x165:0x0]: returning size 4096 00000004:00200000:1.0:1787739056.513437:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739056.513475:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac068 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x165:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c83 expref: 11 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.513498:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac06f lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x165:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c83 expref: 10 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.513506:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x165:0x0].0x0 (ffffa07307064700) 00010000:00010000:1.0:1787739056.513509:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739056.513513:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d800, rc 0) 00000100:00100000:1.0:1787739056.513584:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17476:x1874579652436224:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 393us (2113us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.513591:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23265 00000100:00100000:2.0:1787739056.531774:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739056.532175:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652436352 00000100:00100000:0.0:1787739056.532185:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652436352 00000100:00100000:0.0:1787739056.532245:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23266 00000100:00100000:0.0:1787739056.532256:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305619680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17476:x1874579652436352:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739056.532277:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739056.532297:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ac076 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094c8a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.532354:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac07d lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.532368:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac07d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.532379:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac07d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739056.532443:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xb9:0x0]: returning size 4096 00000004:00200000:0.0:1787739056.532448:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739056.532483:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ac076 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c8a expref: 12 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.532503:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac07d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c8a expref: 11 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.532512:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb9:0x0].0x0 (ffffa07307064200) 00010000:00010000:0.0:1787739056.532516:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739056.532520:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609ac00, rc 0) 00000100:00100000:0.0:1787739056.532576:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305619680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17476:x1874579652436352:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 322us (812us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.532584:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23266 00000100:00100000:2.0:1787739056.547609:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739056.547711:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652436480 00000100:00100000:1.0:1787739056.547721:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652436480 00000100:00100000:1.0:1787739056.547791:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23267 00000100:00100000:1.0:1787739056.547802:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305619e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17476:x1874579652436480:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739056.547823:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739056.547843:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac084 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094c91 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.547907:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac08b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.547923:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac045 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac03e expref: 5 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.547932:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac08b lrc: 4/1,0 mode: --/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.547962:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac045 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ac03e expref: 5 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.547976:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060043c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665585920:0@lo:104: 00000100:00100000:1.0:1787739056.548023:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739056.548083:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665585920 00000100:00100000:3.0:1787739056.548088:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665585920 00000100:00100000:1.0:1787739056.548089:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739056.548129:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1089 00000100:00100000:3.0:1787739056.548138:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306005e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6523:x1874579665585920:12345-0@lo:104: 00010000:00010000:3.0:1787739056.548151:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac03e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac045 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.548223:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739056.548229:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306005e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6523:x1874579665585920:12345-0@lo:104: Request processed in 93us (206us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.548236:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1089 00010000:00010000:2.0:1787739056.548305:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739056.548312:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac03e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac045 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.548321:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739056.548326:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac03e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ac045 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739056.548335:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac03e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ac045 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.548361:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac03e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac045 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.548369:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739056.548405:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac03e lrc: 1/0,0 mode: --/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac045 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.548435:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac03e lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac045 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.548454:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739056.548456:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739056.548457:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739056.548465:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787739056.548476:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665586048:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739056.548507:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.548634:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060043c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665585920:0@lo:104: 00010000:00010000:1.0:1787739056.548651:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac08b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739056.548907:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665586048 00000100:00100000:0.0:1787739056.548912:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665586048 00010000:00010000:0.0:1787739056.548934:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac045 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ac03e expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739056.548966:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1425 00000100:00100000:0.0:1787739056.548973:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074301a2450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+6:3651:x1874579665586048:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739056.548983:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00080000:3.0:1787739056.554771:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 354 batchid updated 00000020:00080000:3.0:1787739056.554798:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 354 00000004:00080000:3.0:1787739056.554851:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07305fcc780 with version 674 00000020:00080000:3.0:1787739056.554860:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 354 cancel update log [0x240000401:0x3:0x0].351: rc = 0 00000020:00080000:3.0:1787739056.554923:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 354 cancel update log [0x200000400:0x3:0x0].352: rc = 0 00000020:00080000:3.0:1787739056.554933:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 354 00000004:00080000:3.0:1787739056.554956:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 674 rpc_version 674 00000020:00100000:3.0:1787739056.554987:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739056.554999:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073061fcb40 x1874579665586304/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739056.555022:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fcb40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665586304:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739056.555096:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739056.555133:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665586304 00000100:00100000:3.0:1787739056.555138:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739056.555143:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665586304 00010000:00010000:2.0:1787739056.555428:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac045 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ac03e expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.555446:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb9:0x0].0x0 (ffffa07307064e00) 00010000:00010000:2.0:1787739056.555452:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac08b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.555461:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac08b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.555468:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac08b lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739056.555715:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2755 00000100:00100000:3.0:1787739056.555725:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fcf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665586304:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739056.555862:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fcf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665586304:12345-0@lo:1000:osp_up1-0.0 Request processed in 139us (767us total) trans 4294968326 rc 0/0 00000100:00100000:3.0:1787739056.555870:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2755 00010000:00010000:3.0:1787739056.555910:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739056.555925:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac08b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.555933:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac08b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739056.555992:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb9:0x0]: returning size 4096 00000004:00200000:3.0:1787739056.555997:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739056.556034:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac084 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c91 expref: 16 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.556051:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac08b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c91 expref: 15 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.556058:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb9:0x0].0x0 (ffffa07307064e00) 00010000:00010000:3.0:1787739056.556071:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739056.556074:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1ce00, rc 0) 00000100:00100000:3.0:1787739056.556114:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305619e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17476:x1874579652436480:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 8314us (8514us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.556121:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23267 00000100:00100000:3.0:1787739056.556169:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074301a2450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665586048:12345-0@lo:103:ldlm_bl.0 Request processed in 7196us (7661us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.556178:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1425 00000100:00100000:2.0:1787739056.556613:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665586048:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739056.556768:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fcb40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665586304:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739056.569389:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739056.570133:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652436608 00000100:00100000:3.0:1787739056.570145:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652436608 00000100:00100000:3.0:1787739056.570211:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1227 00000100:00100000:3.0:1787739056.570222:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060043c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2408:x1874579652436608:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:3.0:1787739056.570237:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739056.570262:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac061 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x165:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c7c expref: 15 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.570277:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x165:0x0].0x0 (ffffa074419d5700) 00010000:00010000:3.0:1787739056.570286:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739056.570341:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060043c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2408:x1874579652436608:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 121us (959us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.570349:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1227 00000100:00100000:3.0:1787739056.571284:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739056.571363:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652436864 00000100:00100000:1.0:1787739056.571371:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652436864 00000100:00100000:1.0:1787739056.571436:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23268 00000100:00100000:1.0:1787739056.571446:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fd2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17476:x1874579652436864:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:1.0:1787739056.571497:0:13793:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739056.571516:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac06f lrc: 0/0,0 mode: --/PR res: [0x240000402:0x165:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c83 expref: 11 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.571532:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x165:0x0].0x0 (ffffa07307064700) 00010000:00010000:1.0:1787739056.571536:0:13793:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739056.571579:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac092 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.571586:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac092 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.571595:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac092 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.571627:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306005680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665586432:0@lo:101:mdt00_004.0 00000100:00100000:1.0:1787739056.571668:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.571706:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739056.572840:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665586432 00000100:00100000:3.0:1787739056.572850:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665586432 00000100:00100000:3.0:1787739056.572903:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2756 00000100:00100000:3.0:1787739056.572911:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306006d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+5:13793:x1874579665586432:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739056.572929:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739056.572948:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ac099 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac092 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.572981:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ac099 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac092 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.572995:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ac099 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac092 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.573003:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305d56a00) 00010000:00010000:3.0:1787739056.573005:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739056.573008:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa800, rc 0) 00000100:00100000:3.0:1787739056.573082:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306006d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:13793:x1874579665586432:12345-0@lo:101:mdt00_004.0 Request processed in 172us (1413us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.573089:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2756 00000100:00100000:1.0:1787739056.573174:0:13793:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983336 gen 2 00000004:00080000:1.0:1787739056.573181:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983330 committed transno 4294983336 00000020:00080000:1.0:1787739056.573186:0:13793:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa073068ba9c0 refcount 2 committed 0 result 0 batchid 345 00000020:00080000:1.0:1787739056.573190:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730415f400 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa073068ba3c0 00000020:00080000:1.0:1787739056.573193:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].339 00000020:00080000:1.0:1787739056.573196:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730415f200 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304010a00 00000020:00080000:1.0:1787739056.573199:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].338 00000004:00080000:1.0:1787739056.573227:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983332 committed transno 4294983336 00000004:00080000:1.0:1787739056.573237:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983333 committed transno 4294983336 00000004:00080000:1.0:1787739056.573245:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983334 committed transno 4294983336 00000004:00080000:1.0:1787739056.573252:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983336 committed transno 4294983336 00000100:00100000:1.0:1787739056.573268:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306005680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665586432:0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739056.573284:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac092 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac099 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.573295:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac092 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac099 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739056.573311:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739056.573336:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1ca00/0x7ee4453e4a1ac0a0 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.573343:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1ca00/0x7ee4453e4a1ac0a0 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.573352:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1ca00/0x7ee4453e4a1ac0a0 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.573366:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306005680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665586560:0@lo:101:mdt00_004.0 00000100:00100000:1.0:1787739056.573396:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.573442:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739056.573458:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665586560 00000100:00100000:1.0:1787739056.573461:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665586560 00000100:00100000:1.0:1787739056.573487:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2757 00000100:00100000:1.0:1787739056.573493:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060061c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:13793:x1874579665586560:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739056.573501:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739056.573514:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac0a7 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac0a0 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.573530:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac0a7 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac0a0 expref: 7 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.573540:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac0a7 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac0a0 expref: 7 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.573547:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd431a204 (ffffa07433235800) 00010000:00010000:1.0:1787739056.573550:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739056.573553:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c800, rc 0) 00000100:00100000:1.0:1787739056.573580:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060061c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665586560:12345-0@lo:101:mdt00_004.0 Request processed in 87us (183us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.573586:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2757 00000100:00100000:1.0:1787739056.573614:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306005680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665586560:0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739056.573625:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1ca00/0x7ee4453e4a1ac0a0 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac0a7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.573632:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1ca00/0x7ee4453e4a1ac0a0 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac0a7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739056.573642:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:1.0:1787739056.573671:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306005680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665586688:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739056.573697:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.573712:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e240 going to sleep for 11 seconds 00000100:00100000:1.0:1787739056.573724:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665586688 00000100:00100000:1.0:1787739056.573727:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665586688 00000100:00100000:1.0:1787739056.573759:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2758 00000100:00100000:1.0:1787739056.573765:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306006580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665586688:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739056.573818:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306006580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665586688:12345-0@lo:1000:mdt00_004.0 Request processed in 52us (119us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.573824:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2758 00000100:00100000:1.0:1787739056.573850:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306005680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665586688:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739056.573883:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306005680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665586816:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739056.573907:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.573920:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e240 going to sleep for 11 seconds 00000100:00100000:1.0:1787739056.573932:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665586816 00000100:00100000:1.0:1787739056.573935:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665586816 00000100:00100000:1.0:1787739056.573954:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2759 00000100:00100000:1.0:1787739056.573960:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665586816:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739056.574027:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665586816:12345-0@lo:1000:mdt00_004.0 Request processed in 66us (118us total) trans 0 rc 1/1 00000100:00100000:1.0:1787739056.574033:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2759 00000100:00100000:1.0:1787739056.574057:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306005680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665586816:0@lo:1000:mdt00_004.0 00010000:00010000:1.0:1787739056.574106:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac0ae lrc: 3/0,1 mode: --/EX res: [0x240000402:0x165:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.574113:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac0ae lrc: 3/0,1 mode: --/EX res: [0x240000402:0x165:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.574121:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac0ae lrc: 3/0,1 mode: --/EX res: [0x240000402:0x165:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.574142:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306005680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665586944:0@lo:101:mdt00_004.0 00000100:00100000:1.0:1787739056.574162:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.574176:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739056.574186:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665586944 00000100:00100000:1.0:1787739056.574189:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665586944 00000100:00100000:1.0:1787739056.574210:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2760 00000100:00100000:1.0:1787739056.574216:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665586944:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739056.574222:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739056.574233:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ac0b5 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x165:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac0ae expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.574248:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ac0b5 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x165:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac0ae expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.574257:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ac0b5 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x165:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac0ae expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.574269:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x165:0x0].0x0 (ffffa07433235b00) 00010000:00010000:1.0:1787739056.574272:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739056.574275:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c600, rc 0) 00000100:00100000:1.0:1787739056.574299:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665586944:12345-0@lo:101:mdt00_004.0 Request processed in 83us (135us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.574304:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2760 00000100:00100000:1.0:1787739056.574329:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306005680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665586944:0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739056.574339:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac0ae lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x165:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac0b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.574347:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac0ae lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x165:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac0b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.574366:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac0bc lrc: 3/0,1 mode: --/EX res: [0x240000402:0x165:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.574375:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac0bc lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x165:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.574383:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac0bc lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x165:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.574400:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac0c3 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.574411:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac07d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c8a expref: 10 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.574446:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac0c3 lrc: 4/0,1 mode: --/EX res: [0x240000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.574468:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac07d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c8a expref: 10 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.574480:0:13793:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac07d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c8a expref: 10 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.574489:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306005680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665587072:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739056.574545:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000020:00080000:3.0:1787739056.575912:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 345 committed batchid 344 00080000:02000000:3.0:1787739056.575938:0:7475:0:(osd_handler.c:2218:osd_trans_start()) *** cfs_fail_loc=19a, val=0*** 00000020:00080000:3.0:1787739056.575952:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 345 committed batchid 344 00000100:00100000:0.0:1787739056.579628:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665587200:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787739056.579713:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739056.579823:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665587200 00000100:00100000:2.0:1787739056.579831:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665587200 00000100:00100000:2.0:1787739056.579887:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 352 00000100:00100000:2.0:1787739056.579898:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f712d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665587200:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1787739056.579929:0:8441:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 25431488 pending 0 free 3923496960 avail 3691974656 00000100:00100000:2.0:1787739056.579978:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f712d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665587200:12345-0@lo:13:osp-pre-1-0.0 Request processed in 83us (265us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.579986:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 352 00000100:00080000:2.0:1787739056.580004:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1787739056.580220:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665587200:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1787739056.580221:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874579665587456:0@lo:400:kworker.0 00000100:00100000:0.0:1787739056.580240:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304002940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3651:1874579665587712:0@lo:400:kworker.0 00000100:00100000:1.0:1787739056.580251:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739056.580272:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.580278:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304003480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665587968:0@lo:400:kworker.0 00000100:00100000:0.0:1787739056.580285:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304001a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3651:1874579665588224:0@lo:400:kworker.0 00000100:00100000:1.0:1787739056.580300:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739056.580303:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.580316:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304001e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665588480:0@lo:400:kworker.0 00000100:00100000:1.0:1787739056.580336:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739056.580355:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2:3652:1874579665587328:0@lo:400:kworker.0 00000100:00100000:1.0:1787739056.580365:0:8438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665587968 00000100:00100000:1.0:1787739056.580368:0:8438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665587968 00000100:00100000:2.0:1787739056.580383:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.580398:0:8438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665588480 00000100:00100000:1.0:1787739056.580400:0:8438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665588480 00000100:00100000:2.0:1787739056.580404:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304000000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3652:1874579665587840:0@lo:400:kworker.0 00000100:00100000:1.0:1787739056.580431:0:8438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 343 00000100:00100000:1.0:1787739056.580438:0:8438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304000f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3653:x1874579665587968:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1787739056.580440:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787739056.580442:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787739056.580449:0:6516:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665587328 00000100:00100000:2.0:1787739056.580452:0:6516:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665587328 00000100:00100000:1.0:1787739056.580474:0:8438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304000f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3653:x1874579665587968:12345-0@lo:400:kworker.0 Request processed in 36us (172us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.580477:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 446 00000100:00100000:2.0:1787739056.580482:0:6516:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410b16450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2+24:3652:x1874579665587328:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787739056.580483:0:8438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 343 00000100:00100000:1.0:1787739056.580495:0:8438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 344 00000100:00100000:1.0:1787739056.580500:0:8438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304003840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3653:x1874579665588480:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739056.580512:0:6516:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410b16450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2+24:3652:x1874579665587328:12345-0@lo:400:kworker.0 Request processed in 30us (129us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.580518:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 446 00000100:00100000:1.0:1787739056.580522:0:8438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304003840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3653:x1874579665588480:12345-0@lo:400:kworker.0 Request processed in 23us (185us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.580527:0:8438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 344 00000100:00100000:1.0:1787739056.580547:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304000b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3650:1874579665587584:0@lo:400:kworker.0 00000100:00100000:1.0:1787739056.580582:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304003480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665587968:0@lo:400:kworker.0 00000100:00100000:1.0:1787739056.580591:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304001e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665588480:0@lo:400:kworker.0 00000100:00100000:1.0:1787739056.584837:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.588533:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304002d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665588096:0@lo:400:kworker.0 00000100:00100000:1.0:1787739056.588594:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.588612:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305618f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665588608:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1787739056.588636:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.588686:0:14680:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665588096 00000100:00100000:1.0:1787739056.588693:0:14680:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665588096 00000100:00100000:1.0:1787739056.588755:0:14680:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 345 00000100:00100000:1.0:1787739056.588764:0:14680:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304003480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_011:lustre-MDT0000-mdtlov_UUID+5:3650:x1874579665588096:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787739056.588817:0:14680:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304003480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_011:lustre-MDT0000-mdtlov_UUID+5:3650:x1874579665588096:12345-0@lo:400:kworker.0 Request processed in 53us (222us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.588824:0:14680:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 345 00000100:00100000:2.0:1787739056.588867:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665587456 00000100:00100000:2.0:1787739056.588872:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665587456 00000100:00100000:1.0:1787739056.588874:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304002d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665588096:0@lo:400:kworker.0 00000100:00100000:2.0:1787739056.588895:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665587712 00000100:00100000:2.0:1787739056.588897:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665587712 00000100:00100000:2.0:1787739056.588911:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665588224 00000100:00100000:2.0:1787739056.588913:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665588224 00000100:00100000:2.0:1787739056.588925:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665587584 00000100:00100000:2.0:1787739056.588927:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665587584 00000100:00100000:2.0:1787739056.588938:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 648 00000100:00100000:2.0:1787739056.588944:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073040030c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874579665587456:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739056.588991:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073040030c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3653:x1874579665587456:12345-0@lo:400:kworker.0 Request processed in 47us (8739us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.588996:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 648 00000100:00100000:2.0:1787739056.589010:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 649 00000100:00100000:2.0:1787739056.589016:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f78780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1874579665587712:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739056.589048:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f78780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3651:x1874579665587712:12345-0@lo:400:kworker.0 Request processed in 33us (8775us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.589053:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 649 00000100:00100000:2.0:1787739056.589076:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 650 00000100:00100000:2.0:1787739056.589083:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f78f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3651:x1874579665588224:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739056.589110:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f78f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3651:x1874579665588224:12345-0@lo:400:kworker.0 Request processed in 28us (8805us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.589116:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 650 00000100:00100000:2.0:1787739056.589126:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 651 00000100:00100000:2.0:1787739056.589132:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304001e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3650:x1874579665587584:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739056.589162:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304001e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3650:x1874579665587584:12345-0@lo:400:kworker.0 Request processed in 31us (4334us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.589167:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 651 00000100:00100000:2.0:1787739056.589186:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739056.589789:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304001680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3652:1874579665588352:0@lo:400:kworker.0 00000100:00100000:2.0:1787739056.589819:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739056.589846:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2:3652:1874579665587328:0@lo:400:kworker.0 00000100:00100000:2.0:1787739056.589872:0:10361:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665588608 00000100:00100000:2.0:1787739056.589875:0:10361:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665588608 00000100:00100000:2.0:1787739056.589904:0:10361:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 353 00000100:00100000:2.0:1787739056.589911:0:10361:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073040021c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1874579665588608:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1787739056.589927:0:10361:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 25431488 pending 0 free 3923496960 avail 3691974656 00000100:00100000:2.0:1787739056.589958:0:10361:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073040021c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1874579665588608:12345-0@lo:13:osp-pre-1-1.0 Request processed in 48us (1320us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.589964:0:10361:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 353 00000100:00080000:2.0:1787739056.589980:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787739056.589985:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787739056.589996:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665587840 00000100:00100000:2.0:1787739056.589999:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665587840 00000100:00100000:2.0:1787739056.590019:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665588352 00000100:00100000:2.0:1787739056.590022:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665588352 00000100:00100000:2.0:1787739056.590037:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 652 00000100:00100000:2.0:1787739056.590043:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305618b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1874579665587840:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739056.590088:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305618b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3652:x1874579665587840:12345-0@lo:400:kworker.0 Request processed in 45us (900us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.590094:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 652 00000100:00100000:2.0:1787739056.590105:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 653 00000100:00100000:2.0:1787739056.590111:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1874579665588352:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739056.590137:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3652:x1874579665588352:12345-0@lo:400:kworker.0 Request processed in 27us (318us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.590143:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 653 00000100:00080000:2.0:1787739056.590155:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1787739056.590172:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304000000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3652:1874579665587840:0@lo:400:kworker.0 00000100:00100000:2.0:1787739056.590181:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304001680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3652:1874579665588352:0@lo:400:kworker.0 00000100:00100000:2.0:1787739056.590232:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304000b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3650:1874579665587584:0@lo:400:kworker.0 00000100:00100000:2.0:1787739056.590248:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305618f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665588608:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1787739056.590279:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304002940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3651:1874579665587712:0@lo:400:kworker.0 00000100:00100000:2.0:1787739056.590290:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304001a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3651:1874579665588224:0@lo:400:kworker.0 00000100:00100000:1.0:1787739056.590979:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3653:1874579665587456:0@lo:400:kworker.0 00000100:00100000:3.0:1787739056.592970:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739056.593039:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652436992 00000100:00100000:2.0:1787739056.593047:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652436992 00010000:00010000:2.0:1787739056.593094:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac07d lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c8a expref: 11 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739056.593131:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1426 00000100:00100000:2.0:1787739056.593139:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fe940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2407:x1874579652436992:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739056.593153:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739056.593159:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac07d lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c8a expref: 11 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.593172:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac07d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c8a expref: 11 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.593275:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb9:0x0].0x0 (ffffa07307064200) 00010000:00010000:2.0:1787739056.593280:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac0c3 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.593289:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac0c3 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.593297:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac0c3 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.593316:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739056.593375:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fe940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2407:x1874579652436992:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 236us (411us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.593382:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1426 00010000:00010000:1.0:1787739056.599168:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac07d lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c8a expref: 10 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.599194:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306005680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665587072:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739056.599216:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac0c3 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.599226:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac0c3 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.599231:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac0c3 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.599285:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac0ca lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.599292:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac0ca lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.599303:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac0ca lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.599322:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304002d00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665588736:0@lo:101:mdt00_004.0 00000100:00100000:1.0:1787739056.599381:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.599466:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739056.603236:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665588736 00000100:00100000:2.0:1787739056.603251:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665588736 00000100:00100000:2.0:1787739056.603320:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2761 00000100:00100000:2.0:1787739056.603330:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304003c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665588736:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739056.603350:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739056.603372:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac0d1 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac0ca expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.603403:0:8447:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac08b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c91 expref: 13 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.603449:0:8447:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac0d1 lrc: 4/0,0 mode: --/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac0ca expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.603483:0:8447:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac08b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c91 expref: 13 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.603496:0:8447:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac08b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c91 expref: 13 pid: 6523 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739056.603509:0:8447:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305618f00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8447:1874579665588864:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739056.603912:0:8447:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739056.609980:0:8447:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305618f00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8447:1874579665588864:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739056.610016:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac0d1 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac0ca expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.610036:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb9:0x0].0x0 (ffffa07307064e00) 00010000:00010000:1.0:1787739056.610039:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739056.610043:0:8447:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac0d1 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac0ca expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.610053:0:8447:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac0d1 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac0ca expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.610185:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633cc00, rc 0) 00000100:00100000:1.0:1787739056.610258:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304003c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665588736:12345-0@lo:101:mdt00_004.0 Request processed in 6929us (10876us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.610266:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2761 00000100:00100000:2.0:1787739056.610361:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304002d00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665588736:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739056.610378:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac0ca lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac0d1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.610387:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac0ca lrc: 4/0,1 mode: --/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac0d1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739056.619203:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739056.620230:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652437120 00000100:00100000:1.0:1787739056.620241:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652437120 00010000:00010000:1.0:1787739056.620277:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac08b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c91 expref: 14 pid: 6523 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.620320:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1427 00000100:00100000:1.0:1787739056.620330:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f783c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652437120:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739056.620345:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739056.620351:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac08b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c91 expref: 14 pid: 6523 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.620363:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac08b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c91 expref: 14 pid: 6523 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.620389:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac08b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c91 expref: 14 pid: 6523 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.620402:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb9:0x0].0x0 (ffffa07307064e00) 00010000:00010000:1.0:1787739056.620408:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac0d1 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac0ca expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.620437:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac0d1 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac0ca expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.620446:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac0d1 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac0ca expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.620475:0:10340:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac0d1 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac0ca expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.620488:0:10340:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073040012c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:10340:1874579665588992:0@lo:105: 00000100:00100000:1.0:1787739056.620539:0:10340:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.620583:0:10340:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739056.624167:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665588992 00000100:00100000:0.0:1787739056.624181:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665588992 00000100:00100000:0.0:1787739056.624261:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1090 00000100:00100000:0.0:1787739056.624272:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304000780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:10340:x1874579665588992:12345-0@lo:105: 00010000:00010000:0.0:1787739056.624288:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac0ca lrc: 5/0,1 mode: --/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac0d1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.624301:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac0ca lrc: 5/0,1 mode: --/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac0d1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.624310:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac0ca lrc: 5/0,1 mode: --/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac0d1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.624320:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac0ca lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac0d1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.624329:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac0ca lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac0d1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.624338:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac0ca lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ac0d1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.624361:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07441a1d800) 00000100:00100000:0.0:1787739056.624455:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304000780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:10340:x1874579665588992:12345-0@lo:105: Request processed in 186us (3916us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.624466:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1090 00000100:00100000:0.0:1787739056.624513:0:10340:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073040012c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:10340:1874579665588992:0@lo:105: 00010000:00010000:0.0:1787739056.624530:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739056.624571:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f783c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:2409:x1874579652437120:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 4240us (5378us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.624579:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1427 00010000:00010000:2.0:1787739056.624785:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac0ca lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac0d1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.624796:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac0ca lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac0d1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739056.625432:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073056192c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665589120:0@lo:403:mdt00_004.0 00000100:00100000:2.0:1787739056.625484:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739056.625505:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623df00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739056.625756:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665589120 00000100:00100000:0.0:1787739056.625761:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665589120 00000100:00100000:0.0:1787739056.625804:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2762 00000100:00100000:0.0:1787739056.625813:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665589120:12345-0@lo:403:mdt00_004.0 00000100:00100000:0.0:1787739056.627444:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665589120:12345-0@lo:403:mdt00_004.0 Request processed in 1626us (1953us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.627456:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2762 00000100:00100000:2.0:1787739056.627712:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073056192c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665589120:0@lo:403:mdt00_004.0 00000040:00080000:2.0:1787739056.628572:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].340, 952 off517872 00000040:00080000:2.0:1787739056.628732:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].341, 1032 off566680 00000004:00080000:2.0:1787739056.628738:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730623df00 with version 649 00000004:00080000:1.0:1787739056.628968:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 649 rpc_version 649 00000020:00100000:1.0:1787739056.629003:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739056.629012:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073040003c0 x1874579665589376/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739056.629031:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073040003c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665589376:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739056.629095:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.629134:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e840 going to sleep for 11 seconds 00000100:00100000:2.0:1787739056.629234:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665589376 00000100:00100000:2.0:1787739056.629240:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665589376 00000100:00100000:2.0:1787739056.629287:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2763 00000100:00100000:2.0:1787739056.629295:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304002d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665589376:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739056.629781:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304002d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665589376:12345-0@lo:1000:osp_up0-1.0 Request processed in 485us (684us total) trans 4294983337 rc 0/0 00000100:00100000:2.0:1787739056.629789:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2763 00000100:00100000:1.0:1787739056.643441:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073040003c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665589376:0@lo:1000:osp_up0-1.0 00000004:00200000:3.0:1787739056.643521:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x165:0x0]: returning size 4096 00010000:00010000:3.0:1787739056.643545:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac0c3 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643561:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac0c3 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643569:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac0c3 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643576:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739056.643580:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac0c3 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643588:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac0c3 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643602:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb9:0x0].0x0 (ffffa07307064200) 00010000:00010000:3.0:1787739056.643606:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739056.643610:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac0c3 lrc: 2/0,0 mode: --/EX res: [0x240000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643616:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac0c3 lrc: 0/0,0 mode: --/EX res: [0x240000400:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643630:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac0ca lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac0d1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643637:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac0ca lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac0d1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643643:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac0ca lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac0d1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643676:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739056.643686:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xb9:0x0] 00010000:00010000:3.0:1787739056.643693:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac0bc lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x165:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643699:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac0bc lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x165:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643705:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac0bc lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x165:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643710:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739056.643713:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac0bc lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x165:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643719:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac0bc lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x165:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643726:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x165:0x0].0x0 (ffffa07307064700) 00010000:00010000:3.0:1787739056.643729:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739056.643733:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac0bc lrc: 2/0,0 mode: --/EX res: [0x240000402:0x165:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643738:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac0bc lrc: 0/0,0 mode: --/EX res: [0x240000402:0x165:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643747:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac0ae lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x165:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac0b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643754:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac0ae lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x165:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac0b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643761:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac0ae lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x165:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac0b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643775:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:3.0:1787739056.643784:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739056.643789:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac092 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac099 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643795:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac092 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac099 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643802:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac092 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac099 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643815:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:3.0:1787739056.643819:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739056.643823:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1ca00/0x7ee4453e4a1ac0a0 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac0a7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643830:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1ca00/0x7ee4453e4a1ac0a0 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac0a7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739056.643908:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fd2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17476:x1874579652436864:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 72462us (72627us total) trans 4294968327 rc 0/0 00000100:00100000:3.0:1787739056.643918:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23268 00010000:00010000:3.0:1787739056.643967:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1787739056.643973:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac0ca lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac0d1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.643990:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739056.643994:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac0ca lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac0d1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739056.644003:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac0ca lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac0d1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.644189:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac0ca lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac0d1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.644197:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739056.644219:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac0ca lrc: 1/0,0 mode: --/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac0d1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.644227:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac0ca lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac0d1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.644242:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787739056.644246:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac0ae lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x165:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac0b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.644252:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1cc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739056.644255:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac0ae lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x165:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac0b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739056.644262:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac0ae lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x165:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac0b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.644284:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac0ae lrc: 2/0,0 mode: --/EX res: [0x240000402:0x165:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac0b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.644289:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739056.644302:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac0ae lrc: 1/0,0 mode: --/EX res: [0x240000402:0x165:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac0b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.644309:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ac0ae lrc: 0/0,0 mode: --/EX res: [0x240000402:0x165:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac0b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.644330:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739056.644334:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac092 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac099 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.644339:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1de00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739056.644343:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac092 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac099 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739056.644349:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac092 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac099 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.644366:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac092 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac099 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.644372:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739056.644378:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac092 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac099 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.644389:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac092 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac099 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.644400:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739056.644484:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739056.644487:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739056.644501:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fda40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665589504:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739056.644556:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787739056.644604:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739056.644617:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665589760:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739056.644622:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fe580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665589632:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739056.644659:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739056.644693:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665589504 00000100:00100000:3.0:1787739056.644701:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665589504 00000100:00100000:1.0:1787739056.644705:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665589760 00000100:00100000:1.0:1787739056.644709:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665589760 00000100:00100000:3.0:1787739056.644758:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1228 00000100:00100000:3.0:1787739056.644768:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061ff840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3651:x1874579665589504:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739056.644781:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739056.644798:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac0d1 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac0ca expref: 11 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.644811:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb9:0x0].0x0 (ffffa07307064e00) 00010000:00010000:3.0:1787739056.644821:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739056.644860:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061ff840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+10:3651:x1874579665589504:12345-0@lo:103:ldlm_bl.0 Request processed in 94us (305us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.644868:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1228 00010000:00010000:0.0:1787739056.644930:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739056.645560:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739056.645817:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665589632 00000100:00100000:0.0:1787739056.645823:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665589632 00000100:00100000:1.0:1787739056.647717:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1229 00000100:00100000:1.0:1787739056.647732:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073040012c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3653:x1874579665589760:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739056.647749:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739056.647773:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ac099 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac092 expref: 11 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.647793:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305d56a00) 00010000:00010000:1.0:1787739056.647801:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739056.650343:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073040012c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+9:3653:x1874579665589760:12345-0@lo:103:ldlm_bl.0 Request processed in 2597us (5666us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.650358:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1229 00000100:00100000:1.0:1787739056.650430:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665589760:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739056.657161:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1230 00000100:00100000:0.0:1787739056.657180:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+8:3652:x1874579665589632:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739056.657205:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739056.657231:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ac0b5 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x165:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac0ae expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.657252:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x165:0x0].0x0 (ffffa07433235b00) 00010000:00010000:0.0:1787739056.657263:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739056.657336:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+7:3652:x1874579665589632:12345-0@lo:103:ldlm_bl.0 Request processed in 165us (11777us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.657343:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1230 00000100:00100000:0.0:1787739056.657454:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fe580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665589632:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739056.660360:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fda40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665589504:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739056.686810:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739056.686887:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652437248 00000100:00100000:2.0:1787739056.686896:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652437248 00000100:00100000:2.0:1787739056.686951:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23269 00000100:00100000:2.0:1787739056.686959:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17478:x1874579652437248:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739056.686982:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739056.687001:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac0d8 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094c98 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.687059:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac0df lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.687084:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac0df lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.687096:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac0df lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.687109:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac0e6 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.687117:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac0e6 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.687123:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac0e6 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.687161:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac0df lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.687168:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac0df lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.687173:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac0df lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.687180:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac0df lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.687186:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739056.687191:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac0df lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.687198:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac0df lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.687209:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07433235400) 00010000:00010000:2.0:1787739056.687212:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739056.687216:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac0df lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.687222:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac0df lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.687238:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac0e6 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.687245:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac0e6 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.687250:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac0e6 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.687255:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739056.687258:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac0e6 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.687263:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac0e6 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.687269:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xe928ecd7 (ffffa074419d5f00) 00010000:00010000:2.0:1787739056.687271:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739056.687274:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac0e6 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.687280:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac0e6 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.687309:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac0d8 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c98 expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.687318:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07433235400) 00010000:00010000:2.0:1787739056.687321:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739056.687336:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac0d8 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c98 expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.687345:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633cc00, rc 0) 00000100:00100000:2.0:1787739056.687403:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17478:x1874579652437248:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 444us (603us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739056.687440:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23269 00000100:00100000:0.0:1787739056.700206:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739056.700306:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652437376 00000100:00100000:3.0:1787739056.700315:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652437376 00000100:00100000:3.0:1787739056.700365:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23270 00000100:00100000:3.0:1787739056.700375:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f792c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17478:x1874579652437376:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739056.700394:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739056.700439:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ac0ed lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094c9f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.700495:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac0f4 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.700508:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac0f4 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.700519:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac0f4 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739056.700559:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:3.0:1787739056.700564:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739056.700594:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ac0ed lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c9f expref: 15 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.700611:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac0f4 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c9f expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.700620:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305d56300) 00010000:00010000:3.0:1787739056.700623:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739056.700626:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab400, rc 0) 00000100:00100000:3.0:1787739056.700682:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f792c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17478:x1874579652437376:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 309us (493us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.700689:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23270 00000100:00100000:1.0:1787739056.710680:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739056.710818:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739056.710852:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739056.710877:0:8885:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652437760 00000100:00100000:1.0:1787739056.710885:0:8885:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652437760 00000100:00100000:2.0:1787739056.710914:0:14661:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652437632 00000100:00100000:2.0:1787739056.710919:0:14661:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652437632 00000100:00100000:1.0:1787739056.710941:0:8885:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 346 00000100:00100000:1.0:1787739056.711968:0:8885:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304001a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2406:x1874579652437760:12345-192.168.203.15@tcp:400:kworker.0 00000100:00100000:1.0:1787739056.712053:0:8885:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304001a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2406:x1874579652437760:12345-192.168.203.15@tcp:400:kworker.0 Request processed in 1102us (1234us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.714425:0:14661:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 347 00000100:00100000:2.0:1787739056.714448:0:14661:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304002940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2406:x1874579652437632:12345-192.168.203.15@tcp:400:kworker.0 00000100:00100000:2.0:1787739056.714514:0:14661:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304002940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2406:x1874579652437632:12345-192.168.203.15@tcp:400:kworker.0 Request processed in 76us (3662us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.714522:0:14661:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 347 00000100:00100000:0.0:1787739056.714792:0:6517:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652437504 00000100:00100000:0.0:1787739056.714801:0:6517:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652437504 00000100:00100000:0.0:1787739056.714835:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 447 00000100:00100000:0.0:1787739056.714842:0:6517:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07418bcc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:53427813-ebb7-48fc-94b6-ef03330d7ee5+10:2406:x1874579652437504:12345-192.168.203.15@tcp:400:kworker.0 00000100:00100000:0.0:1787739056.714870:0:6517:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07418bcc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:53427813-ebb7-48fc-94b6-ef03330d7ee5+10:2406:x1874579652437504:12345-192.168.203.15@tcp:400:kworker.0 Request processed in 28us (4201us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.714876:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 447 00000100:00100000:1.0:1787739056.714941:0:8885:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 346 00000100:00080000:1.0:1787739056.714962:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787739056.714967:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787739056.719605:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739056.727623:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652438016 00000100:00100000:0.0:1787739056.727637:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652438016 00000100:00100000:0.0:1787739056.727712:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23271 00000100:00100000:0.0:1787739056.727723:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fe580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17478:x1874579652438016:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:0.0:1787739056.727833:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7b0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665589888:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739056.727901:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739056.727955:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739056.728094:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665589888 00000100:00100000:2.0:1787739056.728098:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665589888 00000100:00100000:2.0:1787739056.728189:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2764 00000100:00100000:2.0:1787739056.728262:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665589888:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739056.728343:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665589888:12345-0@lo:1000:mdt00_004.0 Request processed in 83us (441us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.728350:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2764 00000100:00100000:0.0:1787739056.728440:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7b0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665589888:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739056.728485:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7b0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665590016:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739056.728509:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739056.737224:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665590016 00000100:00100000:3.0:1787739056.737240:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665590016 00000100:00100000:1.0:1787739056.737588:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2765 00000100:00100000:1.0:1787739056.737600:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665590016:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739056.737776:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665590016:12345-0@lo:1000:mdt00_004.0 Request processed in 178us (9265us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739056.737783:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2765 00000100:00100000:0.0:1787739056.740997:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234600 going to sleep for 11 seconds 00000100:00100000:0.0:1787739056.741032:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7b0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665590016:0@lo:1000:mdt00_004.0 00010000:00010000:0.0:1787739056.741120:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ac0fb lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.741131:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ac0fb lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.741143:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ac0fb lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739056.741163:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7b0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665590144:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739056.741212:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739056.741241:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739056.741940:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665590144 00000100:00100000:2.0:1787739056.741949:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665590144 00000100:00100000:2.0:1787739056.742014:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2766 00000100:00100000:2.0:1787739056.742024:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f78b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665590144:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739056.742040:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739056.742058:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac102 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac0fb expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.742099:0:18535:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac00d lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c75 expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.742110:0:18535:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac102 lrc: 4/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac0fb expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.742135:0:18535:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac00d lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c75 expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.742150:0:18535:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac00d lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c75 expref: 9 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739056.742162:0:18535:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073056192c0 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:18535:1874579665590272:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739056.742331:0:18535:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623de40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739056.749617:0:18535:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073056192c0 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:18535:1874579665590272:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739056.749632:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00010000:00010000:2.0:1787739056.749665:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac102 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac0fb expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.749680:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07433235400) 00010000:00010000:2.0:1787739056.749683:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739056.749687:0:18535:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac102 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac0fb expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.749697:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633cc00, rc 0) 00000100:00100000:2.0:1787739056.749766:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f78b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:13793:x1874579665590144:12345-0@lo:101:mdt00_004.0 Request processed in 7744us (8554us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.749775:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2766 00000100:00100000:0.0:1787739056.750236:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7b0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665590144:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739056.750257:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ac0fb lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac102 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.750268:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ac0fb lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac102 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739056.750502:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652438144 00000100:00100000:2.0:1787739056.750508:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652438144 00010000:00010000:2.0:1787739056.750536:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac00d lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c75 expref: 10 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739056.750566:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1428 00000100:00100000:2.0:1787739056.750576:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074301a3c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2409:x1874579652438144:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739056.750592:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739056.750598:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac00d lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c75 expref: 10 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.750611:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac00d lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c75 expref: 10 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.750630:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac00d lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c75 expref: 10 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.750651:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07433235400) 00010000:00010000:2.0:1787739056.750657:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac102 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac0fb expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.750667:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac102 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac0fb expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.750675:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac102 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac0fb expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.750716:0:6508:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac102 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac0fb expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739056.750857:0:6508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073056192c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001_UUID:6508:1874579665590400:0@lo:105: 00000100:00100000:2.0:1787739056.750903:0:6508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739056.750933:0:6508:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623de40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739056.750955:0:15907:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:2.0:1787739056.750965:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665590400 00000100:00100000:2.0:1787739056.750969:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665590400 00000100:00100000:2.0:1787739056.750996:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1091 00000100:00100000:2.0:1787739056.751002:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f804850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6508:x1874579665590400:12345-0@lo:105: 00010000:00010000:2.0:1787739056.751013:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ac0fb lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac102 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.751021:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ac0fb lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac102 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.751034:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ac0fb lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac102 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.751041:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ac0fb lrc: 5/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac102 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.751048:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ac0fb lrc: 6/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac102 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.751056:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ac0fb lrc: 6/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ac102 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.751091:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609ac00) 00010000:00010000:0.0:1787739056.751102:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ac0fb lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac102 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.751113:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ac0fb lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac102 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739056.751126:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f804850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6508:x1874579665590400:12345-0@lo:105: Request processed in 123us (222us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.751134:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1091 00000004:00080000:0.0:1787739056.751141:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739056.751172:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a000/0x7ee4453e4a1ac109 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.751184:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a000/0x7ee4453e4a1ac109 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739056.751189:0:6508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073056192c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001_UUID:6508:1874579665590400:0@lo:105: 00010000:00010000:0.0:1787739056.751196:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a000/0x7ee4453e4a1ac109 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.751201:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739056.751215:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7b0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665590528:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1787739056.751243:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074301a3c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:2409:x1874579652438144:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 668us (1613us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.751251:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739056.751254:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1428 00000100:00100000:0.0:1787739056.751278:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739056.751298:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665590528 00000100:00100000:0.0:1787739056.751315:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665590528 00000100:00100000:0.0:1787739056.751762:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2767 00000100:00100000:0.0:1787739056.751770:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f79a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:13793:x1874579665590528:12345-0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739056.751785:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739056.751808:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac110 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac109 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.751827:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac110 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac109 expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.751839:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac110 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac109 expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.751846:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xe928ecd7 (ffffa073068a9f00) 00010000:00010000:0.0:1787739056.751849:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739056.751852:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a400, rc 0) 00000100:00100000:0.0:1787739056.751891:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f79a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665590528:12345-0@lo:101:mdt00_004.0 Request processed in 122us (642us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.751898:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2767 00000100:00100000:1.0:1787739056.751970:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7b0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665590528:0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739056.751984:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a000/0x7ee4453e4a1ac109 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac110 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.751993:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a000/0x7ee4453e4a1ac109 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac110 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739056.752006:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:1.0:1787739056.752037:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304000b40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665590656:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739056.752078:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739056.759151:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665590656 00000100:00100000:2.0:1787739056.759167:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665590656 00000100:00100000:1.0:1787739056.759192:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e900 going to sleep for 11 seconds 00000100:00100000:2.0:1787739056.759235:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2768 00000100:00100000:2.0:1787739056.759246:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304001680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665590656:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739056.759661:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304001680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665590656:12345-0@lo:1000:mdt00_004.0 Request processed in 416us (7580us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.765205:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304000b40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665590656:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739056.765285:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665590784:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739056.765356:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739056.765386:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739056.765433:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665590784 00000100:00100000:0.0:1787739056.765439:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665590784 00000100:00100000:2.0:1787739056.767624:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2768 00000100:00100000:0.0:1787739056.768083:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2769 00000100:00100000:0.0:1787739056.768102:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665590784:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739056.768241:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665590784:12345-0@lo:1000:mdt00_004.0 Request processed in 146us (2883us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739056.768250:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2769 00000100:00100000:0.0:1787739056.768301:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665590784:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739056.768456:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665590912:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739056.768497:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739056.768524:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665590912 00000100:00100000:0.0:1787739056.768527:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739056.768531:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665590912 00000100:00100000:3.0:1787739056.768570:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2770 00000100:00100000:3.0:1787739056.768578:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665590912:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739056.768659:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665590912:12345-0@lo:1000:mdt00_004.0 Request processed in 82us (162us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739056.768666:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2770 00000100:00100000:1.0:1787739056.769989:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665590912:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739056.770077:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304000000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665591040:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739056.770126:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.770160:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739056.770439:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665591040 00000100:00100000:0.0:1787739056.770445:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665591040 00000100:00100000:0.0:1787739056.770487:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2771 00000100:00100000:0.0:1787739056.770496:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304000b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665591040:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739056.770567:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304000b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665591040:12345-0@lo:1000:mdt00_004.0 Request processed in 71us (440us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.770574:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2771 00000100:00100000:1.0:1787739056.770717:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304000000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665591040:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739056.770953:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304000000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665591168:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739056.770983:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.771009:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739056.771457:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665591168 00000100:00100000:0.0:1787739056.771464:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665591168 00000100:00100000:0.0:1787739056.771503:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2772 00000100:00100000:0.0:1787739056.771510:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073040012c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665591168:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739056.771599:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073040012c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665591168:12345-0@lo:1000:mdt00_004.0 Request processed in 89us (615us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.771607:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2772 00000100:00100000:1.0:1787739056.771745:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304000000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665591168:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739056.771791:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304000000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665591296:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739056.771831:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739056.772080:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665591296 00000100:00100000:0.0:1787739056.772085:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665591296 00000100:00100000:0.0:1787739056.772110:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2773 00000100:00100000:0.0:1787739056.772115:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304000780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665591296:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739056.777136:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304000780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665591296:12345-0@lo:1000:mdt00_004.0 Request processed in 5009us (5293us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.777152:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2773 00000100:00100000:1.0:1787739056.777282:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e900 going to sleep for 11 seconds 00000100:00100000:1.0:1787739056.777317:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304000000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665591296:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739056.777394:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304000000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665591424:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739056.777595:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.777613:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739056.777692:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665591424 00000100:00100000:0.0:1787739056.777699:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665591424 00000100:00100000:0.0:1787739056.777757:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2774 00000100:00100000:0.0:1787739056.777767:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665591424:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739056.777854:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665591424:12345-0@lo:1000:mdt00_004.0 Request processed in 88us (258us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.777860:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2774 00000100:00100000:1.0:1787739056.778104:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304000000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665591424:0@lo:1000:mdt00_004.0 00000040:00080000:1.0:1787739056.779246:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].354, 2168 off543736 00000040:00080000:1.0:1787739056.779330:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].353, 2368 off588008 00000004:00080000:1.0:1787739056.779336:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730610e840 with version 675 00000004:00080000:3.0:1787739056.780229:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 675 rpc_version 675 00000020:00100000:3.0:1787739056.780274:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739056.780284:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073061fed00 x1874579665591680/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739056.780306:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fed00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665591680:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739056.780373:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739056.780434:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89780 going to sleep for 11 seconds 00000100:00100000:1.0:1787739056.781826:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665591680 00000100:00100000:1.0:1787739056.781837:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665591680 00000100:00100000:1.0:1787739056.781889:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2775 00000100:00100000:1.0:1787739056.781899:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665591680:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739056.785035:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:7740:x1874579665591680:12345-0@lo:1000:osp_up1-0.0 Request processed in 3128us (4652us total) trans 4294968328 rc 0/0 00000100:00100000:1.0:1787739056.785049:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2775 00000100:00100000:3.0:1787739056.788189:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fed00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665591680:0@lo:1000:osp_up1-0.0 00010000:00010000:1.0:1787739056.788489:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac117 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1503:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.788507:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac117 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1503:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.788523:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac117 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1503:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.788543:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac11e lrc: 3/0,1 mode: --/PW res: [0x200000400:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.788550:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac11e lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.788557:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac11e lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.788592:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1cc00/0x7ee4453e4a1ac125 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.788599:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1cc00/0x7ee4453e4a1ac125 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.788609:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1cc00/0x7ee4453e4a1ac125 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.788629:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665591808:0@lo:101:mdt00_004.0 00000100:00100000:1.0:1787739056.788685:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.788892:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054180 going to sleep for 11 seconds 00000100:00100000:3.0:1787739056.788957:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665591808 00000100:00100000:3.0:1787739056.788964:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665591808 00000100:00100000:3.0:1787739056.789012:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2776 00000100:00100000:3.0:1787739056.789020:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073025283c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:13793:x1874579665591808:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739056.789038:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739056.789054:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ac12c lrc: 2/0,0 mode: --/PW res: [0x240000401:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac125 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.789087:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ac12c lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac125 expref: 12 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.789102:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ac12c lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac125 expref: 12 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.789110:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xbc:0x0].0x0 (ffffa07305d56400) 00010000:00010000:3.0:1787739056.789113:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739056.789116:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa800, rc 0) 00000100:00100000:3.0:1787739056.789162:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073025283c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:13793:x1874579665591808:12345-0@lo:101:mdt00_004.0 Request processed in 142us (476us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.789168:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2776 00000100:00100000:1.0:1787739056.789290:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665591808:0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739056.789304:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1cc00/0x7ee4453e4a1ac125 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac12c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.789313:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1cc00/0x7ee4453e4a1ac125 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac12c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739056.789336:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005173619e reply state 000000004f7b48b1 transno 4294983338 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac11e lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.789347:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac11e lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.789353:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xbc:0x0].0x0 (ffffa07433235c00) 00010000:00010000:1.0:1787739056.789355:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739056.789358:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787739056.789363:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1cc00/0x7ee4453e4a1ac125 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac12c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.789369:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1cc00/0x7ee4453e4a1ac125 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac12c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739056.789379:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xbc:0x0] 00000004:00010000:1.0:1787739056.789384:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005173619e reply state 000000004f7b48b1 transno 4294983338 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac117 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1503:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.789391:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac117 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1503:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.789408:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1503:0x0].0x0 (ffffa07433235300) 00010000:00010000:1.0:1787739056.789410:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739056.789437:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:1.0:1787739056.789443:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1503:0x0]: returning size 4096 00000004:00080000:1.0:1787739056.789451:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739056.789460:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ac0fb lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac102 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.789472:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ac0fb lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac102 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.789479:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ac0fb lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac102 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.789498:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739056.789503:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739056.789507:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a000/0x7ee4453e4a1ac109 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac110 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.789513:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a000/0x7ee4453e4a1ac109 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac110 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.789591:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fe580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17478:x1874579652438016:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 61868us (69994us total) trans 4294983338 rc 0/0 00000100:00100000:1.0:1787739056.789603:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23271 00010000:00010000:3.0:1787739056.789778:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739056.789784:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ac0fb lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac102 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.789792:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ac00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739056.789857:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ac0fb lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac102 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739056.789868:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ac0fb lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac102 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.789895:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ac0fb lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac102 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.789902:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739056.789932:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ac0fb lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac102 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.790014:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fc3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665591936:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739056.790048:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.790098:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665591936 00000100:00100000:1.0:1787739056.790102:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665591936 00000100:00100000:1.0:1787739056.790135:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1231 00000100:00100000:1.0:1787739056.790143:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+12:3650:x1874579665591936:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739056.790152:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739056.790168:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac102 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac0fb expref: 12 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.790187:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07433235400) 00010000:00010000:1.0:1787739056.790194:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739056.790219:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+11:3650:x1874579665591936:12345-0@lo:103:ldlm_bl.0 Request processed in 78us (171us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.790225:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1231 00000100:00100000:1.0:1787739056.790254:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fc3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665591936:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739056.790299:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ac0fb lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac102 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.790313:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739056.790315:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739056.790316:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739056.823208:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739056.823249:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652438400 00000100:00100000:0.0:1787739056.823258:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652438400 00000100:00100000:0.0:1787739056.823575:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23272 00000100:00100000:0.0:1787739056.823588:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17487:x1874579652438400:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:0.0:1787739056.823905:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17487:x1874579652438400:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 319us (705us total) trans 0 rc -17/-17 00000100:00100000:0.0:1787739056.823916:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23272 00000100:00100000:2.0:1787739056.843581:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739056.843872:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652438528 00000100:00100000:3.0:1787739056.843881:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652438528 00000100:00100000:3.0:1787739056.844388:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23273 00000100:00100000:3.0:1787739056.844401:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073056192c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17488:x1874579652438528:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739056.844450:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739056.844474:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ac133 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094ca6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.844540:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac13a lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.844554:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac13a lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.844565:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac13a lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.844574:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ac141 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.844584:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac110 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac109 expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.844597:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ac141 lrc: 4/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.844629:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac110 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ac109 expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739056.844643:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fc780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665592064:0@lo:104: 00000100:00100000:3.0:1787739056.844694:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739056.846120:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89780 going to sleep for 11 seconds 00000100:00100000:1.0:1787739056.851015:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665592064 00000100:00100000:1.0:1787739056.851030:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665592064 00000100:00100000:1.0:1787739056.851099:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1092 00000100:00100000:1.0:1787739056.851108:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665592064:12345-0@lo:104: 00010000:00010000:1.0:1787739056.851127:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a000/0x7ee4453e4a1ac109 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac110 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.851238:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787739056.851245:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665592064:12345-0@lo:104: Request processed in 138us (6551us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.851252:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1092 00000100:00100000:2.0:1787739056.851274:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fc780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665592064:0@lo:104: 00010000:00010000:2.0:1787739056.851296:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ac141 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.851344:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739056.851352:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a000/0x7ee4453e4a1ac109 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac110 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.851361:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739056.851366:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a000/0x7ee4453e4a1ac109 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ac110 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739056.851375:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a000/0x7ee4453e4a1ac109 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ac110 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.851409:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a000/0x7ee4453e4a1ac109 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac110 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.851453:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739056.851473:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a000/0x7ee4453e4a1ac109 lrc: 1/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac110 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.851481:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a000/0x7ee4453e4a1ac109 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac110 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.851501:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739056.851503:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739056.851505:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739056.851642:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fe940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665592192:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739056.851688:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739056.851745:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665592192 00000100:00100000:2.0:1787739056.851750:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665592192 00010000:00010000:2.0:1787739056.851769:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac110 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ac109 expref: 11 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739056.851793:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1429 00000100:00100000:2.0:1787739056.851812:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561b840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+11:3651:x1874579665592192:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739056.851824:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739056.857142:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1ca00/0x7ee4453e4a1ac0a0 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac0a7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739056.857163:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1ca00/0x7ee4453e4a1ac0a0 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac0a7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.857210:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1ca00/0x7ee4453e4a1ac0a0 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac0a7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.857218:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739056.857257:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1ca00/0x7ee4453e4a1ac0a0 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac0a7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787739056.857278:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968328 is committed 00000001:00080000:2.0:1787739056.857296:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968327 is committed 00000020:00080000:2.0:1787739056.857316:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 345 batchid updated 00000020:00080000:2.0:1787739056.857337:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa074154fc180 refcount 2 committed 0 result 0 batchid 346 00000020:00080000:2.0:1787739056.857341:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730424df00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07307054480 00000020:00080000:2.0:1787739056.857344:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].340 00000020:00080000:2.0:1787739056.857347:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730424d500 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa0730541e400 00000020:00080000:2.0:1787739056.857349:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].339 00010000:00010000:2.0:1787739056.857376:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac030 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x165:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.857385:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac030 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x165:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.857391:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac030 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x165:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.857397:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609aa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739056.857401:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac030 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x165:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.857408:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac030 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x165:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.857493:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x165:0x0].0x0 (ffffa07307064700) 00010000:00010000:2.0:1787739056.857496:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739056.857502:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac030 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x165:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.857511:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac030 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x165:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.857526:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac037 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.857532:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac037 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.857537:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac037 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.857542:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609be00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739056.857545:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac037 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.857550:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac037 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.857557:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xb9:0x0].0x0 (ffffa07307064200) 00010000:00010000:2.0:1787739056.857559:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739056.857562:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac037 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.857567:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac037 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739056.857591:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 346 committed batchid 345 00000004:00080000:2.0:1787739056.857699:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730623dc00 with version 650 00000020:00080000:2.0:1787739056.857722:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 345 cancel update log [0x200000401:0x3:0x0].339: rc = 0 00000020:00080000:2.0:1787739056.857764:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 345 cancel update log [0x240000400:0x3:0x0].338: rc = 0 00000020:00080000:2.0:1787739056.857773:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 345 00000004:00080000:2.0:1787739056.857782:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 650 rpc_version 650 00000020:00100000:2.0:1787739056.857814:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739056.857825:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0730561a1c0 x1874579665592576/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739056.857845:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561a1c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665592576:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739056.857917:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.857949:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787739056.857962:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305618f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665592320:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:2.0:1787739056.857963:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d840 going to sleep for 11 seconds 00000100:00100000:1.0:1787739056.857992:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787739056.858017:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac110 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ac109 expref: 7 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.858022:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665592576 00010000:00010000:2.0:1787739056.858027:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xe928ecd7 (ffffa073068a9f00) 00000100:00100000:1.0:1787739056.858029:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665592576 00010000:00010000:2.0:1787739056.858031:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ac141 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.858040:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ac141 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.858046:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ac141 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.858079:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739056.858082:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2777 00000100:00100000:1.0:1787739056.858090:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665592576:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739056.858121:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561b840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+6:3651:x1874579665592192:12345-0@lo:103:ldlm_bl.0 Request processed in 6322us (6432us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.858129:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1429 00000100:00100000:2.0:1787739056.858147:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665592320 00000100:00100000:2.0:1787739056.858150:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665592320 00000100:00100000:2.0:1787739056.858176:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1232 00000100:00100000:2.0:1787739056.858183:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+9:3653:x1874579665592320:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:2.0:1787739056.858193:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739056.858208:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac0a7 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd431a204 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac0a0 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.858218:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665592576:12345-0@lo:1000:osp_up0-1.0 Request processed in 128us (302us total) trans 4294983339 rc 0/0 00010000:00010000:2.0:1787739056.858220:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd431a204 (ffffa07433235800) 00000100:00100000:1.0:1787739056.858225:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2777 00010000:00010000:2.0:1787739056.858227:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739056.858253:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+7:3653:x1874579665592320:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 71us (260us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.858259:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1232 00000100:00100000:2.0:1787739056.858301:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305618f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665592320:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:2.0:1787739056.858332:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fe940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665592192:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739056.858364:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561a1c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665592576:0@lo:1000:osp_up0-1.0 00010000:00010000:2.0:1787739056.858471:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ac141 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.858478:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ac141 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739056.858568:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305618f00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665592704:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739056.858603:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739056.858629:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d780 going to sleep for 11 seconds 00000100:00100000:1.0:1787739056.858717:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665592704 00000100:00100000:1.0:1787739056.858722:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665592704 00000100:00100000:1.0:1787739056.858756:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2778 00000100:00100000:1.0:1787739056.858763:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f7c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665592704:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739056.858833:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f7c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665592704:12345-0@lo:1000:mdt00_003.0 Request processed in 70us (229us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.858839:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2778 00000100:00100000:2.0:1787739056.859535:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305618f00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665592704:0@lo:1000:mdt00_003.0 00010000:00010000:2.0:1787739056.859586:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac148 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1503:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.859602:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac148 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1503:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.859610:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac148 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1503:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739056.859629:0:7944:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac148 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1503:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.859642:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac13a lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.859648:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac13a lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.859654:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac13a lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.859661:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac13a lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.859666:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739056.859670:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac13a lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.859676:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac13a lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.859687:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305d56f00) 00010000:00010000:2.0:1787739056.859689:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739056.859693:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac13a lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.859700:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac13a lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.859781:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ac141 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.859788:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ac141 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.859793:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ac141 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.859856:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739056.859859:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ac141 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.859865:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ac141 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.859874:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xe928ecd7 (ffffa073068a9f00) 00010000:00010000:2.0:1787739056.859876:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739056.859880:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ac141 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.859886:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ac141 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.859925:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ac133 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ca6 expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.861558:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac148 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1503:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ca6 expref: 7 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.861575:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1503:0x0].0x0 (ffffa07303c78800) 00010000:00010000:2.0:1787739056.861578:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739056.861582:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c200, rc 0) 00000100:00100000:2.0:1787739056.861661:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073056192c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17488:x1874579652438528:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 17261us (18087us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.861670:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23273 00000100:00100000:0.0:1787739056.880222:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739056.885378:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652438656 00000100:00100000:2.0:1787739056.885397:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652438656 00000100:00100000:2.0:1787739056.885485:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23274 00000100:00100000:2.0:1787739056.885495:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17488:x1874579652438656:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739056.885687:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739056.885709:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac14f lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1503:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094cad expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.885770:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac156 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1503:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.885783:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac156 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1503:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.885794:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac156 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1503:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739056.886313:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1503:0x0]: returning size 4096 00000004:00200000:2.0:1787739056.886318:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739056.886361:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac14f lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1503:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cad expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.886386:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac156 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1503:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cad expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.886395:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1503:0x0].0x0 (ffffa07433235300) 00010000:00010000:2.0:1787739056.886397:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739056.886401:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633dc00, rc 0) 00000100:00100000:2.0:1787739056.886506:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17488:x1874579652438656:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 1011us (6289us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.886514:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23274 00000100:00100000:0.0:1787739056.894755:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739056.894918:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652438784 00000100:00100000:3.0:1787739056.894927:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652438784 00000100:00100000:3.0:1787739056.894984:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23275 00000100:00100000:3.0:1787739056.894995:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ee940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17488:x1874579652438784:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739056.895014:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739056.895033:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac15d lrc: 2/0,0 mode: --/CR res: [0x200000400:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094cb4 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.895096:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac164 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.895111:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac164 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.895122:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac164 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739056.895172:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xbc:0x0]: returning size 4096 00000004:00200000:3.0:1787739056.895178:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739056.895216:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac15d lrc: 0/0,0 mode: --/CR res: [0x200000400:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cb4 expref: 19 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.895234:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac164 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cb4 expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.895247:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xbc:0x0].0x0 (ffffa07433235c00) 00010000:00010000:3.0:1787739056.895250:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739056.895254:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab000, rc 0) 00000100:00100000:3.0:1787739056.895315:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ee940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17488:x1874579652438784:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 321us (566us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.895323:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23275 00000100:00100000:0.0:1787739056.903678:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739056.903923:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652438912 00000100:00100000:2.0:1787739056.903934:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652438912 00000100:00100000:2.0:1787739056.903995:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23276 00000100:00100000:2.0:1787739056.904006:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17488:x1874579652438912:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739056.904026:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739056.904046:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac16b lrc: 2/0,0 mode: --/CR res: [0x240000401:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094cbb expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.904120:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac172 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.904136:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ac12c lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac125 expref: 5 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.904144:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac172 lrc: 4/1,0 mode: --/PR res: [0x240000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.904177:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ac12c lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ac125 expref: 5 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739056.904193:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305618f00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665592832:0@lo:104: 00000100:00100000:2.0:1787739056.904243:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739056.904285:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d840 going to sleep for 11 seconds 00000100:00100000:3.0:1787739056.905056:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665592832 00000100:00100000:3.0:1787739056.905074:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665592832 00000100:00100000:3.0:1787739056.905179:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1093 00000100:00100000:3.0:1787739056.905191:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060043c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665592832:12345-0@lo:104: 00010000:00010000:3.0:1787739056.905207:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1cc00/0x7ee4453e4a1ac125 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac12c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.905305:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739056.905312:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060043c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665592832:12345-0@lo:104: Request processed in 125us (1068us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.905320:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1093 00000100:00100000:0.0:1787739056.905398:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305618f00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665592832:0@lo:104: 00010000:00010000:0.0:1787739056.905443:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac172 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.905811:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739056.905819:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1cc00/0x7ee4453e4a1ac125 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac12c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.905828:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1cc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739056.905833:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1cc00/0x7ee4453e4a1ac125 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xbc:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ac12c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739056.905842:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1cc00/0x7ee4453e4a1ac125 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xbc:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ac12c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.905876:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1cc00/0x7ee4453e4a1ac125 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xbc:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac12c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.905883:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739056.905914:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1cc00/0x7ee4453e4a1ac125 lrc: 1/0,0 mode: --/PW res: [0x240000401:0xbc:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac12c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.905922:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1cc00/0x7ee4453e4a1ac125 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xbc:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac12c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.905949:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739056.905951:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739056.905952:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739056.905960:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787739056.905973:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061ff840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665592960:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739056.906014:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739056.906055:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665592960 00000100:00100000:3.0:1787739056.906071:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665592960 00010000:00010000:3.0:1787739056.906091:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ac12c lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ac125 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739056.906115:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1430 00000100:00100000:3.0:1787739056.906121:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fe940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+6:3652:x1874579665592960:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739056.906133:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00080000:0.0:1787739056.908384:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 346 batchid updated 00010000:00010000:2.0:1787739056.908555:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ac12c lrc: 0/0,0 mode: --/PW res: [0x240000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ac125 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.908576:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xbc:0x0].0x0 (ffffa07305d56400) 00010000:00010000:2.0:1787739056.908581:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac172 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.908590:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac172 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.908597:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac172 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.908630:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739056.908701:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fe940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+5:3652:x1874579665592960:12345-0@lo:103:ldlm_bl.0 Request processed in 2579us (2685us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.908778:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1430 00010000:00010000:0.0:1787739056.908883:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac172 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.908893:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac172 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739056.909023:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xbc:0x0]: returning size 4096 00000004:00200000:0.0:1787739056.909028:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739056.909081:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac16b lrc: 0/0,0 mode: --/CR res: [0x240000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cbb expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.909100:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac172 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cbb expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.909108:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xbc:0x0].0x0 (ffffa07305d56400) 00010000:00010000:0.0:1787739056.909111:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739056.909113:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c600, rc 0) 00000100:00100000:0.0:1787739056.909162:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ed2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17488:x1874579652438912:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 5158us (5494us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.909174:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23276 00000020:00080000:3.0:1787739056.909498:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 346 00000004:00080000:3.0:1787739056.909550:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07302e89000 with version 651 00000020:00080000:3.0:1787739056.909562:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 346 cancel update log [0x200000401:0x3:0x0].340: rc = 0 00000020:00080000:3.0:1787739056.909625:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 346 cancel update log [0x240000400:0x3:0x0].339: rc = 0 00000020:00080000:3.0:1787739056.909640:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 346 00000100:00100000:3.0:1787739056.909683:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061ff840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665592960:0@lo:103:ldlm_bl.0 00000004:00080000:2.0:1787739056.909859:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 651 rpc_version 651 00000020:00100000:2.0:1787739056.909889:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739056.909897:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306005e00 x1874579665593216/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739056.909916:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306005e00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665593216:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739056.910029:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739056.910069:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d780 going to sleep for 11 seconds 00000100:00100000:0.0:1787739056.910128:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665593216 00000100:00100000:0.0:1787739056.910134:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665593216 00000100:00100000:0.0:1787739056.910180:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2779 00000100:00100000:0.0:1787739056.910188:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306005680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665593216:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739056.910392:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306005680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665593216:12345-0@lo:1000:osp_up0-1.0 Request processed in 204us (363us total) trans 4294983340 rc 0/0 00000100:00100000:0.0:1787739056.910399:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2779 00000100:00100000:3.0:1787739056.910618:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306005e00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665593216:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739056.919732:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739056.919782:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739056.919857:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652439296 00000100:00100000:2.0:1787739056.919880:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652439296 00000100:00100000:2.0:1787739056.919937:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23277 00000100:00100000:2.0:1787739056.919948:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fc780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17488:x1874579652439296:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:2.0:1787739056.920003:0:13793:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739056.920029:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac156 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1503:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cad expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.920050:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1503:0x0].0x0 (ffffa07433235300) 00010000:00010000:2.0:1787739056.920055:0:13793:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739056.920110:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ac179 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.920120:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ac179 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.920131:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ac179 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739056.920152:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665593344:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1787739056.920199:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739056.920234:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739056.920281:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665593344 00000100:00100000:3.0:1787739056.920286:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665593344 00000100:00100000:3.0:1787739056.920325:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2780 00000100:00100000:3.0:1787739056.920332:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+5:13793:x1874579665593344:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739056.920342:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739056.920358:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac180 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac179 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.920380:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac180 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac179 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.920402:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac180 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac179 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.920429:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305d56500) 00010000:00010000:3.0:1787739056.920432:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739056.920436:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abe00, rc 0) 00000100:00100000:3.0:1787739056.920486:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:13793:x1874579665593344:12345-0@lo:101:mdt00_004.0 Request processed in 154us (286us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739056.920493:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2780 00000100:00100000:2.0:1787739056.920583:0:13793:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968328 gen 2 00000004:00080000:2.0:1787739056.920589:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968323 committed transno 4294968328 00000020:00080000:2.0:1787739056.920595:0:13793:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa074062f3540 refcount 2 committed 0 result 0 batchid 355 00000020:00080000:2.0:1787739056.920600:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730415f900 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa074062f3b40 00000020:00080000:2.0:1787739056.920604:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].352 00000020:00080000:2.0:1787739056.920606:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730415ff00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304011200 00000020:00080000:2.0:1787739056.920611:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].353 00000004:00080000:2.0:1787739056.920633:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968325 committed transno 4294968328 00000004:00080000:2.0:1787739056.920643:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968326 committed transno 4294968328 00000004:00080000:2.0:1787739056.920654:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968328 committed transno 4294968328 00000100:00100000:2.0:1787739056.920670:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665593344:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739056.920686:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ac179 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac180 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.920695:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ac179 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac180 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739056.920709:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:2.0:1787739056.920749:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ac187 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.920756:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ac187 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.920763:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ac187 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739056.920775:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665593472:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1787739056.920798:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739056.920826:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739056.920893:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665593472 00000100:00100000:0.0:1787739056.920898:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665593472 00000020:00080000:3.0:1787739056.920924:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 355 committed batchid 354 00000100:00100000:0.0:1787739056.920936:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2781 00000100:00100000:0.0:1787739056.920945:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306007c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:13793:x1874579665593472:12345-0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739056.920953:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739056.920968:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac18e lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac187 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.920985:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac18e lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac187 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.920996:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac18e lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac187 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1787739056.920998:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 354 00010000:00010000:0.0:1787739056.921007:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xe928ecd7 (ffffa073068a9f00) 00010000:00010000:0.0:1787739056.921009:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739056.921012:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609bc00, rc 0) 00000100:00100000:0.0:1787739056.921046:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306007c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665593472:12345-0@lo:101:mdt00_004.0 Request processed in 103us (246us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.921054:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2781 00000100:00100000:1.0:1787739056.921391:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652439040 00000100:00100000:1.0:1787739056.921397:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652439040 00000100:00100000:1.0:1787739056.921456:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1233 00000100:00100000:1.0:1787739056.921465:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061ff840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2409:x1874579652439040:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:1.0:1787739056.921475:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739056.921490:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac148 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1503:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ca6 expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.921504:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1503:0x0].0x0 (ffffa07303c78800) 00010000:00010000:1.0:1787739056.921512:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739056.921855:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665593472:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739056.921868:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ac187 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac18e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.921877:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ac187 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac18e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739056.921890:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:2.0:1787739056.921923:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665593600:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739056.921961:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.921961:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061ff840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2409:x1874579652439040:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 494us (2234us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.921972:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1233 00000100:00100000:2.0:1787739056.921981:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739056.922001:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665593600 00000100:00100000:2.0:1787739056.922004:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665593600 00000100:00100000:2.0:1787739056.922038:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2782 00000100:00100000:2.0:1787739056.922045:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665593600:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739056.922124:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665593600:12345-0@lo:1000:mdt00_004.0 Request processed in 78us (164us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.922131:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2782 00000100:00100000:2.0:1787739056.922159:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665593600:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739056.922193:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304261e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665593728:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739056.922216:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739056.922238:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d600 going to sleep for 11 seconds 00000100:00100000:0.0:1787739056.922643:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665593728 00000100:00100000:0.0:1787739056.922649:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665593728 00000100:00100000:0.0:1787739056.922678:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2783 00000100:00100000:0.0:1787739056.922685:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042630c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665593728:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739056.922769:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042630c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665593728:12345-0@lo:1000:mdt00_004.0 Request processed in 84us (551us total) trans 0 rc 1/1 00000100:00100000:0.0:1787739056.922776:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2783 00000100:00100000:1.0:1787739056.923009:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304261e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665593728:0@lo:1000:mdt00_004.0 00010000:00010000:1.0:1787739056.923068:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac195 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.923078:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac195 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.923088:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac195 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.923102:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665593856:0@lo:101:mdt00_004.0 00000100:00100000:1.0:1787739056.923130:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.923197:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739056.926860:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665593856 00000100:00100000:2.0:1787739056.926923:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665593856 00000100:00100000:2.0:1787739056.927016:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2784 00000100:00100000:2.0:1787739056.927028:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665593856:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739056.927048:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739056.927082:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac19c lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac195 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.927115:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac19c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac195 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.927137:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac19c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac195 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.927146:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1503:0x0].0x0 (ffffa07303c78c00) 00010000:00010000:2.0:1787739056.927149:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739056.927153:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633da00, rc 0) 00000100:00100000:2.0:1787739056.927242:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665593856:12345-0@lo:101:mdt00_004.0 Request processed in 217us (4110us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.927249:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2784 00000100:00100000:0.0:1787739056.927325:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665593856:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739056.927346:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac195 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac19c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.927357:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac195 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac19c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.927386:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac1a3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.927396:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac1a3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.927405:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac1a3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.927443:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac1aa lrc: 3/0,1 mode: --/EX res: [0x200000400:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.927454:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac164 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cb4 expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.927462:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac1aa lrc: 4/0,1 mode: --/EX res: [0x200000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.927491:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac164 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cb4 expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739056.927504:0:13793:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac164 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cb4 expref: 17 pid: 6523 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739056.927528:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665593984:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739056.927596:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739056.951867:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739056.951939:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665593984:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739056.951969:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac1aa lrc: 3/0,1 mode: --/EX res: [0x200000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739056.952658:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652439424 00000100:00100000:2.0:1787739056.952666:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652439424 00010000:00010000:2.0:1787739056.952704:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac164 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cb4 expref: 18 pid: 6523 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739056.952742:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1431 00000100:00100000:2.0:1787739056.952751:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2409:x1874579652439424:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739056.952767:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739056.952775:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac164 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cb4 expref: 18 pid: 6523 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.952788:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac164 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cb4 expref: 18 pid: 6523 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.952810:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac164 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cb4 expref: 18 pid: 6523 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.952833:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xbc:0x0].0x0 (ffffa07433235c00) 00010000:00010000:2.0:1787739056.952838:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac1aa lrc: 3/0,1 mode: --/EX res: [0x200000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.952847:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac1aa lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.952855:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac1aa lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.953031:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739056.953102:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652439424:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 351us (1243us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.953111:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1431 00010000:00010000:1.0:1787739056.953167:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac1aa lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.953175:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac1aa lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.953219:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ac1b1 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.953226:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ac1b1 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.953235:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ac1b1 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.953252:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665594112:0@lo:101:mdt00_004.0 00000100:00100000:1.0:1787739056.953296:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.953331:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739056.953696:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665594112 00000100:00100000:2.0:1787739056.953702:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665594112 00000100:00100000:2.0:1787739056.953742:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2785 00000100:00100000:2.0:1787739056.953749:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665594112:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739056.953765:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739056.953780:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac1b8 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac1b1 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.953801:0:18535:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac172 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cbb expref: 6 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.953811:0:18535:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac1b8 lrc: 4/0,0 mode: --/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac1b1 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.953839:0:18535:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac172 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cbb expref: 6 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.953852:0:18535:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac172 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cbb expref: 6 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739056.953964:0:18535:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304263840 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:18535:1874579665594240:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739056.954078:0:18535:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d780 going to sleep for 11 seconds 00000100:00100000:1.0:1787739056.965073:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739056.965171:0:18535:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304263840 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:18535:1874579665594240:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739056.965205:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac1b8 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac1b1 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.965219:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xbc:0x0].0x0 (ffffa07305d56400) 00010000:00010000:2.0:1787739056.965222:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739056.965226:0:18535:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac1b8 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac1b1 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.965237:0:18535:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac1b8 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac1b1 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739056.965245:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633dc00, rc 0) 00000100:00100000:2.0:1787739056.965314:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:13793:x1874579665594112:12345-0@lo:101:mdt00_004.0 Request processed in 11565us (12017us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.965324:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2785 00000100:00100000:3.0:1787739056.965689:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652439552 00000100:00100000:3.0:1787739056.965697:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652439552 00010000:00010000:3.0:1787739056.965729:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac172 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cbb expref: 7 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739056.965760:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1432 00000100:00100000:3.0:1787739056.965769:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074301a3850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652439552:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739056.965783:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739056.965790:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac172 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cbb expref: 7 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.965801:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac172 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cbb expref: 7 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.965836:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac172 lrc: 0/0,0 mode: --/PR res: [0x240000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cbb expref: 7 pid: 7944 timeout: 743 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.965850:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xbc:0x0].0x0 (ffffa07305d56400) 00010000:00010000:3.0:1787739056.965854:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac1b8 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac1b1 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.965863:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac1b8 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac1b1 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.965871:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac1b8 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac1b1 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.965901:0:6509:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac1b8 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac1b1 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739056.965914:0:6509:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fcb40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001_UUID:6509:1874579665594368:0@lo:105: 00000100:00100000:3.0:1787739056.965954:0:6509:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739056.965994:0:6509:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89780 going to sleep for 11 seconds 00000100:00100000:1.0:1787739056.966182:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665594368 00000100:00100000:1.0:1787739056.966187:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665594368 00000100:00100000:1.0:1787739056.966225:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1094 00000100:00100000:1.0:1787739056.966232:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6509:x1874579665594368:12345-0@lo:105: 00010000:00010000:1.0:1787739056.966243:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ac1b1 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac1b8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.966253:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ac1b1 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac1b8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.966260:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ac1b1 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac1b8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.966267:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ac1b1 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac1b8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.966273:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ac1b1 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac1b8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.966279:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ac1b1 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ac1b8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.966291:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07441a1c000) 00000100:00100000:1.0:1787739056.966334:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6509:x1874579665594368:12345-0@lo:105: Request processed in 103us (379us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.966341:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1094 00000100:00100000:1.0:1787739056.966375:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665594112:0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739056.966389:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ac1b1 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac1b8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.967220:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665594496:0@lo:403:mdt00_004.0 00000100:00100000:1.0:1787739056.967265:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.967993:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e840 going to sleep for 11 seconds 00000100:00100000:0.0:1787739056.969187:0:6509:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fcb40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001_UUID:6509:1874579665594368:0@lo:105: 00010000:00010000:0.0:1787739056.969210:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739056.969273:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074301a3850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2408:x1874579652439552:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 3502us (4215us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739056.969284:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1432 00000100:00100000:2.0:1787739056.973650:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665594496 00000100:00100000:2.0:1787739056.973665:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665594496 00000100:00100000:2.0:1787739056.973736:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2786 00000100:00100000:2.0:1787739056.973750:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252bc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:13793:x1874579665594496:12345-0@lo:403:mdt00_004.0 00000100:00100000:2.0:1787739056.976503:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252bc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:13793:x1874579665594496:12345-0@lo:403:mdt00_004.0 Request processed in 2749us (9227us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739056.977409:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2786 00000100:00100000:1.0:1787739056.977963:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665594496:0@lo:403:mdt00_004.0 00000040:00080000:0.0:1787739056.979221:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].355, 664 off544400 00000040:00080000:0.0:1787739056.979398:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].354, 744 off588752 00000004:00080000:0.0:1787739056.979405:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306234d80 with version 676 00000004:00080000:3.0:1787739056.979476:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 676 rpc_version 676 00000020:00100000:3.0:1787739056.979510:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739056.979528:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306be70c0 x1874579665594752/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1808/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739056.979550:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be70c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665594752:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739056.979614:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739056.979661:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739056.979897:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665594752 00000100:00100000:2.0:1787739056.979904:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665594752 00000100:00100000:2.0:1787739056.979958:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2787 00000100:00100000:2.0:1787739056.979968:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665594752:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739056.980323:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665594752:12345-0@lo:1000:osp_up1-0.0 Request processed in 356us (708us total) trans 4294968329 rc 0/0 00000100:00100000:2.0:1787739056.980332:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2787 00000100:00100000:3.0:1787739056.980511:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be70c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665594752:0@lo:1000:osp_up1-0.0 00000004:00200000:1.0:1787739056.980655:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1503:0x0]: returning size 4096 00010000:00010000:1.0:1787739056.980679:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac1aa lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.980688:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac1aa lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.980694:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac1aa lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.980701:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739056.980705:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac1aa lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.980713:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac1aa lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.980727:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xbc:0x0].0x0 (ffffa07433235c00) 00010000:00010000:1.0:1787739056.980730:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739056.980735:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac1aa lrc: 2/0,0 mode: --/EX res: [0x200000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.980741:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac1aa lrc: 0/0,0 mode: --/EX res: [0x200000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.980756:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ac1b1 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac1b8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.980764:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ac1b1 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac1b8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.980770:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ac1b1 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac1b8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.980789:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739056.980797:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xbc:0x0] 00010000:00010000:1.0:1787739056.980806:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac1a3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.980812:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac1a3 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.980817:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac1a3 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.980821:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739056.980824:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac1a3 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.980830:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac1a3 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.980839:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1503:0x0].0x0 (ffffa07433235300) 00010000:00010000:1.0:1787739056.980842:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739056.980845:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac1a3 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.980851:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac1a3 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.980873:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac195 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac19c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.980880:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac195 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac19c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.980886:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac195 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac19c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.980902:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:1.0:1787739056.980910:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739056.980915:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ac179 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac180 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.980922:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ac179 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac180 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.980929:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ac179 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac180 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.980941:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:1.0:1787739056.980944:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739056.980948:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ac187 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac18e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.980960:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ac187 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac18e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739056.981024:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fc780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17488:x1874579652439296:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 61076us (61239us total) trans 4294983341 rc 0/0 00000100:00100000:1.0:1787739056.981032:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23277 00010000:00010000:3.0:1787739056.981258:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1787739056.981265:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ac1b1 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac1b8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.981272:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739056.981276:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ac1b1 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac1b8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739056.981283:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ac1b1 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac1b8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.981312:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ac1b1 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac1b8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.981319:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739056.981344:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ac1b1 lrc: 1/0,0 mode: --/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac1b8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.981351:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1ac1b1 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac1b8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.981365:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787739056.981368:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac195 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac19c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.981374:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739056.981377:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac195 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac19c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739056.981384:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac195 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac19c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.981400:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac195 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac19c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.981406:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739056.981445:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac195 lrc: 1/0,0 mode: --/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac19c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.981453:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac195 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac19c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.981470:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739056.981473:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ac179 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac180 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.981480:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739056.981483:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ac179 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac180 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739056.981489:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ac179 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac180 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.981507:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ac179 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac180 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.981512:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739056.981537:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ac179 lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac180 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.981544:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c400/0x7ee4453e4a1ac179 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac180 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739056.981555:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739056.981557:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739056.981559:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739056.981565:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739056.981576:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c416940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665595136:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739056.981615:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.981847:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c415e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665594880:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739056.981883:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.981912:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787739056.981918:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c416d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665595008:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739056.981939:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739056.981975:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665595136 00000100:00100000:1.0:1787739056.981980:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665595136 00000100:00100000:1.0:1787739056.982014:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665594880 00000100:00100000:1.0:1787739056.982017:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665594880 00000100:00100000:1.0:1787739056.982032:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665595008 00000100:00100000:1.0:1787739056.982034:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665595008 00000100:00100000:1.0:1787739056.982048:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1234 00000100:00100000:1.0:1787739056.982054:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c414b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+12:3652:x1874579665595136:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739056.982073:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739056.982090:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac180 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac179 expref: 12 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.982105:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305d56500) 00010000:00010000:1.0:1787739056.982112:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739056.982144:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c414b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+11:3652:x1874579665595136:12345-0@lo:103:ldlm_bl.0 Request processed in 89us (527us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.982151:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1234 00000100:00100000:1.0:1787739056.982167:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1235 00000100:00100000:1.0:1787739056.982173:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+10:3650:x1874579665594880:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739056.982181:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739056.982191:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac1b8 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac1b1 expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.982202:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xbc:0x0].0x0 (ffffa07305d56400) 00010000:00010000:1.0:1787739056.982208:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739056.982234:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+9:3650:x1874579665594880:12345-0@lo:103:ldlm_bl.0 Request processed in 61us (350us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.982240:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1235 00000100:00100000:1.0:1787739056.982253:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1236 00000100:00100000:1.0:1787739056.982259:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+8:3650:x1874579665595008:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739056.982266:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739056.982276:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac19c lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1503:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac195 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739056.982285:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1503:0x0].0x0 (ffffa07303c78c00) 00010000:00010000:1.0:1787739056.982292:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739056.982315:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+7:3650:x1874579665595008:12345-0@lo:103:ldlm_bl.0 Request processed in 56us (374us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739056.982320:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1236 00000100:00100000:1.0:1787739056.982355:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c416940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665595136:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739056.982380:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c416d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665595008:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739056.982388:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c415e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665594880:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739056.982637:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739057.027919:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739057.028018:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652439680 00000100:00100000:3.0:1787739057.028027:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652439680 00000100:00100000:3.0:1787739057.028110:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23278 00000100:00100000:3.0:1787739057.028120:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17489:x1874579652439680:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739057.028145:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739057.028164:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac1bf lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094cc2 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.028231:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac1c6 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.028246:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac1c6 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.028257:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac1c6 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.028269:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac1cd lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.028276:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac1cd lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.028282:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac1cd lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.028327:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac1c6 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.028334:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac1c6 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.028344:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac1c6 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.028351:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac1c6 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.028357:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739057.028362:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac1c6 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.028369:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac1c6 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.028381:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305d56300) 00010000:00010000:3.0:1787739057.028384:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739057.028389:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac1c6 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.028394:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac1c6 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787739057.028915:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1787739057.029007:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac1cd lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.029018:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac1cd lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.029025:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac1cd lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.029030:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739057.029034:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac1cd lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.029040:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac1cd lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.029066:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd9a063bc (ffffa07305d56000) 00010000:00010000:3.0:1787739057.029070:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739057.029073:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac1cd lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.029080:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac1cd lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.029118:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac1bf lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cc2 expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.029128:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305d56300) 00010000:00010000:3.0:1787739057.029130:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739057.029133:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac1bf lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cc2 expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.029142:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab600, rc 0) 00000100:00100000:2.0:1787739057.030222:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17489:x1874579652439680:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 2095us (2305us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739057.030233:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23278 00000100:00100000:1.0:1787739057.044486:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739057.044645:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652439808 00000100:00100000:0.0:1787739057.044654:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652439808 00000100:00100000:0.0:1787739057.044727:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23279 00000100:00100000:0.0:1787739057.044737:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17489:x1874579652439808:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739057.044757:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739057.044779:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac1d4 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094cc9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.044845:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac1db lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.044859:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac1db lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.044884:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac1db lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739057.044929:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x5:0x0]: returning size 4096 00000004:00200000:0.0:1787739057.044934:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739057.044968:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac1d4 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cc9 expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.044987:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac1db lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cc9 expref: 7 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.044995:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9700) 00010000:00010000:0.0:1787739057.044998:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739057.045000:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609be00, rc 0) 00000100:00100000:0.0:1787739057.045075:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17489:x1874579652439808:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 340us (597us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.045082:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23279 00000100:00080000:1.0:1787739057.045846:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787739057.060012:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739057.060177:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652440064 00000100:00100000:1.0:1787739057.060187:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652440064 00000100:00100000:1.0:1787739057.060247:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23280 00000100:00100000:1.0:1787739057.060257:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c416580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17489:x1874579652440064:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:1.0:1787739057.060362:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665595264:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739057.060439:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.060483:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ee40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.061009:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665595264 00000100:00100000:3.0:1787739057.061016:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665595264 00000100:00100000:3.0:1787739057.061069:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2788 00000100:00100000:3.0:1787739057.061077:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665595264:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739057.061168:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665595264:12345-0@lo:1000:mdt00_004.0 Request processed in 92us (755us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.061179:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2788 00000100:00100000:0.0:1787739057.061303:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665595264:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739057.061354:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ec000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665595392:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739057.061386:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.061442:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062346c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.064312:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665595392 00000100:00100000:3.0:1787739057.064327:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665595392 00000100:00100000:3.0:1787739057.064389:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2789 00000100:00100000:3.0:1787739057.064398:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665595392:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739057.064684:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665595392:12345-0@lo:1000:mdt00_004.0 Request processed in 286us (3297us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739057.064692:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2789 00000100:00100000:0.0:1787739057.064817:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ec000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665595392:0@lo:1000:mdt00_004.0 00010000:00010000:0.0:1787739057.064895:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac1e2 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.064905:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac1e2 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.064916:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac1e2 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739057.064935:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ec000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665595520:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739057.064973:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.065010:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.065096:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665595520 00000100:00100000:3.0:1787739057.065100:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665595520 00000100:00100000:3.0:1787739057.065132:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2790 00000100:00100000:3.0:1787739057.065138:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156efc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665595520:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739057.065148:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739057.065231:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac1e9 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac1e2 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.065255:0:8448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac0f4 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c9f expref: 16 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.065264:0:8448:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac1e9 lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac1e2 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.065288:0:8448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac0f4 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c9f expref: 16 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.065301:0:8448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac0f4 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c9f expref: 16 pid: 6523 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.065318:0:8448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c4152c0 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8448:1874579665595648:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739057.065376:0:8448:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89600 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.070753:0:8448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c4152c0 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8448:1874579665595648:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739057.070789:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac1e9 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac1e2 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.070802:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305d56300) 00010000:00010000:3.0:1787739057.070805:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739057.070810:0:8448:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac1e9 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac1e2 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.070820:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab600, rc 0) 00000100:00100000:3.0:1787739057.070881:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156efc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665595520:12345-0@lo:101:mdt00_004.0 Request processed in 5742us (5906us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.070889:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2790 00000100:00100000:0.0:1787739057.070973:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ec000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665595520:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739057.070993:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac1e2 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac1e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.071003:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac1e2 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac1e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.072869:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739057.072945:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652440192 00000100:00100000:1.0:1787739057.072955:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652440192 00010000:00010000:1.0:1787739057.072992:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac0f4 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c9f expref: 17 pid: 6523 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739057.073031:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1433 00000100:00100000:1.0:1787739057.073041:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c4152c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2408:x1874579652440192:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739057.073068:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739057.073075:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac0f4 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c9f expref: 17 pid: 6523 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.073088:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac0f4 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c9f expref: 17 pid: 6523 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.073115:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac0f4 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094c9f expref: 17 pid: 6523 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.073130:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305d56300) 00010000:00010000:1.0:1787739057.073136:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac1e9 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac1e2 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.073146:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac1e9 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac1e2 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.073165:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac1e9 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac1e2 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.073196:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac1e9 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac1e2 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739057.073210:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665595776:0@lo:105: 00000100:00100000:1.0:1787739057.073255:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.073294:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ecc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.073637:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665595776 00000100:00100000:3.0:1787739057.073642:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665595776 00000100:00100000:3.0:1787739057.073678:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1095 00000100:00100000:3.0:1787739057.073685:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665595776:12345-0@lo:105: 00010000:00010000:3.0:1787739057.073695:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac1e2 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac1e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.073704:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac1e2 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac1e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.073717:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac1e2 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac1e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.073724:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac1e2 lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac1e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.073730:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac1e2 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac1e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.073738:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac1e2 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ac1e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.073764:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609a400) 00000100:00100000:3.0:1787739057.073820:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665595776:12345-0@lo:105: Request processed in 136us (565us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.073827:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1095 00000100:00100000:2.0:1787739057.073996:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665595776:0@lo:105: 00010000:00010000:2.0:1787739057.074014:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739057.074056:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c4152c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:2408:x1874579652440192:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 1015us (1193us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739057.074072:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1433 00010000:00010000:0.0:1787739057.074076:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac1e2 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac1e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.074085:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac1e2 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac1e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739057.074105:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739057.074138:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ac1f0 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.074161:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ac1f0 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.074170:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ac1f0 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739057.074185:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ec000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665595904:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739057.074218:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.074243:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739057.074295:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665595904 00000100:00100000:1.0:1787739057.074300:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665595904 00000100:00100000:1.0:1787739057.074338:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2791 00000100:00100000:1.0:1787739057.074345:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ecf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665595904:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739057.074360:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739057.074375:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac1f7 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac1f0 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.074401:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac1f7 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac1f0 expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.074437:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac1f7 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac1f0 expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.074447:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd9a063bc (ffffa07433235800) 00010000:00010000:1.0:1787739057.074450:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739057.074453:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1cc00, rc 0) 00000100:00100000:1.0:1787739057.074483:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ecf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665595904:12345-0@lo:101:mdt00_004.0 Request processed in 138us (265us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739057.074489:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2791 00000100:00100000:0.0:1787739057.074656:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ec000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665595904:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739057.074667:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ac1f0 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac1f7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.074676:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ac1f0 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac1f7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739057.074689:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:0.0:1787739057.074729:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ec000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665596032:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739057.074756:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.074781:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062346c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739057.075271:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665596032 00000100:00100000:2.0:1787739057.075277:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665596032 00000100:00100000:2.0:1787739057.075315:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2792 00000100:00100000:2.0:1787739057.075323:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665596032:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739057.075570:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665596032:12345-0@lo:1000:mdt00_004.0 Request processed in 247us (812us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739057.075579:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2792 00000100:00100000:0.0:1787739057.075695:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ec000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665596032:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739057.075741:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ec000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665596160:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739057.075767:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.075819:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062346c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739057.075857:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665596160 00000100:00100000:2.0:1787739057.075861:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665596160 00000100:00100000:2.0:1787739057.075921:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2793 00000100:00100000:2.0:1787739057.075929:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665596160:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739057.076010:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665596160:12345-0@lo:1000:mdt00_004.0 Request processed in 81us (241us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739057.076017:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2793 00000100:00100000:0.0:1787739057.076087:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ec000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665596160:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739057.076176:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ec000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665596288:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739057.076202:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.076491:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062346c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739057.076655:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665596288 00000100:00100000:1.0:1787739057.076660:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665596288 00000100:00100000:0.0:1787739057.076761:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2794 00000100:00100000:0.0:1787739057.076768:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665596288:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739057.076833:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665596288:12345-0@lo:1000:mdt00_004.0 Request processed in 66us (630us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739057.076840:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2794 00000100:00100000:0.0:1787739057.076871:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ec000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665596288:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739057.076918:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ec000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665596416:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739057.076943:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.076957:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062346c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.076968:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665596416 00000100:00100000:0.0:1787739057.076972:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665596416 00000100:00100000:0.0:1787739057.076987:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2795 00000100:00100000:0.0:1787739057.076991:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fe940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665596416:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739057.077271:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fe940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665596416:12345-0@lo:1000:mdt00_004.0 Request processed in 277us (326us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.077278:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2795 00000100:00100000:0.0:1787739057.077307:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ec000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665596416:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739057.077346:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061ff840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665596544:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739057.077371:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.077393:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062346c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739057.077489:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665596544 00000100:00100000:1.0:1787739057.077493:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665596544 00000100:00100000:1.0:1787739057.077524:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2796 00000100:00100000:1.0:1787739057.077530:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665596544:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739057.077593:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665596544:12345-0@lo:1000:mdt00_004.0 Request processed in 62us (220us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739057.077599:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2796 00000100:00100000:0.0:1787739057.078388:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061ff840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665596544:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739057.078474:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061ff840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665596672:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739057.078512:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.078534:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062346c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739057.078603:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665596672 00000100:00100000:1.0:1787739057.078608:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665596672 00000100:00100000:1.0:1787739057.078646:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2797 00000100:00100000:1.0:1787739057.078653:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fcb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665596672:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739057.078729:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fcb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665596672:12345-0@lo:1000:mdt00_004.0 Request processed in 76us (215us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739057.078736:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2797 00000100:00100000:0.0:1787739057.078922:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061ff840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665596672:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739057.078954:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061ff840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665596800:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739057.078979:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.079003:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062346c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.079021:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665596800 00000100:00100000:0.0:1787739057.079024:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665596800 00000100:00100000:2.0:1787739057.079764:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2798 00000100:00100000:2.0:1787739057.079778:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665596800:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739057.079847:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665596800:12345-0@lo:1000:mdt00_004.0 Request processed in 74us (867us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739057.079853:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2798 00000100:00100000:0.0:1787739057.079992:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061ff840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665596800:0@lo:1000:mdt00_004.0 00000040:00080000:0.0:1787739057.081310:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].341, 2168 off520040 00000040:00080000:0.0:1787739057.081391:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].342, 2368 off569048 00000004:00080000:0.0:1787739057.081398:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07306234540 with version 652 00000004:00080000:2.0:1787739057.081682:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 652 rpc_version 652 00000020:00100000:2.0:1787739057.081786:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739057.081795:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304263840 x1874579665597056/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739057.081818:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304263840 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665597056:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739057.081899:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739057.082661:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d780 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.082820:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665597056 00000100:00100000:0.0:1787739057.082827:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665597056 00000100:00100000:0.0:1787739057.082927:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2799 00000100:00100000:0.0:1787739057.082937:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665597056:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739057.087147:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:7467:x1874579665597056:12345-0@lo:1000:osp_up0-1.0 Request processed in 4205us (5241us total) trans 4294983342 rc 0/0 00000100:00100000:0.0:1787739057.087781:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2799 00000100:00100000:2.0:1787739057.088024:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304263840 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665597056:0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1787739057.091618:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac1fe lrc: 3/0,1 mode: --/PW res: [0x240000402:0x167:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.091637:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac1fe lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x167:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.091650:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac1fe lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x167:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.091671:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac205 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xba:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.091679:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac205 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.091686:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac205 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.091733:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c800/0x7ee4453e4a1ac20c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xba:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.091740:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c800/0x7ee4453e4a1ac20c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.091749:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c800/0x7ee4453e4a1ac20c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739057.091772:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665597184:0@lo:101:mdt00_004.0 00000100:00100000:1.0:1787739057.091831:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.092271:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665597184 00000100:00100000:1.0:1787739057.092279:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665597184 00000100:00100000:1.0:1787739057.092389:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2800 00000100:00100000:1.0:1787739057.092399:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304000000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665597184:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739057.092672:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739057.092690:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac213 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xba:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac20c expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.092761:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac213 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac20c expref: 12 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.092775:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac213 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac20c expref: 12 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.092784:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xba:0x0].0x0 (ffffa07433235600) 00010000:00010000:1.0:1787739057.092787:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739057.092791:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1ca00, rc 0) 00000100:00100000:1.0:1787739057.092834:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304000000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:13793:x1874579665597184:12345-0@lo:101:mdt00_004.0 Request processed in 437us (1002us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739057.092841:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2800 00000100:00100000:1.0:1787739057.092867:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ecc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739057.092891:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665597184:0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739057.092906:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c800/0x7ee4453e4a1ac20c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac213 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.092916:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c800/0x7ee4453e4a1ac20c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac213 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739057.092938:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c5425594 reply state 000000009a736ede transno 4294968330 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac205 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.092950:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac205 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.092957:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xba:0x0].0x0 (ffffa07433235500) 00010000:00010000:1.0:1787739057.092960:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739057.092962:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787739057.092968:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c800/0x7ee4453e4a1ac20c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac213 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.092976:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c800/0x7ee4453e4a1ac20c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac213 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739057.092986:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xba:0x0] 00000004:00010000:1.0:1787739057.092993:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c5425594 reply state 000000009a736ede transno 4294968330 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac1fe lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x167:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.093001:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac1fe lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x167:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.093007:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x167:0x0].0x0 (ffffa07433235900) 00010000:00010000:1.0:1787739057.093009:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739057.093011:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:1.0:1787739057.093015:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x167:0x0]: returning size 4096 00000004:00080000:1.0:1787739057.093023:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739057.093031:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac1e2 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac1e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.093039:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac1e2 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac1e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.093050:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac1e2 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac1e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.093137:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739057.093142:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739057.093147:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ac1f0 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac1f7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.093154:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ac1f0 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac1f7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739057.093783:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c416580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17489:x1874579652440064:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 33524us (33778us total) trans 4294968330 rc 0/0 00000100:00100000:1.0:1787739057.093792:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23280 00010000:00010000:0.0:1787739057.095230:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739057.095242:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac1e2 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac1e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.095252:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739057.095259:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac1e2 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac1e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739057.095274:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac1e2 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac1e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.095316:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac1e2 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac1e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.095323:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739057.095344:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac1e2 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac1e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.095352:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac1e2 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac1e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.095370:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739057.095371:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739057.095373:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739057.095387:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061ff840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665597312:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739057.095460:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.095507:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665597312 00000100:00100000:0.0:1787739057.095513:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665597312 00000100:00100000:0.0:1787739057.095559:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1237 00000100:00100000:0.0:1787739057.095567:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561b840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+12:3653:x1874579665597312:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739057.095579:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739057.095597:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac1e9 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac1e2 expref: 12 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.095613:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305d56300) 00010000:00010000:0.0:1787739057.095622:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739057.095652:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561b840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+11:3653:x1874579665597312:12345-0@lo:103:ldlm_bl.0 Request processed in 86us (192us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.095658:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1237 00000100:00100000:0.0:1787739057.095692:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061ff840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665597312:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739057.130790:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739057.130872:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652440448 00000100:00100000:2.0:1787739057.130883:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652440448 00000100:00100000:2.0:1787739057.130938:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23281 00000100:00100000:2.0:1787739057.130948:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c415e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17490:x1874579652440448:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:2.0:1787739057.131124:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c415e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17490:x1874579652440448:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 176us (343us total) trans 0 rc -17/-17 00000100:00100000:2.0:1787739057.131131:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23281 00000100:00100000:3.0:1787739057.165014:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739057.166323:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652440576 00000100:00100000:2.0:1787739057.166338:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652440576 00000100:00100000:2.0:1787739057.166450:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23282 00000100:00100000:2.0:1787739057.166461:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c416d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17491:x1874579652440576:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739057.166483:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739057.166505:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ac21a lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094cd0 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.166709:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac221 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.166723:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac221 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.166734:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac221 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.166745:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac228 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.166756:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac1f7 lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac1f0 expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.166765:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac228 lrc: 4/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.166799:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac1f7 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ac1f0 expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739057.166814:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304260b40 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6525:1874579665597440:0@lo:104: 00000100:00100000:2.0:1787739057.166872:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739057.166927:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d780 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.167158:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665597440 00000100:00100000:3.0:1787739057.167163:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665597440 00000100:00100000:3.0:1787739057.167204:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1096 00000100:00100000:3.0:1787739057.167211:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6525:x1874579665597440:12345-0@lo:104: 00010000:00010000:3.0:1787739057.167224:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ac1f0 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac1f7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.167283:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739057.167290:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6525:x1874579665597440:12345-0@lo:104: Request processed in 80us (418us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.167296:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1096 00010000:00010000:3.0:1787739057.167315:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739057.167320:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ac1f0 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac1f7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.167329:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739057.167334:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ac1f0 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ac1f7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739057.167342:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ac1f0 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ac1f7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.167365:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ac1f0 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac1f7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.167372:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739057.167391:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ac1f0 lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac1f7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.167397:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ac1f0 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac1f7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.167455:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739057.167457:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739057.167459:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739057.167469:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c416940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665597568:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739057.167497:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739057.167533:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665597568 00000100:00100000:3.0:1787739057.167538:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665597568 00010000:00010000:3.0:1787739057.167553:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac1f7 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ac1f0 expref: 11 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.167574:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1434 00000100:00100000:3.0:1787739057.167581:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c382580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3650:x1874579665597568:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739057.167591:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787739057.168266:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304260b40 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6525:1874579665597440:0@lo:104: 00010000:00010000:3.0:1787739057.168289:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac228 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.170989:0:6502:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ac187 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac18e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739057.171011:0:6502:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ac187 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac18e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.171069:0:6502:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ac187 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac18e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.171077:0:6502:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739057.171110:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ac187 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac18e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787739057.171131:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983342 is committed 00000100:00100000:2.0:1787739057.171148:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c380f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665597696:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787739057.171216:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739057.171269:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665597696 00000001:00080000:3.0:1787739057.171271:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983341 is committed 00000100:00100000:2.0:1787739057.171275:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665597696 00000020:00080000:3.0:1787739057.171280:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 355 batchid updated 00000020:00080000:3.0:1787739057.171292:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa0730610e900 refcount 2 committed 0 result 0 batchid 356 00000020:00080000:3.0:1787739057.171296:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2500 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730610e840 00000020:00080000:3.0:1787739057.171299:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].353 00000020:00080000:3.0:1787739057.171301:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2a00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304cf4800 00000020:00080000:3.0:1787739057.171303:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].354 00000100:00100000:2.0:1787739057.171317:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1238 00000020:00080000:3.0:1787739057.171322:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 356 committed batchid 355 00000100:00100000:2.0:1787739057.171324:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074301a1850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+7:3651:x1874579665597696:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:2.0:1787739057.171336:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739057.171353:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac18e lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xe928ecd7 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac187 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.171364:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xe928ecd7 (ffffa073068a9f00) 00010000:00010000:2.0:1787739057.171373:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739057.171404:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074301a1850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+6:3651:x1874579665597696:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 81us (189us total) trans 0 rc 0/0 00000004:00080000:3.0:1787739057.171431:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07302e89600 with version 677 00000100:00100000:2.0:1787739057.171432:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1238 00000020:00080000:3.0:1787739057.171440:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 355 cancel update log [0x240000401:0x3:0x0].352: rc = 0 00000100:00100000:2.0:1787739057.171471:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c380f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665597696:0@lo:103:jbd2/dm-0-8.0 00000020:00080000:3.0:1787739057.171481:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 355 cancel update log [0x200000400:0x3:0x0].353: rc = 0 00000020:00080000:3.0:1787739057.171489:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 355 00000004:00080000:3.0:1787739057.171497:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 677 rpc_version 677 00000020:00100000:3.0:1787739057.171514:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739057.171522:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743c381680 x1874579665597952/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739057.171537:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c381680 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665597952:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739057.171567:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739057.171607:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89900 going to sleep for 11 seconds 00000100:00100000:2.0:1787739057.171616:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665597952 00000100:00100000:2.0:1787739057.171620:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665597952 00000100:00100000:2.0:1787739057.171645:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2801 00000100:00100000:2.0:1787739057.171651:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c3821c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665597952:12345-0@lo:1000:osp_up1-0.0 00010000:00010000:3.0:1787739057.171658:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac1f7 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ac1f0 expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.171669:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd9a063bc (ffffa07433235800) 00010000:00010000:3.0:1787739057.171674:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac228 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.171682:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac228 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.171690:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac228 lrc: 4/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739057.171771:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c3821c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665597952:12345-0@lo:1000:osp_up1-0.0 Request processed in 119us (202us total) trans 4294968331 rc 0/0 00000100:00100000:2.0:1787739057.171778:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2801 00000100:00100000:2.0:1787739057.171814:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c381680 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665597952:0@lo:1000:osp_up1-0.0 00010000:00010000:3.0:1787739057.172137:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739057.172215:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac228 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.172224:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac228 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.172245:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c382580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+6:3650:x1874579665597568:12345-0@lo:103:ldlm_bl.0 Request processed in 4665us (4747us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.172253:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1434 00000100:00100000:2.0:1787739057.172301:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304261680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665598080:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739057.172306:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c416940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665597568:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739057.172326:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739057.172356:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d840 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.172501:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665598080 00000100:00100000:3.0:1787739057.172506:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665598080 00000100:00100000:3.0:1787739057.172620:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2802 00000100:00100000:3.0:1787739057.172629:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042621c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6525:x1874579665598080:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739057.172691:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042621c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6525:x1874579665598080:12345-0@lo:1000:mdt00_002.0 Request processed in 64us (363us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.172698:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2802 00000100:00100000:3.0:1787739057.172775:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304261680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665598080:0@lo:1000:mdt00_002.0 00010000:00010000:3.0:1787739057.172815:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac22f lrc: 3/1,0 mode: --/PR res: [0x240000402:0x167:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.172827:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac22f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x167:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.172837:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac22f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x167:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739057.172857:0:6525:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac22f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x167:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.174653:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac221 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.174669:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac221 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.174676:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac221 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.174683:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac221 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.174689:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633ce00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739057.174692:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac221 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.175116:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac221 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.175137:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78f00) 00010000:00010000:3.0:1787739057.175140:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739057.175145:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac221 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.175152:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac221 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.175173:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac228 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.175179:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac228 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.175184:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac228 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.175190:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739057.175193:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac228 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.175200:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac228 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.175207:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd9a063bc (ffffa07433235800) 00010000:00010000:3.0:1787739057.175209:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739057.175212:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac228 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.175217:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac228 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.175267:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ac21a lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cd0 expref: 16 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.175284:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac22f lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x167:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cd0 expref: 15 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.175291:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x167:0x0].0x0 (ffffa074039f2500) 00010000:00010000:3.0:1787739057.175293:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739057.175295:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abc00, rc 0) 00000100:00100000:3.0:1787739057.175362:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c416d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17491:x1874579652440576:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 8903us (10356us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.175371:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23282 00010000:00010000:1.0:1787739057.177862:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac117 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1503:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.177878:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac117 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1503:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.177886:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac117 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1503:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.177894:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ce00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739057.177899:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac117 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1503:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.177906:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac117 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1503:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.177923:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1503:0x0].0x0 (ffffa07433235300) 00010000:00010000:1.0:1787739057.177926:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739057.177931:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac117 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1503:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.177937:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac117 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1503:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.177957:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac11e lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.177963:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac11e lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.177969:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac11e lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.177973:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1de00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739057.177976:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac11e lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.177981:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac11e lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.177989:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xbc:0x0].0x0 (ffffa07433235c00) 00010000:00010000:1.0:1787739057.177991:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739057.177993:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac11e lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.177999:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac11e lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.183373:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739057.183487:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652440704 00000100:00100000:2.0:1787739057.183497:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652440704 00000100:00100000:2.0:1787739057.183564:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23283 00000100:00100000:2.0:1787739057.183574:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c382580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17491:x1874579652440704:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739057.183597:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739057.183619:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac236 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x167:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094cd7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.183675:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac23d lrc: 3/1,0 mode: --/PR res: [0x240000402:0x167:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.183688:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac23d lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x167:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.183698:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac23d lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x167:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739057.183790:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x167:0x0]: returning size 4096 00000004:00200000:2.0:1787739057.183796:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739057.183833:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac236 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x167:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cd7 expref: 11 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.183853:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac23d lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x167:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cd7 expref: 10 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.183862:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x167:0x0].0x0 (ffffa07433235900) 00010000:00010000:2.0:1787739057.183864:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739057.183868:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633da00, rc 0) 00000100:00100000:2.0:1787739057.183924:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c382580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17491:x1874579652440704:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 350us (558us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739057.183931:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23283 00000100:00100000:3.0:1787739057.194206:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739057.194292:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652440832 00000100:00100000:0.0:1787739057.194300:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652440832 00000100:00100000:0.0:1787739057.194357:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23284 00000100:00100000:0.0:1787739057.194367:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c3830c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17491:x1874579652440832:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739057.194388:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739057.194408:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac244 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xba:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094cde expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.194489:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac24b lrc: 3/1,0 mode: --/PR res: [0x240000400:0xba:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.194508:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac24b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.194777:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac24b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739057.197015:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xba:0x0]: returning size 4096 00000004:00200000:1.0:1787739057.197026:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739057.197095:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac244 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xba:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cde expref: 12 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.197113:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac24b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xba:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cde expref: 11 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.197121:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xba:0x0].0x0 (ffffa07433235500) 00010000:00010000:1.0:1787739057.197124:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739057.197128:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a200, rc 0) 00000100:00100000:1.0:1787739057.197241:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c3830c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17491:x1874579652440832:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 2874us (3046us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739057.197251:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23284 00000100:00100000:3.0:1787739057.200852:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739057.204602:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652440960 00000100:00100000:0.0:1787739057.204616:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652440960 00000100:00100000:1.0:1787739057.207913:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23285 00000100:00100000:1.0:1787739057.207929:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c383480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17491:x1874579652440960:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739057.207948:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739057.207970:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac252 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xba:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094ce5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.208029:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac259 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xba:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.208045:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac213 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac20c expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.209252:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac259 lrc: 4/1,0 mode: --/PR res: [0x200000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.209294:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac213 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ac20c expref: 5 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739057.209307:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6525:1874579665598208:0@lo:104: 00000100:00100000:1.0:1787739057.209360:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.209404:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.209502:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665598208 00000100:00100000:0.0:1787739057.209511:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665598208 00000100:00100000:0.0:1787739057.209564:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1097 00000100:00100000:0.0:1787739057.209572:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6525:x1874579665598208:12345-0@lo:104: 00010000:00010000:0.0:1787739057.209587:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c800/0x7ee4453e4a1ac20c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac213 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.209638:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787739057.209644:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6525:x1874579665598208:12345-0@lo:104: Request processed in 74us (285us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.209651:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1097 00010000:00010000:0.0:1787739057.209673:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739057.209679:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c800/0x7ee4453e4a1ac20c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac213 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.209689:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739057.209693:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c800/0x7ee4453e4a1ac20c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ac213 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739057.209703:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c800/0x7ee4453e4a1ac20c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ac213 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.209730:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c800/0x7ee4453e4a1ac20c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac213 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.209736:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739057.209753:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c800/0x7ee4453e4a1ac20c lrc: 1/0,0 mode: --/PW res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac213 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.209759:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c800/0x7ee4453e4a1ac20c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac213 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.209776:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739057.209778:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739057.209779:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739057.209790:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305618f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665598336:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739057.209820:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.209857:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665598336 00000100:00100000:0.0:1787739057.209861:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665598336 00010000:00010000:0.0:1787739057.209877:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac213 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ac20c expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739057.209900:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1435 00000100:00100000:0.0:1787739057.209906:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561b840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+6:3652:x1874579665598336:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739057.209915:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787739057.210262:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6525:1874579665598208:0@lo:104: 00010000:00010000:1.0:1787739057.210278:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac259 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739057.214483:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 356 batchid updated 00000020:00080000:3.0:1787739057.214530:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 356 00000004:00080000:3.0:1787739057.214597:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa074062f3b40 with version 678 00000020:00080000:3.0:1787739057.214616:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 356 cancel update log [0x240000401:0x3:0x0].353: rc = 0 00010000:00010000:0.0:1787739057.214625:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac213 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ac20c expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.214645:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xba:0x0].0x0 (ffffa07433235600) 00010000:00010000:0.0:1787739057.214652:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac259 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.214661:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac259 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.214670:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac259 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1787739057.214691:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 356 cancel update log [0x200000400:0x3:0x0].354: rc = 0 00000020:00080000:3.0:1787739057.214702:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 356 00010000:00010000:0.0:1787739057.214713:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739057.214801:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561b840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+5:3652:x1874579665598336:12345-0@lo:103:ldlm_bl.0 Request processed in 4893us (4978us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.214811:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1435 00000100:00100000:3.0:1787739057.214838:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305618f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665598336:0@lo:103:ldlm_bl.0 00000004:00080000:2.0:1787739057.215138:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 678 rpc_version 678 00000020:00100000:2.0:1787739057.215167:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739057.215174:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304263c00 x1874579665598592/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739057.215194:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304263c00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665598592:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739057.215238:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739057.215273:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d000 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.215288:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665598592 00000100:00100000:3.0:1787739057.215295:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665598592 00000100:00100000:3.0:1787739057.215343:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2803 00000100:00100000:3.0:1787739057.215352:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07436d8f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665598592:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739057.215510:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07436d8f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665598592:12345-0@lo:1000:osp_up1-0.0 Request processed in 158us (270us total) trans 4294968332 rc 0/0 00000100:00100000:3.0:1787739057.215517:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2803 00000100:00100000:0.0:1787739057.215609:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304263c00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665598592:0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1787739057.217528:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac259 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.217545:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac259 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739057.217615:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xba:0x0]: returning size 4096 00000004:00200000:2.0:1787739057.217620:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739057.217672:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac252 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xba:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ce5 expref: 16 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.217694:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac259 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ce5 expref: 15 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.217704:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xba:0x0].0x0 (ffffa07433235600) 00010000:00010000:2.0:1787739057.217707:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739057.217711:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1ce00, rc 0) 00000100:00100000:2.0:1787739057.217781:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c383480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17491:x1874579652440960:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 9855us (16935us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739057.217789:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23285 00000100:00100000:3.0:1787739057.224628:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739057.224668:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739057.224763:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652441344 00000100:00100000:0.0:1787739057.224772:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652441344 00000100:00100000:0.0:1787739057.224832:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23286 00000100:00100000:0.0:1787739057.224844:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c3821c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17491:x1874579652441344:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:0.0:1787739057.224897:0:6523:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739057.224924:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac23d lrc: 0/0,0 mode: --/PR res: [0x240000402:0x167:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cd7 expref: 11 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.224943:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x167:0x0].0x0 (ffffa07433235900) 00010000:00010000:0.0:1787739057.224948:0:6523:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739057.225007:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac260 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.225015:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac260 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.225025:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac260 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739057.225045:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561b480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665598720:0@lo:101:mdt00_000.0 00000100:00100000:0.0:1787739057.225104:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.225134:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d000 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.225151:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665598720 00000100:00100000:0.0:1787739057.225155:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665598720 00000100:00100000:0.0:1787739057.225186:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2804 00000100:00100000:0.0:1787739057.225193:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305618f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+5:6523:x1874579665598720:12345-0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739057.225201:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739057.225215:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ac267 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac260 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.225260:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ac267 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac260 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739057.225623:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652441088 00000100:00100000:1.0:1787739057.225629:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652441088 00000100:00100000:1.0:1787739057.225671:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1239 00000100:00100000:1.0:1787739057.225679:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2407:x1874579652441088:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:1.0:1787739057.225689:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739057.225703:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac22f lrc: 0/0,0 mode: --/PR res: [0x240000402:0x167:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cd0 expref: 15 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.225716:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x167:0x0].0x0 (ffffa074039f2500) 00010000:00010000:1.0:1787739057.225724:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739057.225768:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2407:x1874579652441088:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 90us (1099us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739057.225776:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1239 00010000:00010000:0.0:1787739057.226980:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ac267 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac260 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.227065:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9200) 00010000:00010000:0.0:1787739057.227068:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739057.227072:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b000, rc 0) 00000100:00100000:0.0:1787739057.227145:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305618f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:6523:x1874579665598720:12345-0@lo:101:mdt00_000.0 Request processed in 1950us (2037us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.227153:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2804 00000100:00100000:0.0:1787739057.227215:0:6523:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983342 gen 2 00000004:00080000:0.0:1787739057.227588:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983337 committed transno 4294983342 00000020:00080000:0.0:1787739057.227595:0:6523:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa0730623d900 refcount 2 committed 0 result 0 batchid 347 00000020:00080000:0.0:1787739057.227599:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730415f700 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730623df00 00000020:00080000:0.0:1787739057.227602:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].341 00000020:00080000:0.0:1787739057.227604:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730415fc00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304011600 00000020:00080000:0.0:1787739057.227607:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].340 00000020:00080000:3.0:1787739057.227640:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 347 committed batchid 346 00000020:00080000:3.0:1787739057.227701:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 346 00000004:00080000:0.0:1787739057.227867:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983339 committed transno 4294983342 00000004:00080000:0.0:1787739057.227878:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983340 committed transno 4294983342 00000004:00080000:0.0:1787739057.227887:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983342 committed transno 4294983342 00000100:00100000:0.0:1787739057.227906:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561b480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665598720:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739057.227929:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac260 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac267 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.227948:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac260 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac267 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739057.227967:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739057.228332:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac26e lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.228341:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac26e lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.228350:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac26e lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739057.228369:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561b480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665598848:0@lo:101:mdt00_000.0 00000100:00100000:0.0:1787739057.229067:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.229103:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d000 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.229159:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665598848 00000100:00100000:3.0:1787739057.229165:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665598848 00000100:00100000:3.0:1787739057.229213:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2805 00000100:00100000:3.0:1787739057.229367:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:6523:x1874579665598848:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739057.229385:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739057.229519:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac275 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac26e expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.229553:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac275 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac26e expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.229566:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac275 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac26e expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.229574:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd9a063bc (ffffa074039f2f00) 00010000:00010000:3.0:1787739057.229577:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739057.229581:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa000, rc 0) 00000100:00100000:3.0:1787739057.232161:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665598848:12345-0@lo:101:mdt00_000.0 Request processed in 2789us (3447us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.232174:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2805 00000100:00100000:1.0:1787739057.236971:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561b480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665598848:0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739057.237086:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac26e lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac275 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.237102:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac26e lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac275 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739057.237127:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:3.0:1787739057.237371:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665598976:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739057.237596:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739057.237675:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074154fc180 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.237771:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665598976 00000100:00100000:0.0:1787739057.237779:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665598976 00000100:00100000:0.0:1787739057.237865:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2806 00000100:00100000:0.0:1787739057.237875:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c380f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665598976:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739057.237966:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c380f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665598976:12345-0@lo:1000:mdt00_000.0 Request processed in 92us (370us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.237973:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2806 00000100:00100000:3.0:1787739057.238046:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665598976:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739057.238100:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665599104:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739057.238126:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739057.238444:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074154fce40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739057.238543:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665599104 00000100:00100000:1.0:1787739057.238549:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665599104 00000100:00100000:1.0:1787739057.238615:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2807 00000100:00100000:1.0:1787739057.238622:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665599104:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739057.238718:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665599104:12345-0@lo:1000:mdt00_000.0 Request processed in 95us (590us total) trans 0 rc 1/1 00000100:00100000:1.0:1787739057.238725:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2807 00000100:00100000:1.0:1787739057.240214:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665599104:0@lo:1000:mdt00_000.0 00010000:00010000:1.0:1787739057.243159:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d000/0x7ee4453e4a1ac27c lrc: 3/0,1 mode: --/EX res: [0x240000402:0x167:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.243178:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d000/0x7ee4453e4a1ac27c lrc: 3/0,1 mode: --/EX res: [0x240000402:0x167:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.243191:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d000/0x7ee4453e4a1ac27c lrc: 3/0,1 mode: --/EX res: [0x240000402:0x167:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739057.243218:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665599232:0@lo:101:mdt00_000.0 00000100:00100000:1.0:1787739057.243290:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.243336:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665599232 00000100:00100000:1.0:1787739057.243343:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665599232 00000100:00100000:1.0:1787739057.243389:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2808 00000100:00100000:1.0:1787739057.245521:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665599232:12345-0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739057.245548:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739057.245576:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac283 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x167:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac27c expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.245605:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac283 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x167:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac27c expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.245618:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac283 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x167:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac27c expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.245626:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x167:0x0].0x0 (ffffa07433235300) 00010000:00010000:1.0:1787739057.245629:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739057.245632:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1da00, rc 0) 00000100:00100000:1.0:1787739057.245696:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665599232:12345-0@lo:101:mdt00_000.0 Request processed in 2299us (2406us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.245964:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.246132:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665599232:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739057.246155:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d000/0x7ee4453e4a1ac27c lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x167:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac283 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.246174:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d000/0x7ee4453e4a1ac27c lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x167:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac283 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.246208:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac28a lrc: 3/0,1 mode: --/EX res: [0x240000402:0x167:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.246229:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac28a lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x167:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.246240:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac28a lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x167:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.246262:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac291 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xba:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.246276:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac24b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cde expref: 10 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.246285:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac291 lrc: 4/0,1 mode: --/EX res: [0x240000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.246318:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac24b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cde expref: 10 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.246337:0:6523:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac24b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cde expref: 10 pid: 7944 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739057.246352:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561b480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665599360:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739057.246492:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2808 00000100:00100000:0.0:1787739057.247187:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.254308:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561b480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665599360:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739057.254337:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac291 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739057.259839:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739057.260200:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652441472 00000100:00100000:3.0:1787739057.260210:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652441472 00010000:00010000:3.0:1787739057.260250:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac24b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cde expref: 11 pid: 7944 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.260294:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1436 00000100:00100000:3.0:1787739057.260305:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2408:x1874579652441472:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739057.260322:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739057.260329:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac24b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xba:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cde expref: 11 pid: 7944 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.260343:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac24b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xba:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cde expref: 11 pid: 7944 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.260364:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac24b lrc: 0/0,0 mode: --/PR res: [0x240000400:0xba:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cde expref: 11 pid: 7944 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.260378:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xba:0x0].0x0 (ffffa07433235500) 00010000:00010000:3.0:1787739057.260383:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac291 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.260393:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac291 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.260436:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac291 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.260476:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739057.260505:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac291 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.260517:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac291 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.260533:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652441472:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 229us (703us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.260542:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1436 00010000:00010000:2.0:1787739057.260562:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1ac298 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xba:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.260570:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1ac298 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.260581:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1ac298 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739057.260607:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042612c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665599488:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1787739057.264227:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739057.264461:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.265278:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665599488 00000100:00100000:0.0:1787739057.265288:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665599488 00000100:00100000:0.0:1787739057.265339:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2809 00000100:00100000:0.0:1787739057.265349:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304263480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665599488:12345-0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739057.265369:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739057.265391:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac29f lrc: 2/0,0 mode: --/EX res: [0x200000401:0xba:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac298 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.265769:0:8447:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac259 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ce5 expref: 13 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.265782:0:8447:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac29f lrc: 4/0,0 mode: --/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac298 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.265822:0:8447:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac259 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ce5 expref: 13 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.265838:0:8447:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac259 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ce5 expref: 13 pid: 6525 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739057.265853:0:8447:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561b480 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8447:1874579665599616:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739057.265982:0:8447:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.279145:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739057.279219:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652441600 00000100:00100000:3.0:1787739057.279229:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652441600 00010000:00010000:3.0:1787739057.279269:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac259 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ce5 expref: 14 pid: 6525 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.279319:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1437 00000100:00100000:3.0:1787739057.279329:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fcf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652441600:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739057.279345:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739057.279353:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac259 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ce5 expref: 14 pid: 6525 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.279365:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac259 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ce5 expref: 14 pid: 6525 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.279389:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xba:0x0].0x0 (ffffa07433235600) 00010000:00010000:3.0:1787739057.279396:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac29f lrc: 3/0,0 mode: --/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac298 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.279408:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac29f lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac298 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.279441:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac29f lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac298 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.279475:0:6509:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac29f lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac298 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.279489:0:6509:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7a940 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665599744:0@lo:105: 00000100:00100000:3.0:1787739057.279539:0:6509:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739057.279583:0:6509:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07406576c00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.279611:0:6506:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:3.0:1787739057.279621:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665599744 00000100:00100000:3.0:1787739057.279624:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665599744 00000100:00100000:3.0:1787739057.279651:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1098 00000100:00100000:3.0:1787739057.279659:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306b9a050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665599744:12345-0@lo:105: 00010000:00010000:3.0:1787739057.279669:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1ac298 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac29f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.279679:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1ac298 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac29f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.279688:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1ac298 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac29f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.279695:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1ac298 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac29f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.279778:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1ac298 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac29f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.279788:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1ac298 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ac29f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.279803:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633d000) 00000100:00100000:3.0:1787739057.279846:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306b9a050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665599744:12345-0@lo:105: Request processed in 187us (305us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.279854:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1098 00000100:00100000:3.0:1787739057.279914:0:6509:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7a940 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665599744:0@lo:105: 00010000:00010000:3.0:1787739057.279930:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739057.279967:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fcf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652441600:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 639us (832us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.279973:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1437 00010000:00010000:2.0:1787739057.280444:0:8447:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac259 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xba:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ce5 expref: 13 pid: 6525 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739057.280463:0:8447:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561b480 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8447:1874579665599616:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739057.280483:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac29f lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac298 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.280492:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xba:0x0].0x0 (ffffa07433235600) 00010000:00010000:2.0:1787739057.280495:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739057.280497:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a000, rc 0) 00000100:00100000:2.0:1787739057.280549:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304263480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665599488:12345-0@lo:101:mdt00_000.0 Request processed in 15201us (16335us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739057.280556:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2809 00000100:00100000:0.0:1787739057.280632:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042612c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665599488:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739057.280647:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1ac298 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac29f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739057.281366:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061ff480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665599872:0@lo:403:mdt00_000.0 00000100:00100000:0.0:1787739057.281444:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.281472:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.281557:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665599872 00000100:00100000:3.0:1787739057.281564:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665599872 00000100:00100000:3.0:1787739057.281600:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2810 00000100:00100000:3.0:1787739057.281608:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fe940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665599872:12345-0@lo:403:mdt00_000.0 00000100:00100000:3.0:1787739057.284389:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fe940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665599872:12345-0@lo:403:mdt00_000.0 Request processed in 2774us (2968us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.284403:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2810 00000100:00100000:0.0:1787739057.285526:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061ff480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665599872:0@lo:403:mdt00_000.0 00000040:00080000:0.0:1787739057.286500:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].342, 664 off520704 00000040:00080000:0.0:1787739057.286689:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].343, 744 off569792 00000004:00080000:0.0:1787739057.286695:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07410b4d900 with version 653 00000004:00080000:3.0:1787739057.286912:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 653 rpc_version 653 00000020:00100000:3.0:1787739057.286946:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739057.286956:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07305f7a940 x1874579665600128/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1808/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739057.286977:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7a940 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665600128:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739057.287105:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739057.287149:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07406576c00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.287203:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665600128 00000100:00100000:0.0:1787739057.287210:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665600128 00000100:00100000:0.0:1787739057.287299:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2811 00000100:00100000:0.0:1787739057.287308:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f78b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665600128:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739057.287815:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f78b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665600128:12345-0@lo:1000:osp_up0-1.0 Request processed in 507us (708us total) trans 4294983343 rc 0/0 00000100:00100000:0.0:1787739057.287824:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2811 00000100:00100000:3.0:1787739057.292397:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7a940 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665600128:0@lo:1000:osp_up0-1.0 00000004:00200000:0.0:1787739057.294828:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x167:0x0]: returning size 4096 00010000:00010000:0.0:1787739057.294859:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac291 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.294871:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac291 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.294879:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac291 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.294885:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609bc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739057.294890:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac291 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.294898:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac291 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.294913:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xba:0x0].0x0 (ffffa07433235500) 00010000:00010000:0.0:1787739057.294916:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739057.294921:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac291 lrc: 2/0,0 mode: --/EX res: [0x240000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.294927:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac291 lrc: 0/0,0 mode: --/EX res: [0x240000400:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.294942:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1ac298 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac29f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.294950:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1ac298 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac29f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.294958:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1ac298 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac29f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295006:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739057.295017:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xba:0x0] 00010000:00010000:0.0:1787739057.295105:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac28a lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x167:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295113:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac28a lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x167:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295119:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac28a lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x167:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295125:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739057.295128:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac28a lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x167:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295133:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac28a lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x167:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295141:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x167:0x0].0x0 (ffffa07433235900) 00010000:00010000:0.0:1787739057.295144:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739057.295147:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac28a lrc: 2/0,0 mode: --/EX res: [0x240000402:0x167:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295153:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac28a lrc: 0/0,0 mode: --/EX res: [0x240000402:0x167:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295162:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d000/0x7ee4453e4a1ac27c lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x167:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac283 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295169:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d000/0x7ee4453e4a1ac27c lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x167:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac283 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295176:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d000/0x7ee4453e4a1ac27c lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x167:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac283 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295189:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:0.0:1787739057.295199:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739057.295207:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787739057.295212:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1ac298 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac29f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295223:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739057.295227:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1ac298 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac29f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739057.295234:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1ac298 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac29f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295265:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1ac298 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac29f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295271:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739057.295290:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1ac298 lrc: 1/0,0 mode: --/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac29f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295297:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d000/0x7ee4453e4a1ac298 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac29f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295312:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739057.295315:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d000/0x7ee4453e4a1ac27c lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x167:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac283 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295321:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739057.295325:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d000/0x7ee4453e4a1ac27c lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x167:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac283 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739057.295332:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d000/0x7ee4453e4a1ac27c lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x167:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac283 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295349:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d000/0x7ee4453e4a1ac27c lrc: 2/0,0 mode: --/EX res: [0x240000402:0x167:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac283 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295355:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739057.295368:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d000/0x7ee4453e4a1ac27c lrc: 1/0,0 mode: --/EX res: [0x240000402:0x167:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac283 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295376:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d000/0x7ee4453e4a1ac27c lrc: 0/0,0 mode: --/EX res: [0x240000402:0x167:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac283 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295386:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739057.295388:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739057.295390:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739057.295405:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061ff480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665600256:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739057.295489:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.295536:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665600384:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739057.295556:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.295590:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665600256 00000100:00100000:0.0:1787739057.295596:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665600256 00000100:00100000:0.0:1787739057.295635:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665600384 00000100:00100000:0.0:1787739057.295638:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665600384 00000100:00100000:0.0:1787739057.295654:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1240 00000100:00100000:0.0:1787739057.295661:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fcb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+11:3653:x1874579665600256:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739057.295672:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739057.295687:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac29f lrc: 0/0,0 mode: --/EX res: [0x200000401:0xba:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac298 expref: 11 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295715:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xba:0x0].0x0 (ffffa07433235600) 00010000:00010000:0.0:1787739057.295722:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739057.295751:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fcb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+10:3653:x1874579665600256:12345-0@lo:103:ldlm_bl.0 Request processed in 90us (262us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.295758:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1240 00000100:00100000:0.0:1787739057.295773:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1241 00000100:00100000:0.0:1787739057.295779:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061ff840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3650:x1874579665600384:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739057.295785:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739057.295794:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac283 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x167:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac27c expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295803:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x167:0x0].0x0 (ffffa07433235300) 00010000:00010000:0.0:1787739057.295808:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739057.295827:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061ff840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+8:3650:x1874579665600384:12345-0@lo:103:ldlm_bl.0 Request processed in 49us (271us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.295833:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1241 00000100:00100000:0.0:1787739057.295864:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665600384:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739057.295890:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061ff480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665600256:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739057.295913:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac260 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac267 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295921:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac260 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac267 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295928:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac260 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac267 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295943:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739057.295947:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739057.295954:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739057.295958:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac260 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac267 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295964:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ac00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739057.295967:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac260 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac267 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739057.295973:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac260 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac267 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295991:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac260 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac267 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.295997:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739057.296008:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac260 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac267 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.296015:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac260 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac267 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.296026:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739057.296028:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739057.296030:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739057.296038:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061ff480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665600512:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739057.296077:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.296103:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665600512 00000100:00100000:0.0:1787739057.296107:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665600512 00000100:00100000:0.0:1787739057.296129:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1242 00000100:00100000:0.0:1787739057.296135:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074301a2050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+8:3651:x1874579665600512:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739057.296142:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739057.296152:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ac267 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac260 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.296162:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9200) 00010000:00010000:0.0:1787739057.296167:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739057.296189:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074301a2050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+7:3651:x1874579665600512:12345-0@lo:103:ldlm_bl.0 Request processed in 54us (112us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.296194:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1242 00000100:00100000:0.0:1787739057.296235:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061ff480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665600512:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739057.296249:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac26e lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac275 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.296257:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac26e lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac275 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739057.296316:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c3821c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17491:x1874579652441344:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 71473us (71695us total) trans 4294968333 rc 0/0 00000100:00100000:0.0:1787739057.296322:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23286 00010000:00010000:2.0:1787739057.297823:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739057.336150:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739057.336223:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652441728 00000100:00100000:0.0:1787739057.336232:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652441728 00000100:00100000:0.0:1787739057.336291:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23287 00000100:00100000:0.0:1787739057.336302:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17492:x1874579652441728:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739057.336324:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739057.336345:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ac2a6 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094cec expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.336407:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac2ad lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.336449:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac2ad lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.336461:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac2ad lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.336476:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac2b4 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.336483:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac2b4 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.336490:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac2b4 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.336533:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac2ad lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.336540:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac2ad lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.336547:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac2ad lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.336558:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac2ad lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.336565:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739057.336569:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac2ad lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.336575:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac2ad lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.336587:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9700) 00010000:00010000:0.0:1787739057.336590:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739057.336593:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac2ad lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.336600:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac2ad lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.336611:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac2b4 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.336616:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac2b4 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.336622:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac2b4 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.336626:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739057.336629:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac2b4 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.336638:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac2b4 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.336646:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x60a2d708 (ffffa073068a9200) 00010000:00010000:0.0:1787739057.336648:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739057.336651:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac2b4 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.336656:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac2b4 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.336687:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ac2a6 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cec expref: 11 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.336697:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9700) 00010000:00010000:0.0:1787739057.336700:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739057.336818:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ac2a6 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cec expref: 11 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.336829:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b000, rc 0) 00000100:00100000:0.0:1787739057.336889:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17492:x1874579652441728:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 589us (748us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739057.336896:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23287 00000100:00100000:0.0:1787739057.343608:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739057.343712:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652441856 00000100:00100000:2.0:1787739057.343722:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652441856 00000100:00100000:2.0:1787739057.343779:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23288 00000100:00100000:2.0:1787739057.343789:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061ff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17492:x1874579652441856:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739057.343810:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739057.343841:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac2bb lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094cf3 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.343899:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac2c2 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.343913:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac2c2 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.343925:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac2c2 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739057.343970:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:2.0:1787739057.343976:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739057.344226:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac2bb lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cf3 expref: 15 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.344241:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac2c2 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cf3 expref: 14 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.344247:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78100) 00010000:00010000:2.0:1787739057.344249:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739057.344251:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633cc00, rc 0) 00000100:00100000:2.0:1787739057.344343:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061ff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17492:x1874579652441856:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 554us (746us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739057.344349:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23288 00000100:00100000:1.0:1787739057.350307:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739057.350389:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652442112 00000100:00100000:0.0:1787739057.350399:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652442112 00000100:00100000:0.0:1787739057.351033:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23289 00000100:00100000:0.0:1787739057.351044:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073025283c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17492:x1874579652442112:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:0.0:1787739057.351276:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fcf00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665600640:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739057.351377:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.351548:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665600640 00000100:00100000:1.0:1787739057.351555:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665600640 00000100:00100000:0.0:1787739057.351652:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355edcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739057.351760:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2812 00000100:00100000:1.0:1787739057.351770:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fcb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6523:x1874579665600640:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739057.351849:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fcb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6523:x1874579665600640:12345-0@lo:1000:mdt00_000.0 Request processed in 81us (519us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739057.351856:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2812 00000100:00100000:0.0:1787739057.351957:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fcf00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665600640:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739057.352004:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fcf00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665600768:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739057.352028:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.356035:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355edcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739057.356183:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665600768 00000100:00100000:1.0:1787739057.356194:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665600768 00000100:00100000:1.0:1787739057.356289:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2813 00000100:00100000:1.0:1787739057.356299:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061ff840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6523:x1874579665600768:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739057.356588:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061ff840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6523:x1874579665600768:12345-0@lo:1000:mdt00_000.0 Request processed in 290us (4558us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739057.356596:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2813 00000100:00100000:0.0:1787739057.360147:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fcf00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665600768:0@lo:1000:mdt00_000.0 00010000:00010000:0.0:1787739057.360233:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ac2c9 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.360243:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ac2c9 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.360253:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ac2c9 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739057.360270:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fcf00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665600896:0@lo:101:mdt00_000.0 00000100:00100000:0.0:1787739057.360328:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739057.363820:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665600896 00000100:00100000:0.0:1787739057.363864:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355edf00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.363898:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665600896 00000100:00100000:3.0:1787739057.363952:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2814 00000100:00100000:3.0:1787739057.363961:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6523:x1874579665600896:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739057.364048:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739057.364079:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac2d0 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac2c9 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.364117:0:17834:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac1db lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cc9 expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.364127:0:17834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac2d0 lrc: 4/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac2c9 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.364158:0:17834:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac1db lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cc9 expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.364168:0:17834:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac1db lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cc9 expref: 9 pid: 7944 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.364178:0:17834:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f79a40 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:17834:1874579665601024:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739057.367013:0:17834:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07406576c00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.371727:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739057.371807:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652442240 00000100:00100000:2.0:1787739057.371816:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652442240 00010000:00010000:2.0:1787739057.371865:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac1db lrc: 5/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cc9 expref: 10 pid: 7944 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739057.371903:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1438 00000100:00100000:2.0:1787739057.371911:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652442240:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739057.371928:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739057.371934:0:6510:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac1db lrc: 5/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cc9 expref: 10 pid: 7944 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.371946:0:6510:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac1db lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cc9 expref: 10 pid: 7944 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.371965:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9700) 00010000:00010000:2.0:1787739057.371971:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac2d0 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac2c9 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.371981:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac2d0 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac2c9 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.371999:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac2d0 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac2c9 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.372029:0:6510:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac2d0 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac2c9 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739057.372044:0:6510:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304261a40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6510:1874579665601152:0@lo:105: 00000100:00100000:2.0:1787739057.372097:0:6510:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739057.372137:0:6510:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d780 going to sleep for 11 seconds 00000100:00100000:1.0:1787739057.372228:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665601152 00000100:00100000:1.0:1787739057.372233:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665601152 00000100:00100000:1.0:1787739057.372268:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1099 00000100:00100000:1.0:1787739057.372275:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304260f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6510:x1874579665601152:12345-0@lo:105: 00010000:00010000:1.0:1787739057.372285:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ac2c9 lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac2d0 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.372294:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ac2c9 lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac2d0 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.372303:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ac2c9 lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac2d0 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.372311:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ac2c9 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac2d0 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.372318:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ac2c9 lrc: 5/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac2d0 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.372333:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ac2c9 lrc: 5/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ac2d0 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.372348:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609b000) 00000100:00100000:1.0:1787739057.372382:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304260f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6510:x1874579665601152:12345-0@lo:105: Request processed in 109us (286us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739057.372387:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1099 00000100:00100000:0.0:1787739057.372461:0:6510:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304261a40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6510:1874579665601152:0@lo:105: 00010000:00010000:0.0:1787739057.372476:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739057.372517:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652442240:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 605us (797us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.372524:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1438 00010000:00010000:3.0:1787739057.372887:0:17834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac1db lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cc9 expref: 9 pid: 7944 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.372904:0:17834:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f79a40 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:17834:1874579665601024:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739057.372923:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac2d0 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac2c9 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.372932:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9700) 00010000:00010000:3.0:1787739057.372935:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739057.372938:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aae00, rc 0) 00000100:00100000:0.0:1787739057.373042:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6523:x1874579665600896:12345-0@lo:101:mdt00_000.0 Request processed in 9080us (12712us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.373050:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2814 00000100:00100000:1.0:1787739057.373134:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fcf00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665600896:0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739057.373150:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ac2c9 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac2d0 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739057.373171:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739057.373206:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac2d7 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.373214:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac2d7 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.373223:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac2d7 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739057.373239:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252a580 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665601280:0@lo:101:mdt00_000.0 00000100:00100000:1.0:1787739057.373270:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.373296:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ee40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.373598:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665601280 00000100:00100000:0.0:1787739057.373604:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665601280 00000100:00100000:1.0:1787739057.373955:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2815 00000100:00100000:1.0:1787739057.373964:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6523:x1874579665601280:12345-0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739057.373979:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739057.373993:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac2de lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac2d7 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.374021:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac2de lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac2d7 expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.374034:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac2de lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac2d7 expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.374041:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x60a2d708 (ffffa07433235b00) 00010000:00010000:1.0:1787739057.374044:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739057.374047:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c000, rc 0) 00000100:00100000:1.0:1787739057.374107:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6523:x1874579665601280:12345-0@lo:101:mdt00_000.0 Request processed in 144us (836us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739057.374113:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2815 00000100:00100000:2.0:1787739057.374252:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252a580 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665601280:0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739057.374263:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac2d7 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac2de expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.374273:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac2d7 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac2de expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739057.374285:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:2.0:1787739057.374314:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304261a40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665601408:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739057.374337:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739057.374356:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d180 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.374375:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665601408 00000100:00100000:3.0:1787739057.374380:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665601408 00000100:00100000:3.0:1787739057.374449:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2816 00000100:00100000:3.0:1787739057.374457:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042612c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6523:x1874579665601408:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739057.374567:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042612c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6523:x1874579665601408:12345-0@lo:1000:mdt00_000.0 Request processed in 110us (227us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739057.374574:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2816 00000100:00100000:1.0:1787739057.374639:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304261a40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665601408:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739057.374680:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665601536:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739057.374705:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.374727:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739057.374773:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665601536 00000100:00100000:2.0:1787739057.374777:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665601536 00000100:00100000:2.0:1787739057.374811:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2817 00000100:00100000:2.0:1787739057.374818:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6523:x1874579665601536:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739057.374895:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6523:x1874579665601536:12345-0@lo:1000:mdt00_000.0 Request processed in 77us (189us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739057.374901:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2817 00000100:00100000:3.0:1787739057.374998:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665601536:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739057.375177:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f79a40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665601664:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739057.375206:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739057.375238:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07406576a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739057.375303:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665601664 00000100:00100000:2.0:1787739057.375308:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665601664 00000100:00100000:2.0:1787739057.375340:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2818 00000100:00100000:2.0:1787739057.375346:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6523:x1874579665601664:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739057.375401:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6523:x1874579665601664:12345-0@lo:1000:mdt00_000.0 Request processed in 55us (193us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739057.375408:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2818 00000100:00100000:3.0:1787739057.375754:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f79a40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665601664:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739057.375792:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f79a40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665601792:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739057.375809:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739057.375837:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739057.375849:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665601792 00000100:00100000:2.0:1787739057.375854:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665601792 00000100:00100000:2.0:1787739057.375922:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2819 00000100:00100000:2.0:1787739057.375929:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6523:x1874579665601792:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739057.375994:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6523:x1874579665601792:12345-0@lo:1000:mdt00_000.0 Request processed in 58us (174us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739057.376000:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2819 00000100:00100000:3.0:1787739057.376184:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f79a40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665601792:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739057.376221:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f79a40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665601920:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739057.376244:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739057.376268:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739057.376312:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665601920 00000100:00100000:2.0:1787739057.376316:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665601920 00000100:00100000:2.0:1787739057.376395:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2820 00000100:00100000:2.0:1787739057.376401:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f711e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6523:x1874579665601920:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739057.376493:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f711e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6523:x1874579665601920:12345-0@lo:1000:mdt00_000.0 Request processed in 92us (247us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739057.376500:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2820 00000100:00100000:3.0:1787739057.376567:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f79a40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665601920:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739057.376604:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f713840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665602048:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739057.376627:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739057.376877:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.376931:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665602048 00000100:00100000:0.0:1787739057.376936:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665602048 00000100:00100000:0.0:1787739057.376973:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2821 00000100:00100000:0.0:1787739057.376979:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6523:x1874579665602048:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739057.377049:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6523:x1874579665602048:12345-0@lo:1000:mdt00_000.0 Request processed in 70us (421us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.377064:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2821 00000100:00100000:0.0:1787739057.377092:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f713840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665602048:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739057.377123:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fcf00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665602176:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739057.377148:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.377163:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355ed600 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.377173:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665602176 00000100:00100000:0.0:1787739057.377176:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665602176 00000100:00100000:0.0:1787739057.377192:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2822 00000100:00100000:0.0:1787739057.377198:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6523:x1874579665602176:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739057.377243:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6523:x1874579665602176:12345-0@lo:1000:mdt00_000.0 Request processed in 44us (94us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.377249:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2822 00000100:00100000:0.0:1787739057.377274:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fcf00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665602176:0@lo:1000:mdt00_000.0 00000040:00080000:0.0:1787739057.378303:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].356, 2168 off546568 00000040:00080000:0.0:1787739057.378553:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].356, 2368 off592192 00000004:00080000:0.0:1787739057.378559:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0743c0efc00 with version 679 00000004:00080000:2.0:1787739057.378614:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 679 rpc_version 679 00000020:00100000:2.0:1787739057.378651:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739057.378660:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304262580 x1874579665602432/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739057.378678:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304262580 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665602432:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739057.378729:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739057.378757:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.378812:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665602432 00000100:00100000:0.0:1787739057.378819:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665602432 00000100:00100000:0.0:1787739057.378862:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2823 00000100:00100000:0.0:1787739057.378869:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304263840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665602432:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739057.379959:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304263840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:7740:x1874579665602432:12345-0@lo:1000:osp_up1-0.0 Request processed in 1087us (1225us total) trans 4294968334 rc 0/0 00000100:00100000:0.0:1787739057.379993:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2823 00000100:00100000:2.0:1787739057.380615:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304262580 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665602432:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787739057.380772:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac2e5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1505:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.380787:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac2e5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1505:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.380798:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac2e5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1505:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.380816:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac2ec lrc: 3/0,1 mode: --/PW res: [0x200000400:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.380823:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac2ec lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.380829:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac2ec lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.380860:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1ac2f3 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.380866:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1ac2f3 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.380876:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1ac2f3 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739057.380898:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d87840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665602560:0@lo:101:mdt00_000.0 00000100:00100000:0.0:1787739057.380938:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.381073:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355edf00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.381117:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665602560 00000100:00100000:3.0:1787739057.381123:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665602560 00000100:00100000:3.0:1787739057.381198:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2824 00000100:00100000:3.0:1787739057.381207:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:6523:x1874579665602560:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739057.381226:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739057.381244:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ac2fa lrc: 2/0,0 mode: --/PW res: [0x240000401:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac2f3 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.381265:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ac2fa lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac2f3 expref: 12 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.381276:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ac2fa lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac2f3 expref: 12 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.381284:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xbd:0x0].0x0 (ffffa074039f2000) 00010000:00010000:3.0:1787739057.381287:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739057.381291:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739057.381352:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:6523:x1874579665602560:12345-0@lo:101:mdt00_000.0 Request processed in 147us (413us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.381358:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2824 00000100:00100000:1.0:1787739057.381442:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d87840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665602560:0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739057.381456:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1ac2f3 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac2fa expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.381465:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1ac2f3 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac2fa expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739057.381486:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000096619cd reply state 0000000002fd9474 transno 4294983344 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac2ec lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.381503:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac2ec lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.381509:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xbd:0x0].0x0 (ffffa073068a9500) 00010000:00010000:1.0:1787739057.381511:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739057.381514:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787739057.381519:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1ac2f3 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac2fa expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.381526:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1ac2f3 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac2fa expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739057.381536:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xbd:0x0] 00000004:00010000:1.0:1787739057.381541:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000096619cd reply state 0000000002fd9474 transno 4294983344 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac2e5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1505:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.381549:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac2e5 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1505:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.381554:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1505:0x0].0x0 (ffffa073068a9300) 00010000:00010000:1.0:1787739057.381556:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739057.381558:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:1.0:1787739057.381562:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1505:0x0]: returning size 4096 00000004:00080000:1.0:1787739057.381569:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739057.381577:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ac2c9 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac2d0 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.381584:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ac2c9 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac2d0 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.381596:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ac2c9 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac2d0 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.381612:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739057.381616:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739057.381620:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac2d7 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac2de expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.381627:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac2d7 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac2de expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739057.381698:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073025283c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17492:x1874579652442112:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 30653us (31399us total) trans 4294983344 rc 0/0 00000100:00100000:1.0:1787739057.381705:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23289 00010000:00010000:2.0:1787739057.381773:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739057.381780:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ac2c9 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac2d0 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.381789:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739057.381793:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ac2c9 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac2d0 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739057.381801:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ac2c9 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac2d0 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.381828:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ac2c9 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac2d0 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.381835:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739057.381865:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ac2c9 lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac2d0 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.381873:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ac2c9 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac2d0 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.381889:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739057.381891:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739057.381892:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739057.381938:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304262d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665602688:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739057.381971:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739057.382110:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665602688 00000100:00100000:2.0:1787739057.382116:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665602688 00000100:00100000:2.0:1787739057.382159:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1243 00000100:00100000:2.0:1787739057.382166:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252bc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+12:3652:x1874579665602688:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739057.382176:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739057.382192:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac2d0 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac2c9 expref: 12 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.382203:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9700) 00010000:00010000:2.0:1787739057.382210:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739057.382239:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252bc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+11:3652:x1874579665602688:12345-0@lo:103:ldlm_bl.0 Request processed in 75us (268us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739057.382246:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1243 00000100:00100000:1.0:1787739057.382317:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304262d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665602688:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739057.413377:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739057.413526:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652442496 00000100:00100000:2.0:1787739057.413537:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652442496 00000100:00100000:2.0:1787739057.413597:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23290 00000100:00100000:2.0:1787739057.413608:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d843c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17493:x1874579652442496:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:2.0:1787739057.413783:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d843c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17493:x1874579652442496:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 176us (412us total) trans 0 rc -17/-17 00000100:00100000:2.0:1787739057.413790:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23290 00000100:00100000:0.0:1787739057.444047:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739057.444182:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652442624 00000100:00100000:2.0:1787739057.444191:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652442624 00000100:00100000:2.0:1787739057.444251:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23291 00000100:00100000:2.0:1787739057.444261:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d870c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17494:x1874579652442624:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739057.444282:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739057.444303:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac301 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094cfa expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.444389:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac308 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.444404:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac308 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.444441:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac308 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.444453:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac30f lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.444469:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac2de lrc: 2/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac2d7 expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.444477:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac30f lrc: 4/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.444511:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac2de lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ac2d7 expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739057.444527:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042630c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665602816:0@lo:104: 00000100:00100000:2.0:1787739057.444577:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739057.444623:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.444992:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665602816 00000100:00100000:0.0:1787739057.444997:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665602816 00000100:00100000:0.0:1787739057.445078:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1100 00000100:00100000:0.0:1787739057.445087:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6525:x1874579665602816:12345-0@lo:104: 00010000:00010000:0.0:1787739057.445100:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac2d7 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac2de expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.445163:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787739057.445169:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6525:x1874579665602816:12345-0@lo:104: Request processed in 84us (592us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.445177:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1100 00000100:00100000:2.0:1787739057.445236:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042630c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665602816:0@lo:104: 00010000:00010000:2.0:1787739057.445254:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac30f lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.452563:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739057.452577:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac2d7 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac2de expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.452590:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739057.452598:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac2d7 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ac2de expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739057.452608:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac2d7 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ac2de expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.452656:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac2d7 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac2de expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.452664:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739057.452693:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac2d7 lrc: 1/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac2de expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.452766:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ac2d7 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac2de expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.452785:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739057.452787:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739057.452789:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739057.452799:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787739057.452819:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665602944:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739057.453192:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739057.453932:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665602944 00000100:00100000:3.0:1787739057.453940:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665602944 00010000:00010000:3.0:1787739057.453973:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac2de lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ac2d7 expref: 11 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.454137:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1439 00000100:00100000:3.0:1787739057.454148:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+11:3652:x1874579665602944:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739057.454163:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739057.460261:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac26e lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac275 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739057.460283:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac26e lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac275 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.460566:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac26e lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac275 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.460580:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739057.460612:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac26e lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac275 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787739057.460631:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968334 is committed 00000001:00080000:1.0:1787739057.460647:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968333 is committed 00000100:00100000:3.0:1787739057.460796:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665603072:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:3.0:1787739057.460943:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739057.461077:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665603072 00000100:00100000:3.0:1787739057.461085:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665603072 00000100:00100000:3.0:1787739057.461131:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1244 00000100:00100000:3.0:1787739057.461140:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+7:3650:x1874579665603072:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:3.0:1787739057.461153:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739057.461171:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac275 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd9a063bc bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac26e expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.461187:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd9a063bc (ffffa074039f2f00) 00010000:00010000:3.0:1787739057.461196:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739057.461232:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+6:3650:x1874579665603072:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 93us (290us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.461240:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1244 00000100:00100000:3.0:1787739057.461283:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665603072:0@lo:103:jbd2/dm-1-8.0 00000020:00080000:1.0:1787739057.461491:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 347 batchid updated 00000020:00080000:1.0:1787739057.461515:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa073062346c0 refcount 2 committed 0 result 0 batchid 348 00000020:00080000:1.0:1787739057.461519:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88600 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07306234540 00000020:00080000:1.0:1787739057.461523:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].342 00000020:00080000:1.0:1787739057.461525:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88f00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa0730541ea00 00000020:00080000:1.0:1787739057.461528:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].341 00010000:00010000:1.0:1787739057.461556:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac1fe lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x167:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.461565:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac1fe lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x167:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.461572:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac1fe lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x167:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.461580:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739057.461584:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac1fe lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x167:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.461593:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac1fe lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x167:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.461605:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x167:0x0].0x0 (ffffa07433235900) 00000020:00080000:3.0:1787739057.461608:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 348 committed batchid 347 00010000:00010000:1.0:1787739057.461609:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739057.461615:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac1fe lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x167:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.461623:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac1fe lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x167:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.461639:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac205 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.461646:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac205 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.461652:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac205 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739057.461712:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07302e28f00 with version 654 00000020:00080000:3.0:1787739057.461723:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 347 cancel update log [0x200000401:0x3:0x0].341: rc = 0 00000020:00080000:3.0:1787739057.461761:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 347 cancel update log [0x240000400:0x3:0x0].340: rc = 0 00000020:00080000:3.0:1787739057.461770:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 347 00010000:00010000:3.0:1787739057.461816:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac2de lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ac2d7 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.461823:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739057.461828:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac205 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.461836:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac205 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.461846:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xba:0x0].0x0 (ffffa07433235500) 00010000:00010000:1.0:1787739057.461848:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739057.461851:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac205 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.461858:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac205 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xba:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.461903:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x60a2d708 (ffffa07433235b00) 00010000:00010000:3.0:1787739057.461908:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac30f lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.461918:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac30f lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.461925:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac30f lrc: 4/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.461956:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739057.462037:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac30f lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.462049:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac30f lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739057.462152:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665603200:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739057.462719:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.462759:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ea80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739057.462781:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665603200 00000100:00100000:1.0:1787739057.462787:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665603200 00000100:00100000:1.0:1787739057.462826:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2825 00000100:00100000:1.0:1787739057.462834:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:6525:x1874579665603200:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739057.462912:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:6525:x1874579665603200:12345-0@lo:1000:mdt00_002.0 Request processed in 77us (194us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739057.462920:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2825 00000100:00100000:1.0:1787739057.462960:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665603200:0@lo:1000:mdt00_002.0 00010000:00010000:1.0:1787739057.463003:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac316 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1505:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.463014:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac316 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1505:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.463023:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac316 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1505:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739057.463041:0:6525:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac316 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1505:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.463054:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac308 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.463077:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac308 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.463083:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac308 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.463089:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac308 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.463094:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739057.463098:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac308 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.463104:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac308 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.463113:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07303c78f00) 00010000:00010000:1.0:1787739057.463116:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739057.463120:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac308 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.463126:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac308 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.463138:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac30f lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.463144:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac30f lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.463151:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac30f lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.463156:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739057.463159:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac30f lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.463165:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac30f lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.463173:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x60a2d708 (ffffa07433235b00) 00010000:00010000:1.0:1787739057.463175:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739057.463178:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac30f lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.463184:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac30f lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.463222:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac301 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cfa expref: 8 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.463242:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac316 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1505:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cfa expref: 7 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.463248:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1505:0x0].0x0 (ffffa07305eae500) 00010000:00010000:1.0:1787739057.463250:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739057.463252:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c200, rc 0) 00000100:00100000:1.0:1787739057.463299:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d870c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17494:x1874579652442624:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 19039us (19259us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739057.463307:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23291 00000004:00080000:2.0:1787739057.472962:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 654 rpc_version 654 00000020:00100000:2.0:1787739057.473232:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739057.473245:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073042630c0 x1874579665603456/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739057.473275:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042630c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665603456:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739057.473350:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739057.473386:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739057.473406:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665603456 00000100:00100000:2.0:1787739057.473429:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665603456 00000100:00100000:2.0:1787739057.473480:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2826 00000100:00100000:2.0:1787739057.473489:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042621c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:7467:x1874579665603456:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739057.473644:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042621c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665603456:12345-0@lo:1000:osp_up0-1.0 Request processed in 155us (293us total) trans 4294983345 rc 0/0 00000100:00100000:2.0:1787739057.473652:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2826 00000100:00100000:2.0:1787739057.473693:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042630c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665603456:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739057.474135:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+6:3652:x1874579665602944:12345-0@lo:103:ldlm_bl.0 Request processed in 19987us (20942us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.474145:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1439 00000100:00100000:2.0:1787739057.474967:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665602944:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739057.475574:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00010000:00010000:0.0:1787739057.475699:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739057.483538:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652442752 00000100:00100000:3.0:1787739057.483554:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652442752 00000100:00100000:3.0:1787739057.484489:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23292 00000100:00100000:3.0:1787739057.484506:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17494:x1874579652442752:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739057.484534:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739057.484556:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac31d lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1505:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094d01 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.484622:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac324 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1505:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.484637:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac324 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1505:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.484648:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac324 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1505:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739057.484740:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1505:0x0]: returning size 4096 00000004:00200000:3.0:1787739057.484746:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739057.484783:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac31d lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1505:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d01 expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.484800:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac324 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1505:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d01 expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.484813:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1505:0x0].0x0 (ffffa073068a9300) 00010000:00010000:3.0:1787739057.484816:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739057.484820:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa600, rc 0) 00000100:00100000:3.0:1787739057.484945:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17494:x1874579652442752:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 444us (9373us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.484953:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23292 00000100:00100000:0.0:1787739057.498277:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739057.498334:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652442880 00000100:00100000:3.0:1787739057.498344:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652442880 00000100:00100000:3.0:1787739057.498406:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23293 00000100:00100000:3.0:1787739057.498435:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d85680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17494:x1874579652442880:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739057.498457:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739057.498505:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac32b lrc: 2/0,0 mode: --/CR res: [0x200000400:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094d08 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.498569:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac332 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.500800:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac332 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.500815:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac332 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739057.500936:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xbd:0x0]: returning size 4096 00000004:00200000:0.0:1787739057.500941:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739057.500985:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac32b lrc: 0/0,0 mode: --/CR res: [0x200000400:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d08 expref: 19 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.501001:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac332 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d08 expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.501009:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xbd:0x0].0x0 (ffffa073068a9500) 00010000:00010000:0.0:1787739057.501012:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739057.501016:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b800, rc 0) 00000100:00100000:0.0:1787739057.501107:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d85680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17494:x1874579652442880:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 2674us (2837us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.501117:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23293 00000100:00100000:3.0:1787739057.515879:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739057.516025:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652443008 00000100:00100000:1.0:1787739057.516036:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652443008 00000100:00100000:1.0:1787739057.516118:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23294 00000100:00100000:1.0:1787739057.516130:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17494:x1874579652443008:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739057.516150:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739057.516170:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac339 lrc: 2/0,0 mode: --/CR res: [0x240000401:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094d0f expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.516231:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac340 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.516246:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ac2fa lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac2f3 expref: 5 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.516254:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac340 lrc: 4/1,0 mode: --/PR res: [0x240000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.516285:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ac2fa lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ac2f3 expref: 5 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739057.516299:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665603584:0@lo:104: 00000100:00100000:1.0:1787739057.516349:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.516401:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e840 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.516449:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665603584 00000100:00100000:3.0:1787739057.516454:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665603584 00000100:00100000:3.0:1787739057.516489:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1101 00000100:00100000:3.0:1787739057.516498:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6525:x1874579665603584:12345-0@lo:104: 00010000:00010000:3.0:1787739057.516510:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1ac2f3 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac2fa expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.516582:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739057.516588:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6525:x1874579665603584:12345-0@lo:104: Request processed in 93us (240us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.516594:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1101 00010000:00010000:3.0:1787739057.516613:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739057.516619:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1ac2f3 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac2fa expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.516633:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609bc00 already unused, calling callback (ffffffffc18e6ee0) 00000100:00100000:1.0:1787739057.516643:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665603584:0@lo:104: 00010000:00010000:1.0:1787739057.516662:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac340 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.517455:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1ac2f3 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xbd:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ac2fa expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739057.517470:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1ac2f3 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xbd:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ac2fa expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.517502:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1ac2f3 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xbd:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac2fa expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.517509:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739057.517537:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1ac2f3 lrc: 1/0,0 mode: --/PW res: [0x240000401:0xbd:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac2fa expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.517544:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1ac2f3 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xbd:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac2fa expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.517564:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739057.517567:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739057.517568:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739057.519003:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f79a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665603712:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739057.519194:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739057.523194:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665603712 00000100:00100000:2.0:1787739057.523209:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665603712 00010000:00010000:2.0:1787739057.523252:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ac2fa lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ac2f3 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739057.523295:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1440 00000100:00100000:2.0:1787739057.523304:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c380000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+6:3651:x1874579665603712:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739057.523321:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00080000:3.0:1787739057.530659:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 348 batchid updated 00000020:00080000:0.0:1787739057.530699:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 348 00000004:00080000:0.0:1787739057.530770:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa074355edf00 with version 655 00000020:00080000:0.0:1787739057.530788:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 348 cancel update log [0x200000401:0x3:0x0].342: rc = 0 00000020:00080000:0.0:1787739057.530884:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 348 cancel update log [0x240000400:0x3:0x0].341: rc = 0 00000020:00080000:0.0:1787739057.530892:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 348 00000004:00080000:1.0:1787739057.530927:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 655 rpc_version 655 00000020:00100000:1.0:1787739057.530960:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739057.530972:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07302528000 x1874579665603968/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739057.530996:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665603968:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739057.531082:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.531113:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e840 going to sleep for 11 seconds 00000100:00100000:1.0:1787739057.531134:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665603968 00000100:00100000:1.0:1787739057.531139:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665603968 00000100:00100000:1.0:1787739057.531181:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2827 00000100:00100000:1.0:1787739057.531189:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665603968:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739057.531312:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665603968:12345-0@lo:1000:osp_up0-1.0 Request processed in 123us (229us total) trans 4294983346 rc 0/0 00000100:00100000:1.0:1787739057.531319:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2827 00000100:00100000:1.0:1787739057.531369:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665603968:0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1787739057.533775:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ac2fa lrc: 0/0,0 mode: --/PW res: [0x240000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ac2f3 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.533795:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xbd:0x0].0x0 (ffffa074039f2000) 00010000:00010000:1.0:1787739057.533801:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac340 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.533810:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac340 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.533818:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac340 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.534020:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739057.534245:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac340 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.534254:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac340 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739057.534309:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xbd:0x0]: returning size 4096 00000004:00200000:3.0:1787739057.534315:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739057.534360:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac339 lrc: 0/0,0 mode: --/CR res: [0x240000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d0f expref: 9 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.534376:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac340 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d0f expref: 8 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.534384:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xbd:0x0].0x0 (ffffa074039f2000) 00010000:00010000:3.0:1787739057.534387:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739057.534389:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d600, rc 0) 00000100:00100000:3.0:1787739057.537582:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17494:x1874579652443008:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 21442us (21696us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.537595:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23294 00000100:00100000:1.0:1787739057.540579:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c380000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665603712:12345-0@lo:103:ldlm_bl.0 Request processed in 17268us (21377us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739057.540596:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1440 00000100:00100000:2.0:1787739057.540797:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f79a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665603712:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739057.551244:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739057.551324:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652443136 00000100:00100000:0.0:1787739057.551333:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652443136 00000100:00100000:0.0:1787739057.551393:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1245 00000100:00100000:0.0:1787739057.551404:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f79a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2408:x1874579652443136:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:0.0:1787739057.551444:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739057.551469:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac316 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1505:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cfa expref: 8 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.551485:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1505:0x0].0x0 (ffffa07305eae500) 00010000:00010000:0.0:1787739057.551496:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739057.551545:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f79a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652443136:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 143us (310us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.551552:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1245 00000100:00100000:3.0:1787739057.552043:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739057.552142:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652443392 00000100:00100000:3.0:1787739057.552147:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652443392 00000100:00100000:3.0:1787739057.552179:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23295 00000100:00100000:3.0:1787739057.552187:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17494:x1874579652443392:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:3.0:1787739057.552235:0:6523:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739057.552251:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac324 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1505:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d01 expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.552264:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1505:0x0].0x0 (ffffa073068a9300) 00010000:00010000:3.0:1787739057.552268:0:6523:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739057.552311:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ac347 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.552324:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ac347 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.552339:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ac347 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.552359:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665604096:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739057.552402:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739057.552451:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28840 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.552471:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665604096 00000100:00100000:3.0:1787739057.552475:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665604096 00000100:00100000:3.0:1787739057.552501:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2828 00000100:00100000:3.0:1787739057.552507:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f7c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+5:6523:x1874579665604096:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739057.552517:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739057.552530:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ac34e lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac347 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.552551:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ac34e lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac347 expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.552563:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ac34e lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac347 expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.552569:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074039f2d00) 00010000:00010000:3.0:1787739057.552572:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739057.552575:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa400, rc 0) 00000100:00100000:3.0:1787739057.552607:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f7c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6523:x1874579665604096:12345-0@lo:101:mdt00_000.0 Request processed in 100us (204us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.552617:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2828 00000100:00100000:3.0:1787739057.552723:0:6523:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968334 gen 2 00000004:00080000:3.0:1787739057.552728:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968329 committed transno 4294968334 00000020:00080000:3.0:1787739057.552734:0:6523:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07306234840 refcount 2 committed 0 result 0 batchid 357 00000020:00080000:3.0:1787739057.552738:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88e00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07306234d80 00000020:00080000:3.0:1787739057.552741:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].354 00000020:00080000:3.0:1787739057.552743:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88d00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa0730541f400 00000020:00080000:3.0:1787739057.552746:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].355 00000004:00080000:3.0:1787739057.552768:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968331 committed transno 4294968334 00000004:00080000:3.0:1787739057.552778:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968332 committed transno 4294968334 00000004:00080000:3.0:1787739057.552787:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968334 committed transno 4294968334 00000100:00100000:3.0:1787739057.552807:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665604096:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739057.552821:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ac347 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac34e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.552832:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ac347 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac34e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739057.552847:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739057.552864:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1ac355 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.552871:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1ac355 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.552878:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1ac355 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.552892:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665604224:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739057.552916:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739057.552935:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28840 going to sleep for 11 seconds 00000020:00080000:3.0:1787739057.552945:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 357 committed batchid 356 00000020:00080000:3.0:1787739057.553002:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 356 00000100:00100000:3.0:1787739057.553015:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665604224 00000100:00100000:3.0:1787739057.553018:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665604224 00000100:00100000:3.0:1787739057.553044:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2829 00000100:00100000:3.0:1787739057.553051:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c381680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6523:x1874579665604224:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739057.553073:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739057.553084:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ac35c lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac355 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.553099:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ac35c lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac355 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.553110:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ac35c lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac355 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.553117:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x60a2d708 (ffffa074039f2c00) 00010000:00010000:3.0:1787739057.553119:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739057.553121:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739057.553156:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c381680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665604224:12345-0@lo:101:mdt00_000.0 Request processed in 105us (239us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.553161:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2829 00000100:00100000:3.0:1787739057.553187:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665604224:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739057.553197:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1ac355 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac35c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.553205:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1ac355 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac35c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739057.553218:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:3.0:1787739057.553247:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665604352:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739057.553272:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739057.553287:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28600 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.553298:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665604352 00000100:00100000:3.0:1787739057.553301:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665604352 00000100:00100000:3.0:1787739057.553319:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2830 00000100:00100000:3.0:1787739057.553325:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665604352:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739057.553378:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665604352:12345-0@lo:1000:mdt00_000.0 Request processed in 52us (105us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.553384:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2830 00000100:00100000:3.0:1787739057.553410:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665604352:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739057.553464:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be5e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665604480:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739057.553485:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739057.553509:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28600 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.553522:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665604480 00000100:00100000:3.0:1787739057.553525:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665604480 00000100:00100000:3.0:1787739057.553544:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2831 00000100:00100000:3.0:1787739057.553549:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665604480:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739057.553628:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665604480:12345-0@lo:1000:mdt00_000.0 Request processed in 78us (142us total) trans 0 rc 1/1 00000100:00100000:3.0:1787739057.553635:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2831 00000100:00100000:1.0:1787739057.553651:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665604480:0@lo:1000:mdt00_000.0 00010000:00010000:1.0:1787739057.553692:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac363 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.553700:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac363 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.553707:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac363 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739057.553720:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665604608:0@lo:101:mdt00_000.0 00000100:00100000:1.0:1787739057.553744:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.553759:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e840 going to sleep for 11 seconds 00000100:00100000:1.0:1787739057.553770:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665604608 00000100:00100000:1.0:1787739057.553773:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665604608 00000100:00100000:1.0:1787739057.553790:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2832 00000100:00100000:1.0:1787739057.553795:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6523:x1874579665604608:12345-0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739057.553803:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739057.553814:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac36a lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac363 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.553829:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac36a lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac363 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.553839:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac36a lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac363 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.553851:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1505:0x0].0x0 (ffffa07305eaea00) 00010000:00010000:1.0:1787739057.553853:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739057.553855:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1ca00, rc 0) 00000100:00100000:1.0:1787739057.553896:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6523:x1874579665604608:12345-0@lo:101:mdt00_000.0 Request processed in 100us (152us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739057.553901:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2832 00000100:00100000:1.0:1787739057.553926:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665604608:0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739057.553936:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac363 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac36a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.553944:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac363 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac36a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.553964:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac371 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.553972:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac371 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.553980:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac371 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.553995:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac378 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.554006:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac332 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d08 expref: 17 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.554018:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac378 lrc: 4/0,1 mode: --/EX res: [0x200000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.554037:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac332 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d08 expref: 17 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.554186:0:6523:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac332 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d08 expref: 17 pid: 6524 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739057.554198:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665604736:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739057.554253:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e840 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.559929:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665604736:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739057.559958:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac378 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739057.566306:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739057.566455:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652443520 00000100:00100000:3.0:1787739057.566465:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652443520 00010000:00010000:3.0:1787739057.566504:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac332 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d08 expref: 18 pid: 6524 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.566540:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1441 00000100:00100000:3.0:1787739057.566549:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074301a3050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2409:x1874579652443520:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739057.566563:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739057.566569:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac332 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d08 expref: 18 pid: 6524 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.566581:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac332 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d08 expref: 18 pid: 6524 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.566605:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac332 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d08 expref: 18 pid: 6524 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.566617:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xbd:0x0].0x0 (ffffa073068a9500) 00010000:00010000:3.0:1787739057.566622:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac378 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.566632:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac378 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.567463:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac378 lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.567502:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739057.567513:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac378 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.567521:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac378 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.567567:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074301a3050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652443520:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 1017us (1268us total) trans 0 rc 0/0 00010000:00010000:2.0:1787739057.567567:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ac37f lrc: 3/0,1 mode: --/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.567575:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ac37f lrc: 3/0,1 mode: --/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.567576:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1441 00010000:00010000:2.0:1787739057.567585:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ac37f lrc: 3/0,1 mode: --/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739057.567603:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304261680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665604864:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1787739057.567642:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739057.567678:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fccc00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.569825:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665604864 00000100:00100000:3.0:1787739057.569837:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665604864 00000100:00100000:3.0:1787739057.569913:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2833 00000100:00100000:3.0:1787739057.569923:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304260b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6523:x1874579665604864:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739057.569943:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739057.569965:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac386 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac37f expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.569998:0:17834:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac340 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d0f expref: 6 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.570007:0:17834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac386 lrc: 4/0,0 mode: --/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac37f expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.570037:0:17834:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac340 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d0f expref: 6 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.570068:0:17834:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac340 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d0f expref: 6 pid: 6525 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.570086:0:17834:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be6940 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:17834:1874579665604992:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739057.570159:0:17834:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.581847:0:17834:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be6940 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:17834:1874579665604992:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739057.581882:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac386 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac37f expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.581893:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xbd:0x0].0x0 (ffffa074039f2000) 00010000:00010000:3.0:1787739057.581896:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739057.581900:0:17834:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac386 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac37f expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.581911:0:17834:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac386 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac37f expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.581917:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aae00, rc 0) 00000100:00100000:3.0:1787739057.581979:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304260b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6523:x1874579665604864:12345-0@lo:101:mdt00_000.0 Request processed in 12057us (14335us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.581987:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2833 00000100:00100000:2.0:1787739057.581996:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304261680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665604864:0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739057.582014:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ac37f lrc: 4/0,1 mode: --/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac386 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.582024:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ac37f lrc: 4/0,1 mode: --/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac386 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739057.593561:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739057.594658:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652443648 00000100:00100000:1.0:1787739057.594670:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652443648 00010000:00010000:1.0:1787739057.594721:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac340 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d0f expref: 7 pid: 6525 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739057.594766:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1442 00000100:00100000:1.0:1787739057.594776:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652443648:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739057.594791:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739057.594797:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac340 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d0f expref: 7 pid: 6525 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.594810:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac340 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d0f expref: 7 pid: 6525 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.594834:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac340 lrc: 0/0,0 mode: --/PR res: [0x240000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d0f expref: 7 pid: 6525 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.594852:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xbd:0x0].0x0 (ffffa074039f2000) 00010000:00010000:1.0:1787739057.594870:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac386 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac37f expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.594880:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac386 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac37f expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.594889:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac386 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac37f expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.594923:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac386 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac37f expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739057.594937:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a30000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665605120:0@lo:105: 00000100:00100000:1.0:1787739057.594995:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.595039:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ea80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.595375:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665605120 00000100:00100000:3.0:1787739057.595381:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665605120 00000100:00100000:3.0:1787739057.595443:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1102 00000100:00100000:3.0:1787739057.595451:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a312c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665605120:12345-0@lo:105: 00010000:00010000:3.0:1787739057.595462:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ac37f lrc: 5/0,1 mode: --/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac386 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.595477:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ac37f lrc: 5/0,1 mode: --/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac386 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.595486:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ac37f lrc: 5/0,1 mode: --/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac386 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.595493:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ac37f lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac386 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.595500:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ac37f lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac386 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.595508:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ac37f lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ac386 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.595530:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633c600) 00010000:00010000:0.0:1787739057.595578:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ac37f lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac386 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.595586:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a312c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665605120:12345-0@lo:105: Request processed in 134us (588us total) trans 0 rc 0/0 00010000:00010000:0.0:1787739057.595595:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ac37f lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac386 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.595598:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1102 00000100:00100000:1.0:1787739057.596830:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a30000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665605120:0@lo:105: 00010000:00010000:1.0:1787739057.596847:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739057.596905:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2408:x1874579652443648:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 2129us (3348us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739057.596915:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1442 00000100:00100000:0.0:1787739057.597392:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665605248:0@lo:403:mdt00_000.0 00000100:00100000:0.0:1787739057.597469:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.597499:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0efc00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.597820:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665605248 00000100:00100000:3.0:1787739057.597826:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665605248 00000100:00100000:3.0:1787739057.598203:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2834 00000100:00100000:3.0:1787739057.598214:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d87840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6523:x1874579665605248:12345-0@lo:403:mdt00_000.0 00000100:00100000:3.0:1787739057.602383:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d87840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6523:x1874579665605248:12345-0@lo:403:mdt00_000.0 Request processed in 4162us (4905us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.602398:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2834 00000100:00100000:2.0:1787739057.604333:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665605248:0@lo:403:mdt00_000.0 00000040:00080000:2.0:1787739057.606810:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].357, 952 off547520 00000040:00080000:2.0:1787739057.607178:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].357, 1032 off593224 00000004:00080000:2.0:1787739057.607187:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07305fccc00 with version 680 00000004:00080000:1.0:1787739057.607638:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 680 rpc_version 680 00000020:00100000:1.0:1787739057.607681:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739057.607697:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303a30000 x1874579665605504/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739057.607806:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a30000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665605504:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739057.607882:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.607919:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ea80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.610236:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665605504 00000100:00100000:0.0:1787739057.610250:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665605504 00000100:00100000:0.0:1787739057.610588:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2835 00000100:00100000:0.0:1787739057.610600:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a330c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665605504:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739057.612573:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a330c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665605504:12345-0@lo:1000:osp_up1-0.0 Request processed in 1971us (4685us total) trans 4294968335 rc 0/0 00000100:00100000:0.0:1787739057.612588:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2835 00000100:00100000:1.0:1787739057.616888:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a30000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665605504:0@lo:1000:osp_up1-0.0 00000004:00200000:3.0:1787739057.625469:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1505:0x0]: returning size 4096 00010000:00010000:3.0:1787739057.625498:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac378 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.625509:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac378 lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.625517:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac378 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.625524:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739057.625529:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac378 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.625547:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac378 lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.625562:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xbd:0x0].0x0 (ffffa073068a9500) 00010000:00010000:3.0:1787739057.625566:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739057.625570:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac378 lrc: 2/0,0 mode: --/EX res: [0x200000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.625577:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac378 lrc: 0/0,0 mode: --/EX res: [0x200000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.625591:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ac37f lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac386 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.625599:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ac37f lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac386 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.625606:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ac37f lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac386 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.625630:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739057.625639:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xbd:0x0] 00010000:00010000:3.0:1787739057.625645:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac371 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.625652:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac371 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.625657:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac371 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.625662:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739057.625665:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac371 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.625671:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac371 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.625678:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1505:0x0].0x0 (ffffa073068a9300) 00010000:00010000:3.0:1787739057.625680:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739057.625682:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac371 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.625688:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac371 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.625697:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac363 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac36a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.625795:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac363 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac36a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.625803:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac363 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac36a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.625821:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:3.0:1787739057.625832:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739057.625837:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ac347 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac34e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.625844:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ac347 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac34e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.625851:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ac347 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac34e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.625864:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:3.0:1787739057.625866:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739057.625871:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1ac355 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac35c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.625878:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1ac355 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac35c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.625946:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1787739057.625954:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ac37f lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac386 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.625965:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17494:x1874579652443392:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 73773us (73917us total) trans 4294983347 rc 0/0 00010000:00010000:0.0:1787739057.625967:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739057.625976:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ac37f lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac386 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.625977:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23295 00000004:00010000:0.0:1787739057.625990:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ac37f lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac386 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.626023:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ac37f lrc: 2/0,0 mode: --/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac386 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.626029:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739057.626052:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ac37f lrc: 1/0,0 mode: --/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac386 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.626069:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c600/0x7ee4453e4a1ac37f lrc: 0/0,0 mode: --/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac386 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.626087:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787739057.626091:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac363 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac36a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.626096:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739057.626099:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac363 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac36a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739057.626105:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac363 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac36a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.626125:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac363 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac36a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.626131:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739057.626136:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac363 lrc: 1/0,0 mode: --/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac36a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.626142:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac363 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac36a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.626153:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739057.626156:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ac347 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac34e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.626161:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aac00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739057.626164:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ac347 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac34e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739057.626171:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ac347 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac34e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.626186:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ac347 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac34e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.626191:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739057.626204:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ac347 lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac34e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.626216:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ac347 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac34e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.626228:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739057.626230:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739057.626231:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739057.626237:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739057.626251:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fcf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665605888:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739057.626313:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787739057.626969:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739057.626983:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fc3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665605760:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739057.627017:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.627112:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665605888 00000100:00100000:0.0:1787739057.627119:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665605888 00000100:00100000:0.0:1787739057.627171:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665605760 00000100:00100000:0.0:1787739057.627174:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665605760 00000100:00100000:0.0:1787739057.627198:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1246 00000100:00100000:0.0:1787739057.627207:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157e9e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+11:3650:x1874579665605888:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739057.627221:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739057.627240:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ac34e lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac347 expref: 11 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.627255:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074039f2d00) 00010000:00010000:0.0:1787739057.627263:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739057.627311:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157e9e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+10:3650:x1874579665605888:12345-0@lo:103:ldlm_bl.0 Request processed in 105us (997us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.627318:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1246 00000100:00100000:0.0:1787739057.627333:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1247 00000100:00100000:0.0:1787739057.627339:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304261e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+9:3653:x1874579665605760:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739057.627350:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739057.627360:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac36a lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1505:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac363 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.627365:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665605632:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739057.627373:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1505:0x0].0x0 (ffffa07305eaea00) 00010000:00010000:0.0:1787739057.627382:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739057.627667:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739057.627739:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665605632 00000100:00100000:2.0:1787739057.627744:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665605632 00000100:00100000:2.0:1787739057.627771:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1248 00000100:00100000:2.0:1787739057.627778:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be6940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+9:3652:x1874579665605632:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739057.627786:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739057.627797:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac386 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac37f expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.627812:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xbd:0x0].0x0 (ffffa074039f2000) 00010000:00010000:2.0:1787739057.627817:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739057.627842:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be6940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+8:3652:x1874579665605632:12345-0@lo:103:ldlm_bl.0 Request processed in 65us (175us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739057.627849:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1248 00000100:00100000:3.0:1787739057.629240:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665605632:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739057.629749:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304261e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+7:3653:x1874579665605760:12345-0@lo:103:ldlm_bl.0 Request processed in 2055us (2377us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.629759:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1247 00000100:00100000:0.0:1787739057.629806:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fcf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665605888:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739057.630090:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fc3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665605760:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739057.698784:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739057.698819:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652443776 00000100:00100000:0.0:1787739057.698828:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652443776 00000100:00100000:0.0:1787739057.698879:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23296 00000100:00100000:0.0:1787739057.698890:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157ebc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17495:x1874579652443776:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739057.698913:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739057.698933:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac38d lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094d16 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.698993:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac394 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.699008:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac394 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.699020:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac394 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.699038:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac39b lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.699045:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac39b lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.699051:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac39b lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.699105:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac394 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.699112:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac394 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.699117:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac394 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.699123:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac394 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.699140:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ba00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739057.699144:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac394 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.699151:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac394 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.699163:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78100) 00010000:00010000:0.0:1787739057.699166:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739057.699171:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac394 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.699178:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac394 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.699189:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac39b lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.699195:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac39b lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.699201:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac39b lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.699205:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609bc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739057.699208:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac39b lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.699214:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac39b lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.699222:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xc9300a54 (ffffa073068a9900) 00010000:00010000:0.0:1787739057.699224:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739057.699227:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac39b lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.699232:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac39b lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.699261:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac38d lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d16 expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.699276:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78100) 00010000:00010000:0.0:1787739057.699279:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739057.699283:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac38d lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d16 expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.699292:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b800, rc 0) 00000100:00100000:0.0:1787739057.699339:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157ebc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17495:x1874579652443776:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 451us (564us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739057.699346:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23296 00000100:00100000:1.0:1787739057.722318:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739057.722397:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652443904 00000100:00100000:3.0:1787739057.722407:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652443904 00000100:00100000:3.0:1787739057.722489:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23297 00000100:00100000:3.0:1787739057.722499:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17495:x1874579652443904:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739057.722518:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739057.722537:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac3a2 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094d1d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.722595:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac3a9 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.722609:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac3a9 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.722621:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac3a9 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739057.722663:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x5:0x0]: returning size 4096 00000004:00200000:3.0:1787739057.722668:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739057.722701:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac3a2 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d1d expref: 8 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.722717:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac3a9 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d1d expref: 7 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.722730:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074039f2b00) 00010000:00010000:3.0:1787739057.722733:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739057.722737:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa600, rc 0) 00000100:00100000:3.0:1787739057.722784:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17495:x1874579652443904:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 286us (474us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.722791:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23297 00000100:00100000:3.0:1787739057.738178:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739057.738301:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652444160 00000100:00100000:2.0:1787739057.738311:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652444160 00000100:00100000:2.0:1787739057.738369:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23298 00000100:00100000:2.0:1787739057.738379:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17495:x1874579652444160:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:2.0:1787739057.738516:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304261680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665606016:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739057.738566:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739057.738612:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.738721:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665606016 00000100:00100000:3.0:1787739057.738726:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665606016 00000100:00100000:3.0:1787739057.738770:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2836 00000100:00100000:3.0:1787739057.738777:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665606016:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739057.741065:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665606016:12345-0@lo:1000:mdt00_000.0 Request processed in 2274us (2483us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.741078:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2836 00000100:00100000:1.0:1787739057.745348:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304261680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665606016:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739057.745452:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665606144:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739057.745512:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.745558:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610eb40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.746020:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665606144 00000100:00100000:3.0:1787739057.746028:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665606144 00000100:00100000:3.0:1787739057.746083:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2837 00000100:00100000:3.0:1787739057.746092:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665606144:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739057.746213:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665606144:12345-0@lo:1000:mdt00_000.0 Request processed in 122us (701us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739057.746221:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2837 00000100:00100000:1.0:1787739057.747426:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665606144:0@lo:1000:mdt00_000.0 00010000:00010000:1.0:1787739057.747494:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac3b0 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.747504:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac3b0 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.747514:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac3b0 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739057.747531:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665606272:0@lo:101:mdt00_000.0 00000100:00100000:1.0:1787739057.747575:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.747604:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ea80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.747706:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665606272 00000100:00100000:3.0:1787739057.747712:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665606272 00000100:00100000:3.0:1787739057.747759:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2838 00000100:00100000:3.0:1787739057.747767:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a321c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665606272:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739057.747783:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739057.747798:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac3b7 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac3b0 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.747823:0:18535:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac2c2 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cf3 expref: 16 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.747831:0:18535:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac3b7 lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac3b0 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.747853:0:18535:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac2c2 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cf3 expref: 16 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.747866:0:18535:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac2c2 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cf3 expref: 16 pid: 6525 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.747876:0:18535:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:18535:1874579665606400:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739057.747937:0:18535:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739057.760641:0:18535:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:18535:1874579665606400:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739057.760675:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac3b7 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac3b0 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.760688:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78100) 00010000:00010000:2.0:1787739057.760691:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739057.760695:0:18535:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac3b7 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac3b0 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.760703:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa000, rc 0) 00000100:00100000:2.0:1787739057.760766:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a321c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665606272:12345-0@lo:101:mdt00_000.0 Request processed in 12999us (13190us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739057.760775:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2838 00000100:00100000:0.0:1787739057.760848:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665606272:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739057.760865:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac3b0 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac3b7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.760875:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac3b0 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac3b7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.766751:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739057.768321:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652444288 00000100:00100000:1.0:1787739057.768336:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652444288 00010000:00010000:1.0:1787739057.768378:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac2c2 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cf3 expref: 17 pid: 6525 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739057.768519:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1443 00000100:00100000:1.0:1787739057.768530:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652444288:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739057.768544:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739057.768551:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac2c2 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cf3 expref: 17 pid: 6525 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.768564:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac2c2 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cf3 expref: 17 pid: 6525 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.768586:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac2c2 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094cf3 expref: 17 pid: 6525 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.768599:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78100) 00010000:00010000:1.0:1787739057.768606:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac3b7 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac3b0 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.768617:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac3b7 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac3b0 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.768626:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac3b7 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac3b0 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.768659:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac3b7 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac3b0 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739057.768671:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665606528:0@lo:105: 00000100:00100000:1.0:1787739057.768734:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.768807:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610eb40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.770278:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665606528 00000100:00100000:0.0:1787739057.770290:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665606528 00000100:00100000:0.0:1787739057.770342:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1103 00000100:00100000:0.0:1787739057.770350:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665606528:12345-0@lo:105: 00010000:00010000:0.0:1787739057.770364:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac3b0 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac3b7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.770375:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac3b0 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac3b7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.770389:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac3b0 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac3b7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.770399:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac3b0 lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac3b7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.770407:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac3b0 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac3b7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.770432:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac3b0 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ac3b7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.770454:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07441a1de00) 00000100:00100000:0.0:1787739057.770514:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665606528:12345-0@lo:105: Request processed in 164us (1779us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.770522:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1103 00010000:00010000:0.0:1787739057.770552:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac3b0 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac3b7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.770561:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac3b0 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac3b7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739057.770584:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739057.770617:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ac3be lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.770624:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ac3be lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.770633:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ac3be lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739057.770648:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157e92c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665606656:0@lo:101:mdt00_000.0 00000100:00100000:0.0:1787739057.770680:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.770824:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665606528:0@lo:105: 00010000:00010000:1.0:1787739057.770839:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739057.770877:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:2409:x1874579652444288:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 2348us (4135us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739057.770885:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1443 00000100:00100000:0.0:1787739057.771169:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.771291:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665606656 00000100:00100000:3.0:1787739057.771296:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665606656 00000100:00100000:3.0:1787739057.771337:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2839 00000100:00100000:3.0:1787739057.771345:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157eb0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665606656:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739057.771359:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739057.771375:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac3c5 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac3be expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.771403:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac3c5 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac3be expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.771432:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac3c5 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac3be expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.771442:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xc9300a54 (ffffa074039f2800) 00010000:00010000:3.0:1787739057.771445:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739057.771448:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abc00, rc 0) 00000100:00100000:3.0:1787739057.771492:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157eb0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665606656:12345-0@lo:101:mdt00_000.0 Request processed in 147us (811us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.771498:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2839 00000100:00100000:0.0:1787739057.771656:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074157e92c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665606656:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739057.771669:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ac3be lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac3c5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.771679:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ac3be lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac3c5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739057.771695:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:0.0:1787739057.771737:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157e92c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665606784:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739057.771768:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.771792:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.771884:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665606784 00000100:00100000:3.0:1787739057.771888:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665606784 00000100:00100000:3.0:1787739057.771920:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2840 00000100:00100000:3.0:1787739057.771927:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157ea940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665606784:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739057.771987:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157ea940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665606784:12345-0@lo:1000:mdt00_000.0 Request processed in 60us (218us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.771994:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2840 00000100:00100000:0.0:1787739057.772081:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074157e92c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665606784:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739057.772116:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157e92c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665606912:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739057.772153:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.772212:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739057.775029:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665606912 00000100:00100000:3.0:1787739057.775042:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665606912 00000100:00100000:3.0:1787739057.775137:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2841 00000100:00100000:3.0:1787739057.775145:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157e9a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665606912:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739057.775275:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157e9a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665606912:12345-0@lo:1000:mdt00_000.0 Request processed in 130us (3121us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739057.775285:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2841 00000100:00100000:1.0:1787739057.775354:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074157e92c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665606912:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739057.775475:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665607040:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739057.775512:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.780072:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ef00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.780235:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665607040 00000100:00100000:0.0:1787739057.780244:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665607040 00000100:00100000:0.0:1787739057.780355:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2842 00000100:00100000:0.0:1787739057.780364:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665607040:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739057.780499:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665607040:12345-0@lo:1000:mdt00_000.0 Request processed in 135us (4986us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739057.780507:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2842 00000100:00100000:1.0:1787739057.780805:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665607040:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739057.780885:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665607168:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739057.780919:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.780936:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ef00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.781092:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665607168 00000100:00100000:0.0:1787739057.781097:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665607168 00000100:00100000:0.0:1787739057.781140:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2843 00000100:00100000:0.0:1787739057.781147:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a31a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665607168:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739057.781198:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a31a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665607168:12345-0@lo:1000:mdt00_000.0 Request processed in 52us (278us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.781205:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2843 00000100:00100000:1.0:1787739057.782849:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665607168:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739057.782917:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665607296:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739057.782968:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.782996:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ef00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739057.783016:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665607296 00000100:00100000:1.0:1787739057.783021:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665607296 00000100:00100000:1.0:1787739057.783063:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2844 00000100:00100000:1.0:1787739057.783072:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665607296:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739057.783165:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665607296:12345-0@lo:1000:mdt00_000.0 Request processed in 94us (197us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739057.783172:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2844 00000100:00100000:1.0:1787739057.783203:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665607296:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739057.783238:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665607424:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739057.783259:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.783272:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ef00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739057.783282:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665607424 00000100:00100000:1.0:1787739057.783285:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665607424 00000100:00100000:1.0:1787739057.783301:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2845 00000100:00100000:1.0:1787739057.783306:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665607424:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739057.783353:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665607424:12345-0@lo:1000:mdt00_000.0 Request processed in 46us (93us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739057.783358:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2845 00000100:00100000:1.0:1787739057.783378:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665607424:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739057.783405:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665607552:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739057.783447:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.783460:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ef00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739057.783471:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665607552 00000100:00100000:1.0:1787739057.783474:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665607552 00000100:00100000:1.0:1787739057.783490:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2846 00000100:00100000:1.0:1787739057.783495:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665607552:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739057.783538:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665607552:12345-0@lo:1000:mdt00_000.0 Request processed in 42us (91us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739057.783545:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2846 00000100:00100000:1.0:1787739057.783567:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665607552:0@lo:1000:mdt00_000.0 00080000:02000000:1.0:1787739057.784847:0:6523:0:(osd_handler.c:2218:osd_trans_start()) *** cfs_fail_loc=19a, val=0*** 00000004:00080000:1.0:1787739057.784866:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730610eb40 with version 0 00000020:00080000:1.0:1787739057.784871:0:6523:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa0730610ef00 refcount 3 committed 0 result -5 batchid 350 00000020:00080000:1.0:1787739057.784876:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2100 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result -5 sub_th ffffa0730610eb40 00000020:00080000:1.0:1787739057.784879:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2800 obd lustre-MDT0001-osd committed 1 started 0 stopped 1 result 0 sub_th ffffa07304010e00 00000004:00080000:1.0:1787739057.784907:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739057.784917:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac3b0 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac3b7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.784928:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac3b0 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac3b7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.784935:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac3b0 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac3b7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.784959:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739057.784964:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739057.784968:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ac3be lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac3c5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.784975:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ac3be lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac3c5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.784981:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ac3be lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac3c5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.784994:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1787739057.785074:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17495:x1874579652444160:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 46694us (46903us total) trans 0 rc -5/-5 00000100:00100000:1.0:1787739057.785082:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23298 00000020:00080000:1.0:1787739057.785113:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 348 00010000:00010000:1.0:1787739057.785535:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1787739057.785543:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac3b0 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac3b7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.785551:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1de00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739057.785557:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac3b0 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac3b7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739057.785565:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac3b0 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac3b7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.785596:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac3b0 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac3b7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.785604:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739057.785631:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac3b0 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac3b7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.785638:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac3b0 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac3b7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.785657:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739057.785660:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ac3be lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac3c5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.785666:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739057.785669:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ac3be lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac3c5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739057.785676:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ac3be lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac3c5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.785695:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ac3be lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac3c5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.785781:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739057.785798:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ac3be lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac3c5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.785806:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b800/0x7ee4453e4a1ac3be lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac3c5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.785819:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739057.785821:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739057.785823:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739057.785831:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787739057.785845:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665607680:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739057.785891:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787739057.786636:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739057.786651:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665607808:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739057.786763:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739057.786816:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665607680 00000100:00100000:3.0:1787739057.786823:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665607680 00000100:00100000:3.0:1787739057.786860:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665607808 00000100:00100000:3.0:1787739057.786863:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665607808 00000100:00100000:3.0:1787739057.786880:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1249 00000100:00100000:3.0:1787739057.786888:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c30c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3650:x1874579665607680:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739057.786900:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739057.786918:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac3b7 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac3b0 expref: 11 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.786933:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78100) 00010000:00010000:3.0:1787739057.786942:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739057.786976:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c30c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+10:3650:x1874579665607680:12345-0@lo:103:ldlm_bl.0 Request processed in 88us (1084us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.786982:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1249 00000100:00100000:3.0:1787739057.786997:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1250 00000100:00100000:3.0:1787739057.787002:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074184c7c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+9:3652:x1874579665607808:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739057.787010:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739057.787021:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac3c5 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac3be expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.787032:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xc9300a54 (ffffa074039f2800) 00010000:00010000:3.0:1787739057.787038:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739057.787076:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074184c7c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3652:x1874579665607808:12345-0@lo:103:ldlm_bl.0 Request processed in 74us (313us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.787083:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1250 00000100:00100000:3.0:1787739057.787141:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665607808:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739057.787442:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665607680:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739057.808688:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739057.808731:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652444544 00000100:00100000:3.0:1787739057.808739:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652444544 00000100:00100000:3.0:1787739057.808788:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23299 00000100:00100000:3.0:1787739057.808798:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17505:x1874579652444544:12345-192.168.203.15@tcp:36:mkdir.0 00010000:00010000:3.0:1787739057.808895:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac3cc lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.808911:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac3cc lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.808923:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac3cc lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.808936:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac3d3 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.808959:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac3d3 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.808965:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac3d3 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.809364:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac3da lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1506:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.809375:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac3da lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1506:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.809382:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac3da lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1506:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739057.809396:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000dda58503 reply state 000000000ed37288 transno 4294983348 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac3da lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1506:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.809407:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac3da lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1506:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.809435:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1506:0x0].0x0 (ffffa074039f2e00) 00010000:00010000:3.0:1787739057.809439:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739057.809442:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:3.0:1787739057.809446:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1506:0x0]: returning size 4096 00010000:00010000:3.0:1787739057.809454:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac3cc lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.809461:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac3cc lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.809467:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac3cc lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.809473:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac3cc lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.809479:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739057.809483:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac3cc lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.809493:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac3cc lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.809503:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074039f2800) 00010000:00010000:3.0:1787739057.809506:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739057.809509:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac3cc lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.809514:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac3cc lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739057.809528:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000dda58503 reply state 000000000ed37288 transno 4294983348 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac3d3 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.809535:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac3d3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.809542:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xc9300a54 (ffffa074039f2900) 00010000:00010000:3.0:1787739057.809544:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739057.809546:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000100:00100000:3.0:1787739057.809622:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17505:x1874579652444544:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 824us (941us total) trans 4294983348 rc 0/0 00000100:00100000:3.0:1787739057.809630:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23299 00000100:00100000:1.0:1787739057.836650:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739057.836688:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652444672 00000100:00100000:1.0:1787739057.836697:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652444672 00000100:00100000:1.0:1787739057.836748:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23300 00000100:00100000:1.0:1787739057.836759:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17506:x1874579652444672:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739057.836784:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739057.836807:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac3e1 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094d24 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.836873:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac3e8 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.836892:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac3e8 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.836904:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac3e8 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.836917:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac3ef lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.836926:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac3ef lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.836933:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac3ef lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.837020:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac3f6 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1506:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.837028:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac3f6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1506:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.837037:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac3f6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1506:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739057.837094:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1506:0x0]: returning size 4096 00000004:00200000:1.0:1787739057.837099:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787739057.837120:0:13793:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac3f6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1506:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.837133:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac3e8 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.837144:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac3e8 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.837151:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac3e8 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.837158:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac3e8 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.837165:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ce00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739057.837169:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac3e8 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.837176:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac3e8 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.837188:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae100) 00010000:00010000:1.0:1787739057.837192:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739057.837195:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac3e8 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.837202:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac3e8 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.837215:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac3ef lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.837222:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac3ef lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.837228:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac3ef lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.837234:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ca00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739057.837238:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac3ef lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.837244:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac3ef lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.837254:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xc9300a54 (ffffa074039f2900) 00010000:00010000:1.0:1787739057.837257:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739057.837260:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac3ef lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.837267:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac3ef lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.837297:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac3e1 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d24 expref: 20 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.837317:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac3f6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1506:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d24 expref: 19 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.837324:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1506:0x0].0x0 (ffffa074039f2e00) 00010000:00010000:1.0:1787739057.837327:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787739057.837330:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c800, rc 0) 00000100:00100000:1.0:1787739057.837407:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+19:17506:x1874579652444672:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 649us (768us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739057.837452:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23300 00000100:00100000:3.0:1787739057.849289:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739057.852478:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652444928 00000100:00100000:1.0:1787739057.852496:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652444928 00000100:00100000:1.0:1787739057.852569:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23301 00000100:00100000:1.0:1787739057.852579:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+19:17506:x1874579652444928:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:1.0:1787739057.852634:0:6523:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739057.852664:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac3f6 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1506:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d24 expref: 19 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.852683:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1506:0x0].0x0 (ffffa074039f2e00) 00010000:00010000:1.0:1787739057.852688:0:6523:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739057.852894:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac3fd lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.852908:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac3fd lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.852918:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac3fd lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.852929:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac404 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.852936:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac404 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.852942:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac404 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.853011:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac40b lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1506:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.853029:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac40b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1506:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.853035:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac40b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1506:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739057.853244:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1506:0x0]: returning size 4096 00010000:00010000:2.0:1787739057.853259:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac40b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1506:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.853267:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac40b lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1506:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.853279:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac40b lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1506:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.853286:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633de00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739057.853290:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac40b lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1506:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.853298:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac40b lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1506:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.853306:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1506:0x0].0x0 (ffffa074039f2e00) 00010000:00010000:2.0:1787739057.853309:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739057.853312:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac40b lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1506:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.853318:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac40b lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1506:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.853335:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac3fd lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.853342:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac3fd lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.853348:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac3fd lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.853353:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac3fd lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.853358:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739057.853361:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac3fd lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.853367:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac3fd lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.853373:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78900) 00010000:00010000:2.0:1787739057.853375:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739057.853377:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac3fd lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.853383:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac3fd lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739057.853395:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003d9449a1 reply state 00000000f583368e transno 4294983349 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac404 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.853404:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac404 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.853410:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xc9300a54 (ffffa074039f2900) 00010000:00010000:2.0:1787739057.853436:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739057.853438:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787739057.853508:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+20:17506:x1874579652444928:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 929us (4227us total) trans 4294983349 rc 0/0 00000100:00100000:2.0:1787739057.853515:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23301 00000100:00100000:2.0:1787739057.880453:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739057.880555:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652445056 00000100:00100000:3.0:1787739057.880564:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652445056 00000100:00100000:3.0:1787739057.880625:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23302 00000100:00100000:3.0:1787739057.880635:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304263c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17507:x1874579652445056:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739057.880655:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739057.880686:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac412 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094d2b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.880873:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac419 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.880887:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac419 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.880897:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac419 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.880910:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac420 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.880916:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac420 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.880922:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac420 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.880959:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac419 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.880965:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac419 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.880971:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac419 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.880977:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac419 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.880982:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739057.880986:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac419 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.880993:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac419 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.881004:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074039f2b00) 00010000:00010000:3.0:1787739057.881006:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739057.881010:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac419 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.881016:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac419 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.881031:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac420 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.881037:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac420 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.881042:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac420 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.881047:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739057.881050:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac420 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.881073:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac420 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.881081:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x7a393ffc (ffffa074039f2800) 00010000:00010000:3.0:1787739057.881083:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739057.881086:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac420 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.881092:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac420 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.881124:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac412 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d2b expref: 8 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.881134:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074039f2b00) 00010000:00010000:3.0:1787739057.881136:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739057.881139:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac412 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d2b expref: 8 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.881149:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abc00, rc 0) 00000100:00100000:3.0:1787739057.881349:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304263c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17507:x1874579652445056:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 714us (904us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739057.881356:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23302 00000100:00100000:3.0:1787739057.889764:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739057.889824:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652445184 00000100:00100000:0.0:1787739057.889834:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652445184 00000100:00100000:0.0:1787739057.889899:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23303 00000100:00100000:0.0:1787739057.889910:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+20:17507:x1874579652445184:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739057.889933:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739057.889955:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac427 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094d32 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.890016:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac42e lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.890030:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac42e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.890042:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac42e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739057.890099:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:0.0:1787739057.890105:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739057.890142:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac427 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d32 expref: 22 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.890158:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac42e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d32 expref: 21 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.890180:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9c00) 00010000:00010000:0.0:1787739057.890184:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739057.890187:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a600, rc 0) 00000100:00100000:0.0:1787739057.890249:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:17507:x1874579652445184:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 342us (493us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.890257:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23303 00000100:00100000:2.0:1787739057.896474:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739057.896559:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652445440 00000100:00100000:3.0:1787739057.896569:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652445440 00000100:00100000:3.0:1787739057.896636:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23304 00000100:00100000:3.0:1787739057.896651:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:17507:x1874579652445440:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:3.0:1787739057.896772:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665607936:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739057.896828:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739057.896897:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739057.898372:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665607936 00000100:00100000:2.0:1787739057.898385:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665607936 00000100:00100000:2.0:1787739057.898488:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2847 00000100:00100000:2.0:1787739057.898499:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665607936:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739057.898627:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665607936:12345-0@lo:1000:mdt00_003.0 Request processed in 130us (1798us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739057.898634:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2847 00000100:00100000:3.0:1787739057.898718:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665607936:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739057.898779:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665608064:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739057.898814:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739057.898866:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.902033:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665608064 00000100:00100000:0.0:1787739057.902048:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665608064 00000100:00100000:0.0:1787739057.902107:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2848 00000100:00100000:0.0:1787739057.902117:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665608064:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739057.902505:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665608064:12345-0@lo:1000:mdt00_003.0 Request processed in 388us (3688us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739057.902513:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2848 00000100:00100000:3.0:1787739057.904340:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665608064:0@lo:1000:mdt00_003.0 00010000:00010000:3.0:1787739057.904462:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac435 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.904474:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac435 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.904486:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac435 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739057.904504:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665608192:0@lo:101:mdt00_003.0 00000100:00100000:3.0:1787739057.904552:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739057.904606:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739057.904712:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665608192 00000100:00100000:2.0:1787739057.904718:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665608192 00000100:00100000:2.0:1787739057.904764:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2849 00000100:00100000:2.0:1787739057.904771:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665608192:12345-0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739057.904785:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739057.904799:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac43c lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac435 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.904823:0:18535:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac3a9 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d1d expref: 6 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.904832:0:18535:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac43c lrc: 4/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac435 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.904852:0:18535:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac3a9 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d1d expref: 6 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.904865:0:18535:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac3a9 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d1d expref: 6 pid: 6525 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739057.904875:0:18535:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304261a40 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:18535:1874579665608320:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739057.904938:0:18535:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739057.907967:0:18535:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304261a40 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:18535:1874579665608320:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739057.908006:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac43c lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac435 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.908020:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074039f2b00) 00010000:00010000:2.0:1787739057.908024:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739057.908028:0:18535:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac43c lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac435 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.908038:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633de00, rc 0) 00000100:00100000:2.0:1787739057.908109:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665608192:12345-0@lo:101:mdt00_003.0 Request processed in 3338us (3556us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739057.908118:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2849 00000100:00100000:0.0:1787739057.908285:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665608192:0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739057.908305:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac435 lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac43c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.908316:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac435 lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac43c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739057.909798:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739057.914604:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652445568 00000100:00100000:1.0:1787739057.914617:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652445568 00010000:00010000:1.0:1787739057.914664:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac3a9 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d1d expref: 7 pid: 6525 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739057.914706:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1444 00000100:00100000:1.0:1787739057.914716:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157ead00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652445568:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739057.914732:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739057.914739:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac3a9 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d1d expref: 7 pid: 6525 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.914752:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac3a9 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d1d expref: 7 pid: 6525 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.914774:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac3a9 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d1d expref: 7 pid: 6525 timeout: 744 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.914790:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074039f2b00) 00010000:00010000:1.0:1787739057.914798:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac43c lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac435 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.914809:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac43c lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac435 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.914818:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac43c lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac435 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.914852:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac43c lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac435 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739057.914867:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c1e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665608448:0@lo:105: 00000100:00100000:1.0:1787739057.914940:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.914982:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ee40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739057.914999:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665608448 00000100:00100000:1.0:1787739057.915002:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665608448 00000100:00100000:1.0:1787739057.915030:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1104 00000100:00100000:1.0:1787739057.915037:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c2d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665608448:12345-0@lo:105: 00010000:00010000:1.0:1787739057.915047:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac435 lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac43c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.915065:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac435 lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac43c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.915073:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac435 lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac43c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.915080:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac435 lrc: 5/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac43c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.915087:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac435 lrc: 6/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac43c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.915095:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac435 lrc: 6/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ac43c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.915115:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069abc00) 00000100:00100000:1.0:1787739057.915151:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c2d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665608448:12345-0@lo:105: Request processed in 114us (211us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739057.915158:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1104 00010000:00010000:1.0:1787739057.915195:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac435 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac43c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.915204:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac435 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac43c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739057.915229:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739057.915393:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac443 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.915401:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac443 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.915437:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac443 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739057.915522:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c12c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665608576:0@lo:101:mdt00_003.0 00000100:00100000:1.0:1787739057.915553:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.915575:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610eb40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739057.915602:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c1e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665608448:0@lo:105: 00010000:00010000:1.0:1787739057.915612:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739057.915653:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157ead00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2408:x1874579652445568:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 938us (5859us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739057.915662:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1444 00000100:00100000:1.0:1787739057.915775:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665608576 00000100:00100000:1.0:1787739057.915779:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665608576 00000100:00100000:1.0:1787739057.915807:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2850 00000100:00100000:1.0:1787739057.915814:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c1a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665608576:12345-0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739057.915829:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739057.915843:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac44a lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac443 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.915863:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac44a lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac443 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.915875:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac44a lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac443 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.915882:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x7a393ffc (ffffa07305eae800) 00010000:00010000:1.0:1787739057.915885:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739057.915887:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1ca00, rc 0) 00000100:00100000:1.0:1787739057.915917:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c1a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665608576:12345-0@lo:101:mdt00_003.0 Request processed in 104us (364us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739057.915923:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2850 00000100:00100000:1.0:1787739057.916352:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c12c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665608576:0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739057.916367:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac443 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac44a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739057.916376:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac443 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac44a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739057.916390:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:1.0:1787739057.916450:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c12c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665608704:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739057.917376:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.917439:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ee40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739057.917528:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665608704 00000100:00100000:1.0:1787739057.917534:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665608704 00000100:00100000:1.0:1787739057.917571:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2851 00000100:00100000:1.0:1787739057.917577:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c1e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665608704:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739057.917639:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c1e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665608704:12345-0@lo:1000:mdt00_003.0 Request processed in 62us (264us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739057.917645:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2851 00000100:00100000:1.0:1787739057.917676:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c12c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665608704:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739057.917722:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c12c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665608832:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739057.917743:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.917752:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ee40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739057.917766:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665608832 00000100:00100000:1.0:1787739057.917769:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665608832 00000100:00100000:1.0:1787739057.917787:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2852 00000100:00100000:1.0:1787739057.917793:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c2580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665608832:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739057.917855:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c2580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665608832:12345-0@lo:1000:mdt00_003.0 Request processed in 62us (111us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739057.917862:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2852 00000100:00100000:1.0:1787739057.917885:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c12c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665608832:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739057.919388:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c12c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665608960:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739057.919564:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.919592:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ee40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739057.919617:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665608960 00000100:00100000:1.0:1787739057.919623:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665608960 00000100:00100000:1.0:1787739057.919700:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2853 00000100:00100000:1.0:1787739057.919709:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665608960:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739057.919788:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665608960:12345-0@lo:1000:mdt00_003.0 Request processed in 81us (225us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739057.919796:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2853 00000100:00100000:0.0:1787739057.926664:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c12c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665608960:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739057.927024:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157e9e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665609088:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739057.927098:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.927137:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef540 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.927157:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665609088 00000100:00100000:0.0:1787739057.927162:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665609088 00000100:00100000:0.0:1787739057.927207:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2854 00000100:00100000:0.0:1787739057.927216:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157e92c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665609088:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739057.927291:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157e92c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665609088:12345-0@lo:1000:mdt00_003.0 Request processed in 76us (193us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.927299:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2854 00000100:00100000:0.0:1787739057.927328:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074157e9e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665609088:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739057.927761:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157e9e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665609216:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739057.927792:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.927810:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.927828:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665609216 00000100:00100000:0.0:1787739057.927832:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665609216 00000100:00100000:0.0:1787739057.927858:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2855 00000100:00100000:0.0:1787739057.927866:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665609216:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739057.927930:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665609216:12345-0@lo:1000:mdt00_003.0 Request processed in 65us (137us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.927937:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2855 00000100:00100000:0.0:1787739057.928001:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074157e9e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665609216:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739057.928035:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fcf00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665609344:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739057.928069:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.928082:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.928093:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665609344 00000100:00100000:0.0:1787739057.928096:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665609344 00000100:00100000:0.0:1787739057.928122:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2856 00000100:00100000:0.0:1787739057.928127:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157e9e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665609344:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739057.928171:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157e9e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665609344:12345-0@lo:1000:mdt00_003.0 Request processed in 43us (101us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.928177:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2856 00000100:00100000:0.0:1787739057.928197:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fcf00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665609344:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739057.928225:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fcf00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665609472:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739057.928244:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.928256:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.928265:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665609472 00000100:00100000:0.0:1787739057.928268:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665609472 00000100:00100000:0.0:1787739057.928282:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2857 00000100:00100000:0.0:1787739057.928287:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415701e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665609472:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739057.928325:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415701e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665609472:12345-0@lo:1000:mdt00_003.0 Request processed in 37us (79us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739057.929077:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2857 00000100:00100000:0.0:1787739057.929127:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fcf00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665609472:0@lo:1000:mdt00_003.0 00000040:00080000:0.0:1787739057.936382:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].358, 2168 off549688 00000040:00080000:0.0:1787739057.936489:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].358, 2368 off595592 00000004:00080000:0.0:1787739057.936495:0:7944:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730623dc00 with version 681 00000004:00080000:1.0:1787739057.936560:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 681 rpc_version 681 00000020:00100000:1.0:1787739057.936603:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739057.936614:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073043c2940 x1874579665609728/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739057.936637:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c2940 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665609728:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739057.936706:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739057.936740:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739057.938476:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665609728 00000100:00100000:0.0:1787739057.938490:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665609728 00000100:00100000:0.0:1787739057.938555:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2858 00000100:00100000:0.0:1787739057.938566:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665609728:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739057.941752:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665609728:12345-0@lo:1000:osp_up1-0.0 Request processed in 3181us (5038us total) trans 4294968336 rc 0/0 00000100:00100000:0.0:1787739057.941765:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2858 00000100:00100000:2.0:1787739057.941822:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c2940 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665609728:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787739057.941923:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac451 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1507:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.941938:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac451 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1507:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.941948:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac451 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1507:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.941966:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac458 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.941974:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac458 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.941980:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac458 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.942011:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac45f lrc: 3/0,1 mode: --/PW res: [0x240000401:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.942017:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac45f lrc: 3/0,1 mode: --/PW res: [0x240000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739057.942027:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac45f lrc: 3/0,1 mode: --/PW res: [0x240000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739057.942044:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665609856:0@lo:101:mdt00_003.0 00000100:00100000:0.0:1787739057.942089:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739057.942119:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d840 going to sleep for 11 seconds 00000100:00100000:2.0:1787739057.942554:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665609856 00000100:00100000:2.0:1787739057.942561:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665609856 00000100:00100000:2.0:1787739057.942600:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2859 00000100:00100000:2.0:1787739057.942608:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415701680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665609856:12345-0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739057.942624:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739057.942640:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac466 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac45f expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.942660:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac466 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac45f expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.942672:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac466 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac45f expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.942686:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xbe:0x0].0x0 (ffffa07303c78900) 00010000:00010000:2.0:1787739057.942689:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739057.942692:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d200, rc 0) 00000100:00100000:2.0:1787739057.947267:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415701680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:7944:x1874579665609856:12345-0@lo:101:mdt00_003.0 Request processed in 4651us (5170us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739057.947280:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2859 00000100:00100000:2.0:1787739057.947353:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665609856:0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739057.947377:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac45f lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac466 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.947392:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac45f lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac466 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739057.947433:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000aceb9616 reply state 000000009755b70f transno 4294983350 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac458 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.947448:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac458 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.950951:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xbe:0x0].0x0 (ffffa073068a9d00) 00010000:00010000:2.0:1787739057.950956:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739057.950962:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787739057.950975:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac45f lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac466 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.950986:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac45f lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac466 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739057.951004:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xbe:0x0] 00000004:00010000:2.0:1787739057.951012:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000aceb9616 reply state 000000009755b70f transno 4294983350 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac451 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1507:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.951025:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac451 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1507:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.951032:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1507:0x0].0x0 (ffffa073068a9600) 00010000:00010000:2.0:1787739057.951034:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739057.951036:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:2.0:1787739057.951041:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1507:0x0]: returning size 4096 00000004:00080000:2.0:1787739057.951050:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:2.0:1787739057.951066:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac435 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac43c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.951075:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac435 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac43c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.951082:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac435 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac43c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.951107:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739057.951111:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:2.0:1787739057.951115:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac443 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac44a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739057.951122:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac443 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac44a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739057.951215:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+23:17507:x1874579652445440:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 54567us (54747us total) trans 4294983350 rc 0/0 00000100:00100000:2.0:1787739057.951224:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23304 00010000:00010000:3.0:1787739057.951929:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739057.951937:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac435 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac43c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.951947:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739057.951952:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac435 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac43c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739057.951961:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac435 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac43c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.951996:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac435 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac43c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.952003:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739057.952023:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac435 lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac43c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.952030:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac435 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac43c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739057.952039:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665609984:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739057.952046:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739057.952047:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739057.952049:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739057.952100:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739057.952143:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665609984 00000100:00100000:3.0:1787739057.952150:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665609984 00000100:00100000:3.0:1787739057.952199:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1251 00000100:00100000:3.0:1787739057.952207:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304261a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+11:3653:x1874579665609984:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739057.952220:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739057.952235:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac43c lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac435 expref: 11 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739057.952249:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074039f2b00) 00010000:00010000:3.0:1787739057.952256:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739057.952571:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665609984:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739057.956243:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304261a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+10:3653:x1874579665609984:12345-0@lo:103:ldlm_bl.0 Request processed in 94us (200us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739057.956259:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1251 00000100:00100000:2.0:1787739057.993456:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739057.994538:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652445824 00000100:00100000:1.0:1787739057.994552:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652445824 00000100:00100000:1.0:1787739057.994610:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23305 00000100:00100000:1.0:1787739057.994620:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304261e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17508:x1874579652445824:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:1.0:1787739057.994798:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304261e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17508:x1874579652445824:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 178us (1390us total) trans 0 rc -17/-17 00000100:00100000:1.0:1787739057.994809:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23305 00000100:00100000:2.0:1787739058.027484:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739058.027589:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652445952 00000100:00100000:1.0:1787739058.027600:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652445952 00000100:00100000:1.0:1787739058.027662:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23306 00000100:00100000:1.0:1787739058.027672:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304261680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17509:x1874579652445952:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739058.027694:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739058.027731:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac46d lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094d39 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.027808:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac474 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.027824:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac474 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.027846:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac474 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.027856:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac47b lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.027866:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac44a lrc: 2/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac443 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.027875:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac47b lrc: 4/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.027910:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac44a lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ac443 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739058.027931:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c0000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665610112:0@lo:104: 00000100:00100000:1.0:1787739058.027983:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739058.028039:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739058.029300:0:6507:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787739058.029337:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665610112 00000100:00100000:0.0:1787739058.029345:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665610112 00000100:00100000:0.0:1787739058.029410:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1105 00000100:00100000:0.0:1787739058.029480:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f806850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6523:x1874579665610112:12345-0@lo:104: 00010000:00010000:0.0:1787739058.029496:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac443 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac44a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.029581:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787739058.029589:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f806850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6523:x1874579665610112:12345-0@lo:104: Request processed in 134us (1605us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.029597:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1105 00000100:00100000:2.0:1787739058.029621:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c0000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665610112:0@lo:104: 00010000:00010000:2.0:1787739058.029647:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac47b lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.029723:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739058.029732:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac443 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac44a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.029741:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739058.029747:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac443 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ac44a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739058.029760:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac443 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ac44a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.029791:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac443 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac44a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.029798:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739058.029841:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac443 lrc: 1/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac44a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.029849:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ac443 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac44a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.029868:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739058.029870:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739058.029872:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739058.030030:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787739058.030069:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665610240:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739058.030107:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739058.030236:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665610240 00000100:00100000:3.0:1787739058.030241:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665610240 00010000:00010000:3.0:1787739058.030263:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac44a lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ac443 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.030294:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1445 00000100:00100000:3.0:1787739058.030301:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157012c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+10:3651:x1874579665610240:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739058.030311:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:3.0:1787739058.032885:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968336 is committed 00010000:00010000:2.0:1787739058.042672:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac44a lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ac443 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.042709:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x7a393ffc (ffffa07305eae800) 00010000:00010000:2.0:1787739058.042716:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac47b lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.042726:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac47b lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.042734:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac47b lrc: 4/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.042986:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739058.043108:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157012c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+7:3651:x1874579665610240:12345-0@lo:103:ldlm_bl.0 Request processed in 12805us (12998us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739058.043120:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1445 00000100:00100000:1.0:1787739058.043216:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665610240:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739058.043649:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac47b lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.043659:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac47b lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739058.043794:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304260f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665610368:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739058.043855:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739058.043881:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fccf00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739058.044062:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665610368 00000100:00100000:0.0:1787739058.044070:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665610368 00000100:00100000:0.0:1787739058.044123:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2860 00000100:00100000:0.0:1787739058.044132:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665610368:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739058.044222:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665610368:12345-0@lo:1000:mdt00_000.0 Request processed in 91us (367us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.044229:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2860 00000100:00100000:1.0:1787739058.044362:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304260f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665610368:0@lo:1000:mdt00_000.0 00010000:00010000:1.0:1787739058.044410:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac482 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1507:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.044442:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac482 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1507:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.044452:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac482 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1507:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739058.044475:0:6523:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac482 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1507:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.044489:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac474 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.044496:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac474 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.044502:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac474 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.044509:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac474 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.044521:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739058.044525:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac474 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.044532:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac474 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.044543:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305eae300) 00010000:00010000:1.0:1787739058.044546:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739058.044551:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac474 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.045048:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac474 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.045074:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac47b lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.045080:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac47b lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.045086:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac47b lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.045091:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739058.045094:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac47b lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.045100:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac47b lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.045112:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x7a393ffc (ffffa07305eae800) 00010000:00010000:1.0:1787739058.045115:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739058.045118:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac47b lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.045123:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac47b lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.045162:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac46d lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d39 expref: 8 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.045179:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac482 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1507:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d39 expref: 7 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.045186:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1507:0x0].0x0 (ffffa07305eae000) 00010000:00010000:1.0:1787739058.045188:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739058.045191:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1dc00, rc 0) 00000100:00100000:1.0:1787739058.045256:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304261680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17509:x1874579652445952:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 17585us (17782us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.045264:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23306 00000100:00100000:3.0:1787739058.050357:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739058.050650:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652446080 00000100:00100000:2.0:1787739058.050660:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652446080 00000100:00100000:2.0:1787739058.050727:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23307 00000100:00100000:2.0:1787739058.050738:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+23:17509:x1874579652446080:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739058.050763:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739058.050784:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac489 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1507:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094d40 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.050915:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac490 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1507:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.050930:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac490 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1507:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.050943:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac490 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1507:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739058.051040:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1507:0x0]: returning size 4096 00000004:00200000:2.0:1787739058.051116:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739058.051155:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac489 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1507:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d40 expref: 25 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.051179:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac490 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1507:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d40 expref: 24 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.051187:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1507:0x0].0x0 (ffffa073068a9600) 00010000:00010000:2.0:1787739058.051190:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739058.051193:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633da00, rc 0) 00000100:00100000:2.0:1787739058.051292:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+24:17509:x1874579652446080:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 555us (945us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.051299:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23307 00000100:00100000:0.0:1787739058.069840:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739058.069883:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652446208 00000100:00100000:0.0:1787739058.069893:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652446208 00000100:00100000:0.0:1787739058.069947:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23308 00000100:00100000:0.0:1787739058.069958:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415702d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+24:17509:x1874579652446208:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739058.069979:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739058.070000:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac497 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094d47 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.070068:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ac49e lrc: 3/1,0 mode: --/PR res: [0x200000400:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.070081:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ac49e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.070092:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ac49e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739058.070136:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xbe:0x0]: returning size 4096 00000004:00200000:0.0:1787739058.070142:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739058.070174:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac497 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d47 expref: 26 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.070190:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ac49e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d47 expref: 25 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.070199:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xbe:0x0].0x0 (ffffa073068a9d00) 00010000:00010000:0.0:1787739058.070202:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739058.070206:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a800, rc 0) 00000100:00100000:0.0:1787739058.070253:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415702d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+25:17509:x1874579652446208:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 298us (434us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.070260:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23308 00000100:00100000:0.0:1787739058.077889:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739058.078103:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652446336 00000100:00100000:1.0:1787739058.078113:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652446336 00000100:00100000:1.0:1787739058.078209:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23309 00000100:00100000:1.0:1787739058.078220:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157030c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17509:x1874579652446336:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739058.078243:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739058.078262:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac4a5 lrc: 2/0,0 mode: --/CR res: [0x240000401:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094d4e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.078954:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac4ac lrc: 3/1,0 mode: --/PR res: [0x240000401:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.078971:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac466 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac45f expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.078981:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac4ac lrc: 4/1,0 mode: --/PR res: [0x240000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.079015:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac466 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ac45f expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739058.079030:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665610496:0@lo:104: 00000100:00100000:1.0:1787739058.079097:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739058.079139:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ecc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739058.080811:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665610496 00000100:00100000:3.0:1787739058.080834:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665610496 00000100:00100000:3.0:1787739058.080892:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1106 00000100:00100000:3.0:1787739058.080901:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6523:x1874579665610496:12345-0@lo:104: 00010000:00010000:3.0:1787739058.080917:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac45f lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac466 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.081016:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739058.081023:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6523:x1874579665610496:12345-0@lo:104: Request processed in 124us (1926us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739058.081030:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1106 00010000:00010000:3.0:1787739058.081062:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739058.081068:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac45f lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac466 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.081077:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609be00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739058.081082:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac45f lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xbe:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ac466 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739058.081091:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac45f lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xbe:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ac466 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.081125:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac45f lrc: 2/0,0 mode: --/PW res: [0x240000401:0xbe:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac466 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.081132:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739058.081161:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac45f lrc: 1/0,0 mode: --/PW res: [0x240000401:0xbe:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac466 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.081168:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac45f lrc: 0/0,0 mode: --/PW res: [0x240000401:0xbe:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac466 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.081190:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739058.081192:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739058.081193:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739058.081209:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665610624:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739058.081251:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739058.081887:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665610496:0@lo:104: 00010000:00010000:1.0:1787739058.081908:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac4ac lrc: 3/1,0 mode: --/PR res: [0x240000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739058.083917:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665610624 00000100:00100000:1.0:1787739058.083929:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665610624 00010000:00010000:1.0:1787739058.083964:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac466 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ac45f expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739058.084006:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1446 00000100:00100000:1.0:1787739058.084016:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+7:3651:x1874579665610624:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739058.084030:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739058.088753:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac466 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ac45f expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.088775:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xbe:0x0].0x0 (ffffa07303c78900) 00010000:00010000:3.0:1787739058.088781:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac4ac lrc: 3/1,0 mode: --/PR res: [0x240000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.088791:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac4ac lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.088799:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac4ac lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.088838:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739058.088927:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+6:3651:x1874579665610624:12345-0@lo:103:ldlm_bl.0 Request processed in 4912us (7674us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739058.088936:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1446 00000100:00100000:3.0:1787739058.088999:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665610624:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739058.089064:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac4ac lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.089072:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac4ac lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739058.089129:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xbe:0x0]: returning size 4096 00000004:00200000:1.0:1787739058.089135:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739058.089178:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac4a5 lrc: 0/0,0 mode: --/CR res: [0x240000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d4e expref: 9 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.089195:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac4ac lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d4e expref: 8 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.089202:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xbe:0x0].0x0 (ffffa07303c78900) 00010000:00010000:1.0:1787739058.089205:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739058.089208:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c000, rc 0) 00000100:00100000:1.0:1787739058.089264:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157030c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17509:x1874579652446336:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 11046us (11384us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.089271:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23309 00000100:00100000:0.0:1787739058.104263:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739058.104323:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739058.104407:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652446720 00000100:00100000:2.0:1787739058.104444:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652446720 00000100:00100000:2.0:1787739058.104498:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23310 00000100:00100000:2.0:1787739058.104508:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415703480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+25:17509:x1874579652446720:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:2.0:1787739058.104565:0:7944:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739058.104592:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac490 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1507:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d40 expref: 25 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.104613:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1507:0x0].0x0 (ffffa073068a9600) 00010000:00010000:2.0:1787739058.104618:0:7944:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739058.104662:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ac4b3 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.104671:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ac4b3 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.104681:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ac4b3 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739058.104713:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304260f00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665610752:0@lo:101:mdt00_003.0 00000100:00100000:2.0:1787739058.104764:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739058.104802:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.105008:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665610752 00000100:00100000:1.0:1787739058.105013:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665610752 00000100:00100000:1.0:1787739058.105048:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2861 00000100:00100000:1.0:1787739058.105065:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561b840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7944:x1874579665610752:12345-0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739058.105076:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739058.105091:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac4ba lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac4b3 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.105114:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac4ba lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac4b3 expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.105127:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac4ba lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac4b3 expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.105134:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305eae300) 00010000:00010000:1.0:1787739058.105136:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739058.105139:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d600, rc 0) 00000100:00100000:1.0:1787739058.105186:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561b840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665610752:12345-0@lo:101:mdt00_003.0 Request processed in 122us (421us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.105192:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2861 00000100:00100000:0.0:1787739058.105265:0:7944:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968336 gen 2 00000004:00080000:0.0:1787739058.105271:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968335 committed transno 4294968336 00000004:00080000:0.0:1787739058.105290:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968336 committed transno 4294968336 00000100:00100000:0.0:1787739058.105318:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304260f00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665610752:0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739058.105333:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ac4b3 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac4ba expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.105342:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ac4b3 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac4ba expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739058.105355:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739058.105374:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a200/0x7ee4453e4a1ac4c1 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.105380:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a200/0x7ee4453e4a1ac4c1 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.105387:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a200/0x7ee4453e4a1ac4c1 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739058.105398:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415700780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665610880:0@lo:101:mdt00_003.0 00000100:00100000:0.0:1787739058.105448:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.105476:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dc00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739058.105584:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652446464 00000100:00100000:3.0:1787739058.105589:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652446464 00000100:00100000:3.0:1787739058.105632:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1252 00000100:00100000:3.0:1787739058.105640:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415702580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2408:x1874579652446464:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:3.0:1787739058.105651:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739058.105663:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac482 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1507:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d39 expref: 8 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.105676:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1507:0x0].0x0 (ffffa07305eae000) 00010000:00010000:3.0:1787739058.105683:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739058.105732:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415702580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652446464:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 93us (1479us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739058.105739:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1252 00000100:00100000:1.0:1787739058.106184:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665610880 00000100:00100000:1.0:1787739058.106190:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665610880 00000100:00100000:1.0:1787739058.106229:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2862 00000100:00100000:1.0:1787739058.106237:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415700b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665610880:12345-0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739058.106248:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739058.106263:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ac4c8 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac4c1 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.106283:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ac4c8 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac4c1 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.106296:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ac4c8 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac4c1 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.106314:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x7a393ffc (ffffa07305eae800) 00010000:00010000:1.0:1787739058.106317:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739058.106322:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c600, rc 0) 00000100:00100000:1.0:1787739058.106363:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415700b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665610880:12345-0@lo:101:mdt00_003.0 Request processed in 127us (914us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.106370:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2862 00000100:00100000:3.0:1787739058.107044:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415700780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665610880:0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739058.107075:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a200/0x7ee4453e4a1ac4c1 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac4c8 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.107086:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a200/0x7ee4453e4a1ac4c1 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac4c8 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739058.107106:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:3.0:1787739058.107149:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665611008:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739058.107189:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739058.107235:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739058.107608:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665611008 00000100:00100000:0.0:1787739058.107614:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665611008 00000100:00100000:0.0:1787739058.107649:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2863 00000100:00100000:0.0:1787739058.107657:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665611008:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739058.107737:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665611008:12345-0@lo:1000:mdt00_003.0 Request processed in 81us (547us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.107745:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2863 00000100:00100000:3.0:1787739058.107913:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665611008:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739058.107959:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665611136:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739058.107982:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739058.108009:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739058.108059:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665611136 00000100:00100000:2.0:1787739058.108064:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665611136 00000100:00100000:2.0:1787739058.108096:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2864 00000100:00100000:2.0:1787739058.108103:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f783c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665611136:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739058.108182:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f783c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665611136:12345-0@lo:1000:mdt00_003.0 Request processed in 79us (198us total) trans 0 rc 1/1 00000100:00100000:2.0:1787739058.108188:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2864 00000100:00100000:3.0:1787739058.108242:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c380000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665611136:0@lo:1000:mdt00_003.0 00010000:00010000:3.0:1787739058.108288:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac4cf lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.108311:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac4cf lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.108321:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac4cf lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.108333:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f79a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665611264:0@lo:101:mdt00_003.0 00000100:00100000:3.0:1787739058.108360:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739058.108387:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e890c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.108446:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665611264 00000100:00100000:1.0:1787739058.108450:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665611264 00000100:00100000:1.0:1787739058.108477:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2865 00000100:00100000:1.0:1787739058.108483:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07436d8d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665611264:12345-0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739058.108491:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739058.108505:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac4d6 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac4cf expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.108526:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac4d6 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac4cf expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.108539:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac4d6 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac4cf expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.108546:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1507:0x0].0x0 (ffffa07305eaec00) 00010000:00010000:1.0:1787739058.108548:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739058.108552:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1cc00, rc 0) 00000100:00100000:1.0:1787739058.108583:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07436d8d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665611264:12345-0@lo:101:mdt00_003.0 Request processed in 100us (223us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.108589:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2865 00000100:00100000:3.0:1787739058.109458:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f79a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665611264:0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739058.109621:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac4cf lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac4d6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.109635:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac4cf lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac4d6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.109958:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac4dd lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.109970:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac4dd lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.109978:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac4dd lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.109996:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac4e4 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.110007:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ac49e lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d47 expref: 24 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.110016:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac4e4 lrc: 4/0,1 mode: --/EX res: [0x200000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.110043:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ac49e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d47 expref: 24 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.110066:0:7944:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ac49e lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d47 expref: 24 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739058.110078:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665611392:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739058.110147:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc540 going to sleep for 11 seconds 00000100:00100000:0.0:1787739058.115745:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665611392:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739058.115771:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac4e4 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739058.119795:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739058.119866:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652446848 00000100:00100000:3.0:1787739058.119875:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652446848 00010000:00010000:3.0:1787739058.119913:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ac49e lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d47 expref: 25 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.119959:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1447 00000100:00100000:3.0:1787739058.119969:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415701e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+25:2408:x1874579652446848:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739058.119986:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739058.119994:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ac49e lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d47 expref: 25 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.120008:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ac49e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d47 expref: 25 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.120031:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ac49e lrc: 0/0,0 mode: --/PR res: [0x200000400:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d47 expref: 25 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.120046:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xbe:0x0].0x0 (ffffa073068a9d00) 00010000:00010000:3.0:1787739058.120060:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac4e4 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.120069:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac4e4 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.120076:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac4e4 lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.120107:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739058.120159:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415701e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+24:2408:x1874579652446848:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 192us (373us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739058.120168:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1447 00010000:00010000:0.0:1787739058.120256:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac4e4 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.120265:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac4e4 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.120497:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b600/0x7ee4453e4a1ac4eb lrc: 3/0,1 mode: --/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.120505:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b600/0x7ee4453e4a1ac4eb lrc: 3/0,1 mode: --/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.120515:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b600/0x7ee4453e4a1ac4eb lrc: 3/0,1 mode: --/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739058.120532:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415701680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665611520:0@lo:101:mdt00_003.0 00000100:00100000:0.0:1787739058.120619:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.120684:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623df00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739058.128004:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665611520 00000100:00100000:2.0:1787739058.128020:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665611520 00000100:00100000:2.0:1787739058.128078:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2866 00000100:00100000:2.0:1787739058.128088:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415700b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665611520:12345-0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739058.128109:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739058.128130:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac4f2 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac4eb expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.128161:0:8448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac4ac lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d4e expref: 6 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.128172:0:8448:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac4f2 lrc: 4/0,0 mode: --/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac4eb expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.128205:0:8448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac4ac lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d4e expref: 6 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.128219:0:8448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac4ac lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d4e expref: 6 pid: 6523 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739058.128233:0:8448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561b480 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8448:1874579665611648:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739058.128979:0:8448:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc000 going to sleep for 11 seconds 00000100:00100000:0.0:1787739058.138004:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739058.138026:0:8448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561b480 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8448:1874579665611648:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739058.138069:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac4f2 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac4eb expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.138079:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652446976 00010000:00010000:2.0:1787739058.138083:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xbe:0x0].0x0 (ffffa07303c78900) 00000100:00100000:3.0:1787739058.138087:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652446976 00010000:00010000:2.0:1787739058.138088:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739058.138093:0:8448:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac4f2 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac4eb expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.138105:0:8448:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac4f2 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac4eb expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.138111:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633da00, rc 0) 00010000:00010000:3.0:1787739058.138125:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac4ac lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d4e expref: 7 pid: 6523 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.138166:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1448 00000100:00100000:2.0:1787739058.138175:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415700b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665611520:12345-0@lo:101:mdt00_003.0 Request processed in 10087us (17553us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739058.138179:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074184c7850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2409:x1874579652446976:12345-192.168.203.15@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1787739058.138187:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2866 00010000:00010000:3.0:1787739058.138197:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739058.138204:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac4ac lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d4e expref: 7 pid: 6523 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.138216:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac4ac lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d4e expref: 7 pid: 6523 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.138236:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac4ac lrc: 0/0,0 mode: --/PR res: [0x240000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d4e expref: 7 pid: 6523 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.138249:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xbe:0x0].0x0 (ffffa07303c78900) 00010000:00010000:3.0:1787739058.138254:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac4f2 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac4eb expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739058.138258:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415701680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665611520:0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739058.138270:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac4f2 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac4eb expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.138279:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b600/0x7ee4453e4a1ac4eb lrc: 4/0,1 mode: --/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac4f2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.138280:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac4f2 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac4eb expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.138314:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b600/0x7ee4453e4a1ac4eb lrc: 4/0,1 mode: --/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac4f2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.138321:0:6508:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac4f2 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac4eb expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.138335:0:6508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f79a40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001_UUID:6508:1874579665611776:0@lo:105: 00000100:00100000:3.0:1787739058.138369:0:6508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739058.138461:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665611776 00000100:00100000:3.0:1787739058.138465:0:6508:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e890c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.138466:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665611776 00000100:00100000:1.0:1787739058.138500:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1107 00000100:00100000:1.0:1787739058.138507:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6508:x1874579665611776:12345-0@lo:105: 00010000:00010000:1.0:1787739058.138516:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b600/0x7ee4453e4a1ac4eb lrc: 5/0,1 mode: --/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac4f2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.138525:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b600/0x7ee4453e4a1ac4eb lrc: 5/0,1 mode: --/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac4f2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.138534:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b600/0x7ee4453e4a1ac4eb lrc: 5/0,1 mode: --/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac4f2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.138540:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b600/0x7ee4453e4a1ac4eb lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac4f2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.138548:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b600/0x7ee4453e4a1ac4eb lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac4f2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.138555:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b600/0x7ee4453e4a1ac4eb lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ac4f2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.138577:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609b600) 00000100:00100000:1.0:1787739058.138609:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6508:x1874579665611776:12345-0@lo:105: Request processed in 103us (240us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.138617:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1107 00000100:00100000:3.0:1787739058.138677:0:6508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f79a40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001_UUID:6508:1874579665611776:0@lo:105: 00010000:00010000:3.0:1787739058.138693:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739058.138747:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074184c7850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2409:x1874579652446976:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 571us (753us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739058.138754:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1448 00010000:00010000:3.0:1787739058.138805:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b600/0x7ee4453e4a1ac4eb lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac4f2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.138814:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b600/0x7ee4453e4a1ac4eb lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac4f2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.140248:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f79a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665611904:0@lo:403:mdt00_003.0 00000100:00100000:3.0:1787739058.140306:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739058.140392:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665611904 00000100:00100000:3.0:1787739058.140398:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739058.140399:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665611904 00000100:00100000:2.0:1787739058.140477:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2867 00000100:00100000:2.0:1787739058.140485:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f78b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665611904:12345-0@lo:403:mdt00_003.0 00000100:00100000:2.0:1787739058.143391:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f78b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665611904:12345-0@lo:403:mdt00_003.0 Request processed in 2811us (2988us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.143404:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2867 00000100:00100000:3.0:1787739058.143507:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f79a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665611904:0@lo:403:mdt00_003.0 00000040:00080000:2.0:1787739058.147136:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].359, 664 off550352 00000040:00080000:2.0:1787739058.147349:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].359, 744 off596336 00000004:00080000:2.0:1787739058.147357:0:7944:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07302e89f00 with version 682 00000004:00080000:3.0:1787739058.150983:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 682 rpc_version 682 00000020:00100000:2.0:1787739058.151577:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739058.151591:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0730561b480 x1874579665612160/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2064/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739058.151616:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561b480 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665612160:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739058.151691:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739058.151791:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.151855:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665612160 00000100:00100000:1.0:1787739058.151862:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665612160 00000100:00100000:1.0:1787739058.151918:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2868 00000100:00100000:1.0:1787739058.151928:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305618f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665612160:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739058.152349:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305618f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:7740:x1874579665612160:12345-0@lo:1000:osp_up1-0.0 Request processed in 422us (658us total) trans 4294968337 rc 0/0 00000100:00100000:1.0:1787739058.152357:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2868 00000100:00100000:2.0:1787739058.152638:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561b480 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665612160:0@lo:1000:osp_up1-0.0 00000004:00200000:1.0:1787739058.152744:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1507:0x0]: returning size 4096 00010000:00010000:1.0:1787739058.152765:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac4e4 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.152774:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac4e4 lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.152781:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac4e4 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.152788:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739058.152792:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac4e4 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.152800:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac4e4 lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.152823:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xbe:0x0].0x0 (ffffa073068a9d00) 00010000:00010000:1.0:1787739058.152826:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739058.152830:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac4e4 lrc: 2/0,0 mode: --/EX res: [0x200000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.152837:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac4e4 lrc: 0/0,0 mode: --/EX res: [0x200000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.152855:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b600/0x7ee4453e4a1ac4eb lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac4f2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.152863:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b600/0x7ee4453e4a1ac4eb lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac4f2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.152870:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b600/0x7ee4453e4a1ac4eb lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac4f2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.152884:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739058.152892:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xbe:0x0] 00010000:00010000:1.0:1787739058.152897:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac4dd lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.152903:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac4dd lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.152908:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac4dd lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.152914:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739058.152916:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac4dd lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.152922:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac4dd lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.152928:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1507:0x0].0x0 (ffffa073068a9600) 00010000:00010000:1.0:1787739058.152931:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739058.152934:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac4dd lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.152940:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac4dd lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.152948:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac4cf lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac4d6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.152955:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac4cf lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac4d6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.152962:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac4cf lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac4d6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.152981:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:1.0:1787739058.152989:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739058.152994:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ac4b3 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac4ba expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.153000:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ac4b3 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac4ba expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.153007:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ac4b3 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac4ba expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.153019:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:1.0:1787739058.153022:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739058.153026:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a200/0x7ee4453e4a1ac4c1 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac4c8 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.153039:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a200/0x7ee4453e4a1ac4c1 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac4c8 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739058.153104:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415703480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+24:17509:x1874579652446720:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 48597us (48779us total) trans 4294983351 rc 0/0 00000100:00100000:1.0:1787739058.153112:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23310 00010000:00010000:2.0:1787739058.153201:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1787739058.153207:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b600/0x7ee4453e4a1ac4eb lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac4f2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.153214:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739058.153219:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b600/0x7ee4453e4a1ac4eb lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac4f2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739058.153227:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b600/0x7ee4453e4a1ac4eb lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac4f2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.153251:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b600/0x7ee4453e4a1ac4eb lrc: 2/0,0 mode: --/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac4f2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.153257:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739058.153281:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b600/0x7ee4453e4a1ac4eb lrc: 1/0,0 mode: --/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac4f2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.153288:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b600/0x7ee4453e4a1ac4eb lrc: 0/0,0 mode: --/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac4f2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.153321:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1787739058.153324:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac4cf lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac4d6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.153331:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739058.153334:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac4cf lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac4d6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739058.153341:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac4cf lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac4d6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.153354:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac4cf lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac4d6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.153360:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739058.153365:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac4cf lrc: 1/0,0 mode: --/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac4d6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.153371:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1ac4cf lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac4d6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.153381:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739058.153384:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ac4b3 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac4ba expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.153390:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633cc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739058.153393:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ac4b3 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac4ba expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739058.153400:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ac4b3 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac4ba expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.153433:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ac4b3 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac4ba expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.153439:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739058.153452:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ac4b3 lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac4ba expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.153458:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ac4b3 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac4ba expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.153470:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739058.153471:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739058.153473:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739058.153483:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561b840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665612416:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739058.153519:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739058.153548:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787739058.153553:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665612288:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739058.153570:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739058.153589:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060043c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665612544:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739058.153605:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739058.153630:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665612416 00000100:00100000:2.0:1787739058.153634:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665612416 00000100:00100000:2.0:1787739058.153664:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665612288 00000100:00100000:2.0:1787739058.153666:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665612288 00000100:00100000:2.0:1787739058.153677:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665612544 00000100:00100000:2.0:1787739058.153679:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665612544 00000100:00100000:2.0:1787739058.153690:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1253 00000100:00100000:2.0:1787739058.153696:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306005e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+13:3653:x1874579665612416:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739058.153706:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739058.153721:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac4d6 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1507:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac4cf expref: 13 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.153732:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1507:0x0].0x0 (ffffa07305eaec00) 00010000:00010000:2.0:1787739058.153739:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739058.153762:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306005e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+12:3653:x1874579665612416:12345-0@lo:103:ldlm_bl.0 Request processed in 66us (242us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.153768:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1253 00000100:00100000:2.0:1787739058.153779:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1254 00000100:00100000:2.0:1787739058.153784:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060070c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+11:3653:x1874579665612288:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739058.153790:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739058.153799:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac4f2 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac4eb expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.153819:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xbe:0x0].0x0 (ffffa07303c78900) 00010000:00010000:2.0:1787739058.153825:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739058.153843:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060070c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+10:3653:x1874579665612288:12345-0@lo:103:ldlm_bl.0 Request processed in 59us (273us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.153848:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1254 00000100:00100000:2.0:1787739058.153858:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1255 00000100:00100000:2.0:1787739058.153863:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306007480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+9:3650:x1874579665612544:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739058.153869:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739058.153878:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac4ba lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac4b3 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.153887:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305eae300) 00010000:00010000:2.0:1787739058.153892:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739058.153911:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306007480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+8:3650:x1874579665612544:12345-0@lo:103:ldlm_bl.0 Request processed in 48us (305us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.153916:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1255 00000100:00100000:2.0:1787739058.153942:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060043c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665612544:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739058.153958:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665612288:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739058.153965:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561b840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665612416:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739058.154380:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739058.154386:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739058.186973:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739058.187152:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652447104 00000100:00100000:1.0:1787739058.187162:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652447104 00000100:00100000:1.0:1787739058.187258:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23311 00000100:00100000:1.0:1787739058.187268:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f79a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+24:17510:x1874579652447104:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739058.187292:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739058.187323:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac4f9 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094d55 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.187386:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac500 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.187400:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac500 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.187445:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac500 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.187461:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac507 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.187468:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac507 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.187479:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac507 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.187520:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac500 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.187528:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac500 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.187535:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac500 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.187541:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac500 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.187548:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1de00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739058.187552:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac500 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.187560:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac500 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.187573:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9c00) 00010000:00010000:1.0:1787739058.187576:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739058.187581:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac500 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.187587:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac500 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.187600:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac507 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.187606:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac507 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.187612:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac507 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.187617:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739058.187621:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac507 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.187626:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac507 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.187634:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x83604406 (ffffa07305eaed00) 00010000:00010000:1.0:1787739058.187637:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739058.187639:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac507 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.187645:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac507 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.187676:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac4f9 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d55 expref: 25 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.187687:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9c00) 00010000:00010000:1.0:1787739058.187689:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739058.187693:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac4f9 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d55 expref: 25 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.187710:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1ce00, rc 0) 00000100:00100000:1.0:1787739058.187771:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f79a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+24:17510:x1874579652447104:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 504us (807us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739058.187778:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23311 00000100:00100000:0.0:1787739058.196864:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739058.196919:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652447232 00000100:00100000:2.0:1787739058.196928:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652447232 00000100:00100000:0.0:1787739058.197012:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23312 00000100:00100000:0.0:1787739058.197024:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415700780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17510:x1874579652447232:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739058.197044:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739058.197076:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac50e lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094d5c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.197136:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac515 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.197150:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac515 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.197161:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac515 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739058.197204:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x5:0x0]: returning size 4096 00000004:00200000:0.0:1787739058.197209:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739058.197243:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac50e lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d5c expref: 8 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.197259:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac515 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d5c expref: 7 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.197267:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9e00) 00010000:00010000:0.0:1787739058.197270:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739058.197273:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b800, rc 0) 00000100:00100000:0.0:1787739058.197469:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415700780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17510:x1874579652447232:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 448us (613us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.197476:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23312 00000100:00100000:2.0:1787739058.207885:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739058.207925:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652447488 00000100:00100000:1.0:1787739058.207935:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652447488 00000100:00100000:1.0:1787739058.207993:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23313 00000100:00100000:1.0:1787739058.208003:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060043c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17510:x1874579652447488:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:1.0:1787739058.208123:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c1a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665612672:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.208173:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739058.208218:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610eb40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739058.208265:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665612672 00000100:00100000:0.0:1787739058.208270:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665612672 00000100:00100000:0.0:1787739058.208317:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2869 00000100:00100000:0.0:1787739058.208324:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c1e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665612672:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739058.208398:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c1e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665612672:12345-0@lo:1000:mdt00_003.0 Request processed in 74us (224us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.208405:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2869 00000100:00100000:1.0:1787739058.208576:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c1a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665612672:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.208618:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c1a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665612800:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.208642:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739058.208670:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610eb40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739058.208762:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665612800 00000100:00100000:2.0:1787739058.208767:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665612800 00000100:00100000:2.0:1787739058.208799:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2870 00000100:00100000:2.0:1787739058.208806:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c2580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665612800:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739058.208891:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c2580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665612800:12345-0@lo:1000:mdt00_003.0 Request processed in 85us (248us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739058.208897:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2870 00000100:00100000:0.0:1787739058.208963:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c1a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665612800:0@lo:1000:mdt00_003.0 00010000:00010000:0.0:1787739058.209009:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ac51c lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.209020:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ac51c lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.209030:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ac51c lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739058.209042:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665612928:0@lo:101:mdt00_003.0 00000100:00100000:0.0:1787739058.209080:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.209101:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739058.209123:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665612928 00000100:00100000:2.0:1787739058.209128:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665612928 00000100:00100000:2.0:1787739058.209158:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2871 00000100:00100000:2.0:1787739058.209165:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415701680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665612928:12345-0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739058.209174:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739058.209190:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac523 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac51c expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.209268:0:18535:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac42e lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d32 expref: 23 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.209279:0:18535:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac523 lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac51c expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.209303:0:18535:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac42e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d32 expref: 23 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.209316:0:18535:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac42e lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d32 expref: 23 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739058.209327:0:18535:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306006940 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:18535:1874579665613056:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739058.209444:0:18535:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fcc6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739058.224117:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739058.224144:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652447616 00000100:00100000:2.0:1787739058.224152:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652447616 00010000:00010000:2.0:1787739058.224186:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac42e lrc: 5/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d32 expref: 24 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739058.224217:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1449 00000100:00100000:2.0:1787739058.224225:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306006d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+24:2408:x1874579652447616:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739058.224242:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739058.224248:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac42e lrc: 5/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d32 expref: 24 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.224262:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac42e lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d32 expref: 24 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.224285:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9c00) 00010000:00010000:2.0:1787739058.224292:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac523 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac51c expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.224304:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac523 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac51c expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.224313:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac523 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac51c expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.224343:0:10340:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac523 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac51c expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739058.224358:0:10340:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060061c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:10340:1874579665613184:0@lo:105: 00000100:00100000:2.0:1787739058.224407:0:10340:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739058.224460:0:10340:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07305fccc00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739058.231936:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665613184 00000100:00100000:3.0:1787739058.231951:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665613184 00000100:00100000:3.0:1787739058.232336:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1108 00000100:00100000:3.0:1787739058.232348:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306006580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:10340:x1874579665613184:12345-0@lo:105: 00010000:00010000:3.0:1787739058.232367:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ac51c lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac523 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.232379:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ac51c lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac523 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.232394:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ac51c lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac523 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.232405:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ac51c lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac523 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.232584:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ac51c lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac523 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.232595:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ac51c lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ac523 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.232613:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609bc00) 00000100:00100000:3.0:1787739058.233956:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306006580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:10340:x1874579665613184:12345-0@lo:105: Request processed in 1604us (9541us total) trans 0 rc 0/0 00010000:00010000:0.0:1787739058.233965:0:18535:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac42e lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d32 expref: 24 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.233969:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1108 00000100:00100000:0.0:1787739058.233983:0:18535:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306006940 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:18535:1874579665613056:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739058.234011:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac523 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac51c expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.234022:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9c00) 00010000:00010000:0.0:1787739058.234025:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739058.234028:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c000, rc 0) 00000100:00100000:0.0:1787739058.234095:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415701680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665612928:12345-0@lo:101:mdt00_003.0 Request processed in 24930us (25014us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.234106:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2871 00000100:00100000:0.0:1787739058.234173:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665612928:0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739058.234189:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ac51c lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac523 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739058.234210:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739058.234249:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac52a lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.234257:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac52a lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.234266:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac52a lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739058.234283:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665613312:0@lo:101:mdt00_003.0 00000100:00100000:0.0:1787739058.234331:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.234356:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739058.234371:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665613312 00000100:00100000:0.0:1787739058.234377:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665613312 00000100:00100000:0.0:1787739058.234410:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2872 00000100:00100000:0.0:1787739058.234436:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157012c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665613312:12345-0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739058.234449:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739058.234465:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac531 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac52a expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.234486:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac531 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac52a expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.234503:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac531 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac52a expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.234511:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x83604406 (ffffa073068a9900) 00010000:00010000:0.0:1787739058.234513:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739058.234515:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b600, rc 0) 00000100:00100000:0.0:1787739058.234541:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157012c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665613312:12345-0@lo:101:mdt00_003.0 Request processed in 108us (211us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.234547:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2872 00000100:00100000:0.0:1787739058.234569:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665613312:0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739058.234578:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac52a lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac531 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.234586:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac52a lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac531 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739058.234595:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:0.0:1787739058.234620:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665613440:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739058.234641:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.234654:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.235093:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665613440 00000100:00100000:1.0:1787739058.235099:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665613440 00000100:00100000:1.0:1787739058.235136:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2873 00000100:00100000:1.0:1787739058.235144:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415702580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665613440:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.235212:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415702580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665613440:12345-0@lo:1000:mdt00_003.0 Request processed in 69us (568us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.235219:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2873 00000100:00100000:0.0:1787739058.235260:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665613440:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739058.235429:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665613568:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739058.235456:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.235475:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.236015:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665613568 00000100:00100000:1.0:1787739058.236021:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665613568 00000100:00100000:1.0:1787739058.236067:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2874 00000100:00100000:1.0:1787739058.236074:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415701e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665613568:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.236162:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415701e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665613568:12345-0@lo:1000:mdt00_003.0 Request processed in 88us (704us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739058.236170:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2874 00000100:00100000:0.0:1787739058.236186:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665613568:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739058.236261:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665613696:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739058.236287:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.236321:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.236436:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665613696 00000100:00100000:1.0:1787739058.236440:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665613696 00000100:00100000:1.0:1787739058.236465:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2875 00000100:00100000:1.0:1787739058.236471:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304261a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665613696:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.236521:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304261a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665613696:12345-0@lo:1000:mdt00_003.0 Request processed in 50us (233us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739058.236527:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2875 00000100:00100000:0.0:1787739058.236597:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665613696:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739058.236642:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304263480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665613824:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739058.236665:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.236688:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.236786:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665613824 00000100:00100000:1.0:1787739058.236790:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665613824 00000100:00100000:0.0:1787739058.236842:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2876 00000100:00100000:0.0:1787739058.236849:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042612c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665613824:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739058.236859:0:10340:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060061c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:10340:1874579665613184:0@lo:105: 00010000:00010000:2.0:1787739058.236872:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739058.236920:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306006d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+23:2408:x1874579652447616:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 12694us (12808us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.236928:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1449 00000100:00100000:0.0:1787739058.238327:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042612c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665613824:12345-0@lo:1000:mdt00_003.0 Request processed in 1473us (1655us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.238337:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2876 00000100:00100000:0.0:1787739058.238399:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304263480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665613824:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739058.238473:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304263480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665613952:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739058.238515:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.238537:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.238653:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665613952 00000100:00100000:1.0:1787739058.238659:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665613952 00000100:00100000:1.0:1787739058.238700:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2877 00000100:00100000:1.0:1787739058.238708:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304263840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665613952:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.238781:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304263840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665613952:12345-0@lo:1000:mdt00_003.0 Request processed in 74us (265us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.238788:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2877 00000100:00100000:0.0:1787739058.238914:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304263480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665613952:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739058.238953:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304263480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665614080:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739058.238977:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.238998:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.239039:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665614080 00000100:00100000:1.0:1787739058.239043:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665614080 00000100:00100000:1.0:1787739058.239088:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2878 00000100:00100000:1.0:1787739058.239095:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042621c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665614080:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.239145:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042621c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665614080:12345-0@lo:1000:mdt00_003.0 Request processed in 50us (167us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.239151:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2878 00000100:00100000:0.0:1787739058.239279:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304263480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665614080:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739058.239327:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304263480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665614208:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739058.239353:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.239376:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.239527:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665614208 00000100:00100000:1.0:1787739058.239531:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665614208 00000100:00100000:1.0:1787739058.239558:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2879 00000100:00100000:1.0:1787739058.239566:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304260b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665614208:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.239614:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304260b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665614208:12345-0@lo:1000:mdt00_003.0 Request processed in 49us (260us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.239620:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2879 00000100:00100000:0.0:1787739058.239689:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304263480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665614208:0@lo:1000:mdt00_003.0 00000040:00080000:0.0:1787739058.240776:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].343, 2168 off522872 00000040:00080000:0.0:1787739058.240855:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].344, 2368 off572160 00000004:00080000:0.0:1787739058.240861:0:7944:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa073068ba3c0 with version 656 00000004:00080000:1.0:1787739058.240921:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 656 rpc_version 656 00000020:00100000:1.0:1787739058.240951:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739058.240959:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073043c3c00 x1874579665614464/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739058.240977:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c3c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665614464:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739058.241030:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739058.241069:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.241088:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665614464 00000100:00100000:1.0:1787739058.241092:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665614464 00000100:00100000:1.0:1787739058.241124:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2880 00000100:00100000:1.0:1787739058.241130:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665614464:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739058.241656:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:7467:x1874579665614464:12345-0@lo:1000:osp_up0-1.0 Request processed in 524us (624us total) trans 4294983352 rc 0/0 00000100:00100000:1.0:1787739058.241664:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2880 00000100:00100000:1.0:1787739058.241728:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c3c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665614464:0@lo:1000:osp_up0-1.0 00010000:00010000:3.0:1787739058.241901:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac538 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x16b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.241917:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac538 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x16b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.241927:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac538 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x16b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.241946:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac53f lrc: 3/0,1 mode: --/PW res: [0x240000400:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.241953:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac53f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.241960:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac53f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.241999:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac546 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.242005:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac546 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.242015:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac546 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.242027:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7a580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665614592:0@lo:101:mdt00_003.0 00000100:00100000:3.0:1787739058.242069:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739058.242104:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074062f3b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739058.242648:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665614592 00000100:00100000:0.0:1787739058.242655:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665614592 00000100:00100000:0.0:1787739058.242692:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2881 00000100:00100000:0.0:1787739058.242700:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:7944:x1874579665614592:12345-0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739058.242716:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739058.242732:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ac54d lrc: 2/0,0 mode: --/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac546 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.242752:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ac54d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac546 expref: 12 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.242763:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ac54d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac546 expref: 12 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.242772:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbc:0x0].0x0 (ffffa073068a9100) 00010000:00010000:0.0:1787739058.242775:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739058.242777:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a800, rc 0) 00000100:00100000:0.0:1787739058.242817:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:7944:x1874579665614592:12345-0@lo:101:mdt00_003.0 Request processed in 118us (746us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.242823:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2881 00000100:00100000:0.0:1787739058.242864:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7a580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665614592:0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739058.242877:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac546 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac54d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.242888:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac546 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac54d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739058.242906:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ffe20bdd reply state 00000000c1d58b26 transno 4294968338 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac53f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.242917:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac53f lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.242923:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xbc:0x0].0x0 (ffffa07305d56500) 00010000:00010000:0.0:1787739058.242925:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739058.242927:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739058.242932:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac546 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac54d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.242939:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac546 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac54d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739058.242948:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xbc:0x0] 00000004:00010000:0.0:1787739058.242954:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ffe20bdd reply state 00000000c1d58b26 transno 4294968338 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac538 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x16b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.242961:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac538 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x16b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.242966:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16b:0x0].0x0 (ffffa07305d56400) 00010000:00010000:0.0:1787739058.242968:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739058.242970:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787739058.242974:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x16b:0x0]: returning size 4096 00000004:00080000:0.0:1787739058.242981:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739058.242990:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ac51c lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac523 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.242998:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ac51c lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac523 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.243004:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ac51c lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac523 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.243021:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739058.243025:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739058.243029:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac52a lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac531 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.243035:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac52a lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac531 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.249644:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739058.249851:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ac51c lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac523 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.249867:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609bc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739058.249874:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ac51c lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac523 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739058.249886:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ac51c lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac523 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.249941:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ac51c lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac523 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.249954:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739058.250952:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ac51c lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac523 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.250965:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1ac51c lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac523 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.251005:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739058.251007:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739058.251010:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739058.251034:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787739058.251064:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f783c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665614720:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739058.251137:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739058.251199:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665614720 00000100:00100000:3.0:1787739058.251206:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665614720 00000100:00100000:3.0:1787739058.251260:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1256 00000100:00100000:3.0:1787739058.251270:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7a580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+12:3650:x1874579665614720:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739058.251285:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739058.251305:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac523 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac51c expref: 12 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.251321:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9c00) 00010000:00010000:3.0:1787739058.251331:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739058.251368:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7a580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3650:x1874579665614720:12345-0@lo:103:ldlm_bl.0 Request processed in 99us (231us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739058.251376:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1256 00000100:00100000:3.0:1787739058.251437:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f783c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665614720:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739058.251875:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060043c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17510:x1874579652447488:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 43869us (43996us total) trans 4294968338 rc 0/0 00000100:00100000:0.0:1787739058.251884:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23313 00000100:00100000:0.0:1787739058.305035:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739058.305186:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652447872 00000100:00100000:1.0:1787739058.305194:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652447872 00000100:00100000:1.0:1787739058.305258:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23314 00000100:00100000:1.0:1787739058.305269:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304263480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+23:17511:x1874579652447872:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:1.0:1787739058.306117:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304263480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+23:17511:x1874579652447872:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 847us (1085us total) trans 0 rc -17/-17 00000100:00100000:1.0:1787739058.306130:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23314 00000100:00100000:3.0:1787739058.321397:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739058.323011:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652448000 00000100:00100000:0.0:1787739058.323024:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652448000 00000100:00100000:0.0:1787739058.323100:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23315 00000100:00100000:0.0:1787739058.323110:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f783c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+23:17512:x1874579652448000:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739058.323401:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739058.323460:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ac554 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094d63 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.323528:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac55b lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.323541:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac55b lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.323552:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac55b lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.323563:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac562 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.323572:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac531 lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac52a expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.323580:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac562 lrc: 4/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.323609:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac531 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ac52a expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739058.323625:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304262580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665614848:0@lo:104: 00000100:00100000:0.0:1787739058.323781:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.323833:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068baa80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739058.324084:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665614848 00000100:00100000:2.0:1787739058.324090:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665614848 00000100:00100000:2.0:1787739058.324131:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1109 00000100:00100000:2.0:1787739058.324139:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665614848:12345-0@lo:104: 00010000:00010000:2.0:1787739058.324152:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac52a lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac531 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.324215:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739058.324221:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665614848:12345-0@lo:104: Request processed in 83us (543us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.324228:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1109 00010000:00010000:2.0:1787739058.324250:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739058.324256:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac52a lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac531 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.324264:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b200 already unused, calling callback (ffffffffc18e6ee0) 00000100:00100000:0.0:1787739058.324266:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304262580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665614848:0@lo:104: 00010000:00010000:2.0:1787739058.324269:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac52a lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ac531 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739058.324277:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac52a lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ac531 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.324284:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac562 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.324317:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac52a lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac531 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.324323:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739058.324344:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac52a lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac531 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.324351:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1ac52a lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac531 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.324371:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739058.324373:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739058.324374:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739058.324384:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060061c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665614976:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739058.324432:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739058.324469:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665614976 00000100:00100000:2.0:1787739058.324473:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665614976 00010000:00010000:2.0:1787739058.324489:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac531 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ac52a expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739058.324510:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1450 00000100:00100000:2.0:1787739058.324516:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+11:3652:x1874579665614976:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739058.324526:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739058.329058:0:6502:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a200/0x7ee4453e4a1ac4c1 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac4c8 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739058.329077:0:6502:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a200/0x7ee4453e4a1ac4c1 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac4c8 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.329117:0:6502:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a200/0x7ee4453e4a1ac4c1 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac4c8 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.329124:0:6502:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:3.0:1787739058.329181:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665615104:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:3.0:1787739058.329247:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.329343:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665615104 00000100:00100000:0.0:1787739058.329350:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665615104 00000100:00100000:0.0:1787739058.329399:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1257 00000100:00100000:0.0:1787739058.329409:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074184c5850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+8:3653:x1874579665615104:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:0.0:1787739058.329443:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739058.329463:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ac4c8 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac4c1 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.329476:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x7a393ffc (ffffa07305eae800) 00010000:00010000:0.0:1787739058.329486:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739058.329550:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a200/0x7ee4453e4a1ac4c1 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x7a393ffc bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac4c8 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.329568:0:6502:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1ac355 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac35c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739058.329576:0:6502:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1ac355 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac35c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.329593:0:6502:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1ac355 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac35c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.329600:0:6502:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739058.329614:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1ac355 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac35c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787739058.329625:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983352 is committed 00000001:00080000:2.0:1787739058.329641:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983351 is committed 00000020:00080000:2.0:1787739058.329650:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa0730623d900 refcount 2 committed 0 result 0 batchid 360 00000020:00080000:2.0:1787739058.329653:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88f00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730623dc00 00000020:00080000:2.0:1787739058.329656:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].358 00000020:00080000:2.0:1787739058.329659:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88600 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa0730541fa00 00000020:00080000:2.0:1787739058.329661:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].358 00000020:00080000:2.0:1787739058.329675:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07305fcce40 refcount 2 committed 0 result 0 batchid 359 00000020:00080000:2.0:1787739058.329678:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730415fc00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07305fccc00 00000020:00080000:2.0:1787739058.329680:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].357 00000020:00080000:2.0:1787739058.329682:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c691e00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304277e00 00000020:00080000:2.0:1787739058.329686:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].357 00000020:00080000:2.0:1787739058.329689:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 357 batchid updated 00000020:00080000:2.0:1787739058.329706:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa0743c0ef900 refcount 2 committed 0 result 0 batchid 358 00000020:00080000:2.0:1787739058.329710:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88800 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa0743c0efc00 00000020:00080000:2.0:1787739058.329713:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].356 00000020:00080000:2.0:1787739058.329715:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88900 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073040e8a00 00000020:00080000:2.0:1787739058.329718:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].356 00000100:00100000:2.0:1787739058.329739:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306005680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665615232:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787739058.329768:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000020:00080000:2.0:1787739058.329803:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 360 committed batchid 357 00000004:00080000:2.0:1787739058.329900:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07303c6eb40 with version 683 00000020:00080000:2.0:1787739058.329909:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 357 cancel update log [0x240000401:0x3:0x0].354: rc = 0 00000020:00080000:2.0:1787739058.329948:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 357 cancel update log [0x200000400:0x3:0x0].355: rc = 0 00000020:00080000:2.0:1787739058.329956:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 357 00000100:00100000:2.0:1787739058.329968:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665615232 00000100:00100000:2.0:1787739058.329972:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665615232 00000100:00100000:2.0:1787739058.330001:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1258 00000100:00100000:2.0:1787739058.330008:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+8:3650:x1874579665615232:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:2.0:1787739058.330016:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739058.330027:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ac35c lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x60a2d708 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac355 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.330039:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x60a2d708 (ffffa074039f2c00) 00010000:00010000:2.0:1787739058.330045:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739058.330083:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+7:3650:x1874579665615232:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 76us (314us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.330089:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1258 00000004:00080000:2.0:1787739058.330108:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 683 rpc_version 683 00000020:00100000:2.0:1787739058.330125:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739058.330132:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306007c00 x1874579665615488/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739058.330147:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306007c00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665615488:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739058.330177:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739058.330194:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6ecc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739058.330224:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306005680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665615232:0@lo:103:jbd2/dm-0-8.0 00010000:00010000:2.0:1787739058.330283:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac531 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ac52a expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.330295:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x83604406 (ffffa073068a9900) 00010000:00010000:2.0:1787739058.330300:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac562 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.330855:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac2e5 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1505:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.330865:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac2e5 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1505:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.330872:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac2e5 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1505:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.330879:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739058.330883:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac2e5 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1505:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.330890:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac2e5 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1505:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.330899:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1505:0x0].0x0 (ffffa073068a9300) 00010000:00010000:1.0:1787739058.330903:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739058.330907:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac2e5 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1505:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.330913:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac2e5 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1505:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.330928:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac2ec lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.330934:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac2ec lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.330945:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac2ec lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.330949:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739058.330952:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac2ec lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.330957:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac2ec lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.330964:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xbd:0x0].0x0 (ffffa073068a9500) 00010000:00010000:1.0:1787739058.330966:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739058.330969:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac2ec lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.330974:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac2ec lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.330990:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac3d3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.330996:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac3d3 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331002:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac3d3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331007:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739058.331009:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac3d3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331019:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac3d3 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331027:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xc9300a54 (ffffa074039f2900) 00010000:00010000:1.0:1787739058.331029:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739058.331032:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac3d3 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331037:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac3d3 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331061:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac3da lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1506:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331068:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac3da lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1506:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331073:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac3da lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1506:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331078:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739058.331081:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac3da lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1506:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331087:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac3da lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1506:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331094:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1506:0x0].0x0 (ffffa074039f2e00) 00010000:00010000:1.0:1787739058.331096:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739058.331099:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac3da lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1506:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331105:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac3da lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1506:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331121:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac404 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331127:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac404 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331132:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac404 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331137:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633ce00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739058.331140:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac404 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331146:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac404 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331153:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xc9300a54 (ffffa074039f2900) 00010000:00010000:1.0:1787739058.331155:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739058.331158:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac404 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331164:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac404 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xc9300a54 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331178:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac451 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1507:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331183:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac451 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1507:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331190:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac451 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1507:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331194:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ba00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739058.331197:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac451 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1507:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331202:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac451 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1507:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331209:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1507:0x0].0x0 (ffffa073068a9600) 00010000:00010000:1.0:1787739058.331211:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739058.331214:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac451 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1507:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331220:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ac451 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1507:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331232:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac458 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331238:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac458 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331243:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac458 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331247:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609aa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739058.331250:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac458 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331260:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac458 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331266:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xbe:0x0].0x0 (ffffa073068a9d00) 00010000:00010000:1.0:1787739058.331268:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739058.331271:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac458 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.331276:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac458 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.331610:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac562 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.331617:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac562 lrc: 4/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.331647:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739058.331766:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+6:3652:x1874579665614976:12345-0@lo:103:ldlm_bl.0 Request processed in 7248us (7351us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.331775:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1450 00000100:00100000:2.0:1787739058.331824:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060061c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665614976:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739058.333866:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665615488 00000100:00100000:1.0:1787739058.333881:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665615488 00000100:00100000:1.0:1787739058.334001:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2882 00000100:00100000:1.0:1787739058.334012:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665615488:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739058.334216:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665615488:12345-0@lo:1000:osp_up1-0.0 Request processed in 205us (4036us total) trans 4294968339 rc 0/0 00000100:00100000:1.0:1787739058.334223:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2882 00000100:00100000:1.0:1787739058.336092:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306007c00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665615488:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739058.337213:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074184c5850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+7:3653:x1874579665615104:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 7799us (7958us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.337226:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1257 00010000:00010000:0.0:1787739058.337302:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac562 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.337313:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac562 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739058.337456:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304262580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665615616:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739058.337514:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.337541:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068bac00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.337621:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665615616 00000100:00100000:1.0:1787739058.337628:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665615616 00000100:00100000:1.0:1787739058.337693:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2883 00000100:00100000:1.0:1787739058.337704:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304260f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665615616:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739058.337787:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304260f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665615616:12345-0@lo:1000:mdt00_001.0 Request processed in 86us (272us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.337794:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2883 00000100:00100000:2.0:1787739058.337862:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304262580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665615616:0@lo:1000:mdt00_001.0 00010000:00010000:2.0:1787739058.337909:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac569 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x16b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.337924:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac569 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.337937:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac569 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739058.337958:0:6524:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac569 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.337970:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac55b lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.337976:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac55b lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.337983:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac55b lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.337990:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac55b lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.337997:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609be00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739058.338001:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac55b lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.338009:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac55b lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.338021:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9b00) 00010000:00010000:2.0:1787739058.338024:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739058.338027:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac55b lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.338033:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac55b lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.338044:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac562 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.338060:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac562 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.338065:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac562 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.338070:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739058.338072:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac562 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.338078:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac562 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.338086:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x83604406 (ffffa073068a9900) 00010000:00010000:2.0:1787739058.338088:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739058.338091:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac562 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.338097:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac562 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.338132:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ac554 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d63 expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.338148:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac569 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x16b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d63 expref: 14 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.338160:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16b:0x0].0x0 (ffffa074419d5300) 00010000:00010000:2.0:1787739058.338163:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739058.338165:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c200, rc 0) 00000100:00100000:2.0:1787739058.338218:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f783c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17512:x1874579652448000:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 15109us (16829us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.338226:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23315 00000100:00100000:3.0:1787739058.338709:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665615104:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:1.0:1787739058.342973:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739058.343056:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652448128 00000100:00100000:0.0:1787739058.343066:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652448128 00000100:00100000:0.0:1787739058.343134:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23316 00000100:00100000:0.0:1787739058.343144:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17512:x1874579652448128:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739058.343168:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739058.343187:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac570 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x16b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094d6a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.343253:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac577 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x16b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.343269:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac577 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.343279:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac577 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739058.343366:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x16b:0x0]: returning size 4096 00000004:00200000:0.0:1787739058.343371:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739058.343407:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac570 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x16b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d6a expref: 11 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.343458:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac577 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x16b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d6a expref: 10 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.343467:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16b:0x0].0x0 (ffffa07305d56400) 00010000:00010000:0.0:1787739058.343470:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739058.343474:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a200, rc 0) 00000100:00100000:0.0:1787739058.343534:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17512:x1874579652448128:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 392us (571us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.343542:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23316 00000100:00100000:1.0:1787739058.349721:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739058.349782:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652448256 00000100:00100000:0.0:1787739058.349792:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652448256 00000100:00100000:0.0:1787739058.349854:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23317 00000100:00100000:0.0:1787739058.349864:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17512:x1874579652448256:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739058.349888:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739058.349908:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac57e lrc: 2/0,0 mode: --/CR res: [0x240000400:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094d71 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.349973:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac585 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.350096:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac585 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.350108:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac585 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739058.350156:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xbc:0x0]: returning size 4096 00000004:00200000:0.0:1787739058.350161:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739058.350195:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac57e lrc: 0/0,0 mode: --/CR res: [0x240000400:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d71 expref: 12 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.350214:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac585 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d71 expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.350222:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xbc:0x0].0x0 (ffffa07305d56500) 00010000:00010000:0.0:1787739058.350225:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739058.350228:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609aa00, rc 0) 00000100:00100000:0.0:1787739058.350288:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17512:x1874579652448256:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 426us (575us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.350296:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23317 00000100:00100000:3.0:1787739058.359097:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739058.359255:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652448384 00000100:00100000:2.0:1787739058.359265:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652448384 00000100:00100000:2.0:1787739058.359333:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23318 00000100:00100000:2.0:1787739058.359344:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17512:x1874579652448384:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739058.359362:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739058.359382:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac58c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094d78 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.359478:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac593 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.359492:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ac54d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac546 expref: 5 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.359501:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac593 lrc: 4/1,0 mode: --/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.359534:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ac54d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ac546 expref: 5 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739058.359549:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060061c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665615744:0@lo:104: 00000100:00100000:2.0:1787739058.359596:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739058.359635:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.359833:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665615744 00000100:00100000:1.0:1787739058.359838:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665615744 00000100:00100000:1.0:1787739058.359873:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1110 00000100:00100000:1.0:1787739058.359881:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306005680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665615744:12345-0@lo:104: 00010000:00010000:1.0:1787739058.359893:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac546 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac54d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.359944:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787739058.359949:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306005680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665615744:12345-0@lo:104: Request processed in 70us (354us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.359955:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1110 00000100:00100000:0.0:1787739058.359959:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060061c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665615744:0@lo:104: 00010000:00010000:0.0:1787739058.359977:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac593 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.360013:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739058.360020:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac546 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac54d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.360029:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739058.360033:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac546 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ac54d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739058.360042:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac546 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ac54d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.360082:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac546 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac54d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.360089:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739058.360121:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac546 lrc: 1/0,0 mode: --/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac54d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.360128:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac546 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac54d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.360145:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739058.360147:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739058.360149:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739058.360240:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787739058.360252:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c382940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665615872:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739058.360282:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739058.360550:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665615872 00000100:00100000:3.0:1787739058.360555:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665615872 00010000:00010000:3.0:1787739058.360575:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ac54d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ac546 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.360602:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1451 00000100:00100000:3.0:1787739058.360608:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c12c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+6:3650:x1874579665615872:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739058.360618:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00080000:0.0:1787739058.364586:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 360 batchid updated 00000020:00080000:2.0:1787739058.364644:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 360 00010000:00010000:3.0:1787739058.364809:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ac54d lrc: 0/0,0 mode: --/PW res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ac546 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.364829:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbc:0x0].0x0 (ffffa073068a9100) 00010000:00010000:3.0:1787739058.364835:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac593 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.364845:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac593 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.364851:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac593 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.364882:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739058.364971:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c12c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+5:3650:x1874579665615872:12345-0@lo:103:ldlm_bl.0 Request processed in 4362us (4686us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739058.364980:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1451 00000100:00100000:1.0:1787739058.365088:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c382940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665615872:0@lo:103:ldlm_bl.0 00000004:00080000:2.0:1787739058.365198:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07303c6e180 with version 684 00000020:00080000:2.0:1787739058.365214:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 358 cancel update log [0x240000401:0x3:0x0].356: rc = 0 00010000:00010000:0.0:1787739058.365275:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac593 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.365283:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac593 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739058.365285:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 358 cancel update log [0x200000400:0x3:0x0].356: rc = 0 00000004:00200000:0.0:1787739058.365341:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xbc:0x0]: returning size 4096 00000004:00200000:0.0:1787739058.365346:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739058.365390:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac58c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d78 expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.365407:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac593 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d78 expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.365439:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbc:0x0].0x0 (ffffa073068a9100) 00010000:00010000:0.0:1787739058.365442:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739058.365445:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c000, rc 0) 00000004:00080000:2.0:1787739058.365452:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07303c6e900 with version 685 00000020:00080000:2.0:1787739058.365455:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 359 cancel update log [0x240000401:0x3:0x0].357: rc = 0 00000020:00080000:2.0:1787739058.365477:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 359 cancel update log [0x200000400:0x3:0x0].357: rc = 0 00000100:00100000:0.0:1787739058.365497:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17512:x1874579652448384:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 6155us (6410us total) trans 0 rc 0/0 00000004:00080000:2.0:1787739058.365503:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07303c6e300 with version 686 00000100:00100000:0.0:1787739058.365505:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23318 00000020:00080000:2.0:1787739058.365506:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 360 cancel update log [0x240000401:0x3:0x0].358: rc = 0 00000020:00080000:2.0:1787739058.365523:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 360 cancel update log [0x200000400:0x3:0x0].358: rc = 0 00000020:00080000:2.0:1787739058.365530:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 360 00000004:00080000:1.0:1787739058.365779:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 684 rpc_version 684 00000020:00100000:1.0:1787739058.365806:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739058.365813:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073043c03c0 x1874579665616128/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739058.365833:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c03c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665616128:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739058.365874:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739058.365893:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739058.365945:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665616128 00000100:00100000:2.0:1787739058.365952:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665616128 00000100:00100000:2.0:1787739058.365995:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2884 00000100:00100000:2.0:1787739058.366004:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c30c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665616128:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739058.366218:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c30c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665616128:12345-0@lo:1000:osp_up1-0.0 Request processed in 215us (342us total) trans 4294968340 rc 0/0 00000100:00100000:2.0:1787739058.366225:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2884 00000100:00100000:3.0:1787739058.366514:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c03c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665616128:0@lo:1000:osp_up1-0.0 00000004:00080000:3.0:1787739058.366524:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 685 rpc_version 685 00000020:00100000:3.0:1787739058.366536:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739058.366542:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743c380f00 x1874579665616384/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739058.366555:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c380f00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665616384:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739058.366583:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739058.366608:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074062f3840 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.366675:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665616384 00000100:00100000:1.0:1787739058.366688:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665616384 00000100:00100000:1.0:1787739058.366716:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2885 00000100:00100000:1.0:1787739058.366722:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c381680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665616384:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739058.366804:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c381680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665616384:12345-0@lo:1000:osp_up1-0.0 Request processed in 81us (220us total) trans 4294968341 rc 0/0 00000100:00100000:1.0:1787739058.366810:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2885 00000100:00100000:1.0:1787739058.366839:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c380f00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665616384:0@lo:1000:osp_up1-0.0 00000004:00080000:1.0:1787739058.366846:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 686 rpc_version 686 00000020:00100000:1.0:1787739058.366857:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739058.366861:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073043c1a40 x1874579665616640/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739058.366873:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c1a40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665616640:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739058.366898:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739058.366910:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e180 going to sleep for 11 seconds 00000100:00100000:2.0:1787739058.367291:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665616640 00000100:00100000:2.0:1787739058.367310:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665616640 00000100:00100000:2.0:1787739058.367343:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2886 00000100:00100000:2.0:1787739058.367349:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c2940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665616640:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739058.367458:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c2940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665616640:12345-0@lo:1000:osp_up1-0.0 Request processed in 108us (558us total) trans 4294968342 rc 0/0 00000100:00100000:2.0:1787739058.367468:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2886 00000100:00100000:2.0:1787739058.367501:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c1a40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665616640:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739058.372652:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739058.372793:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652448512 00000100:00100000:2.0:1787739058.372803:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652448512 00000100:00100000:2.0:1787739058.372863:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1259 00000100:00100000:2.0:1787739058.372873:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2409:x1874579652448512:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:2.0:1787739058.372890:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739058.372915:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac569 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x16b:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d63 expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.372935:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16b:0x0].0x0 (ffffa074419d5300) 00010000:00010000:2.0:1787739058.372944:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739058.373001:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652448512:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 130us (359us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.373009:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1259 00000100:00100000:3.0:1787739058.375879:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739058.375946:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652448768 00000100:00100000:0.0:1787739058.375956:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652448768 00000100:00100000:0.0:1787739058.376028:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23319 00000100:00100000:0.0:1787739058.376039:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c380000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17512:x1874579652448768:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:0.0:1787739058.376104:0:7944:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739058.376130:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac577 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x16b:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d6a expref: 11 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.376147:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16b:0x0].0x0 (ffffa07305d56400) 00010000:00010000:0.0:1787739058.376151:0:7944:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739058.376197:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac59a lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.376205:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac59a lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.376215:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac59a lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739058.376237:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304262580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665616768:0@lo:101:mdt00_003.0 00000100:00100000:0.0:1787739058.376283:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.376320:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a5f6e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739058.376382:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665616768 00000100:00100000:2.0:1787739058.376387:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665616768 00000100:00100000:2.0:1787739058.376456:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2887 00000100:00100000:2.0:1787739058.376464:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415701a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+5:7944:x1874579665616768:12345-0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739058.376473:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739058.376487:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac5a1 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac59a expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.376512:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac5a1 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac59a expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.376529:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac5a1 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac59a expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.376536:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5200) 00010000:00010000:2.0:1787739058.376538:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739058.376541:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d600, rc 0) 00000100:00100000:2.0:1787739058.376589:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415701a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:7944:x1874579665616768:12345-0@lo:101:mdt00_003.0 Request processed in 126us (305us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.376594:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2887 00000100:00100000:0.0:1787739058.377074:0:7944:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983352 gen 2 00000004:00080000:0.0:1787739058.377081:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983343 committed transno 4294983352 00000020:00080000:0.0:1787739058.377087:0:7944:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07410b4dcc0 refcount 2 committed 0 result 0 batchid 349 00000020:00080000:0.0:1787739058.377091:0:7944:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88100 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07410b4d900 00000020:00080000:0.0:1787739058.377094:0:7944:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].343 00000020:00080000:0.0:1787739058.377096:0:7944:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88c00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa0730541ec00 00000020:00080000:0.0:1787739058.377099:0:7944:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].342 00000004:00080000:0.0:1787739058.377127:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983345 committed transno 4294983352 00000020:00080000:1.0:1787739058.377140:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 350 committed batchid 348 00000004:00080000:0.0:1787739058.377140:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983346 committed transno 4294983352 00000004:00080000:0.0:1787739058.377148:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983352 committed transno 4294983352 00000100:00100000:0.0:1787739058.377166:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304262580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665616768:0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739058.377182:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac59a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac5a1 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.377192:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac59a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac5a1 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00080000:1.0:1787739058.377199:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 348 00000004:00080000:0.0:1787739058.377207:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739058.377231:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ac5a8 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.377238:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ac5a8 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.377245:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ac5a8 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739058.377258:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665616896:0@lo:101:mdt00_003.0 00000100:00100000:0.0:1787739058.377286:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.377311:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a5f6e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.377944:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665616896 00000100:00100000:1.0:1787739058.377952:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665616896 00000100:00100000:1.0:1787739058.377993:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2888 00000100:00100000:1.0:1787739058.378002:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:7944:x1874579665616896:12345-0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739058.378015:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739058.378030:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac5af lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac5a8 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.378058:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac5af lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac5a8 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.378072:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac5af lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac5a8 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.378079:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x83604406 (ffffa07305eaed00) 00010000:00010000:1.0:1787739058.378082:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739058.378085:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c800, rc 0) 00000100:00100000:1.0:1787739058.378134:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665616896:12345-0@lo:101:mdt00_003.0 Request processed in 134us (848us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.378141:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2888 00000100:00100000:0.0:1787739058.378192:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665616896:0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739058.378205:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ac5a8 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac5af expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.378214:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ac5a8 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac5af expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739058.378228:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:0.0:1787739058.378262:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042630c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665617024:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739058.378291:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.378317:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a5f66c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.378370:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665617024 00000100:00100000:1.0:1787739058.378374:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665617024 00000100:00100000:1.0:1787739058.378406:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2889 00000100:00100000:1.0:1787739058.378441:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665617024:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.378508:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665617024:12345-0@lo:1000:mdt00_003.0 Request processed in 95us (215us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.378515:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2889 00000100:00100000:2.0:1787739058.378653:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042630c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665617024:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739058.378805:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060061c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665617152:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739058.378832:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739058.378858:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6ecc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739058.378905:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665617152 00000100:00100000:0.0:1787739058.378909:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665617152 00000100:00100000:0.0:1787739058.378944:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2890 00000100:00100000:0.0:1787739058.378949:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665617152:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739058.379023:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665617152:12345-0@lo:1000:mdt00_003.0 Request processed in 73us (190us total) trans 0 rc 1/1 00000100:00100000:0.0:1787739058.379029:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2890 00000100:00100000:3.0:1787739058.379258:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060061c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665617152:0@lo:1000:mdt00_003.0 00010000:00010000:3.0:1787739058.379441:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac5b6 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.379449:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac5b6 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.379458:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac5b6 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.379470:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c382940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665617280:0@lo:101:mdt00_003.0 00000100:00100000:3.0:1787739058.379495:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739058.379524:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074062f3840 going to sleep for 11 seconds 00000100:00100000:2.0:1787739058.379580:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665617280 00000100:00100000:2.0:1787739058.379585:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665617280 00000100:00100000:2.0:1787739058.379618:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2891 00000100:00100000:2.0:1787739058.379624:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f7c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665617280:12345-0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739058.379633:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739058.379648:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac5bd lrc: 2/0,0 mode: --/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac5b6 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.379667:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac5bd lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac5b6 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.379796:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac5bd lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac5b6 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.379809:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16b:0x0].0x0 (ffffa074419d5800) 00010000:00010000:2.0:1787739058.379812:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739058.379815:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633cc00, rc 0) 00000100:00100000:2.0:1787739058.379849:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f7c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665617280:12345-0@lo:101:mdt00_003.0 Request processed in 224us (352us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.379855:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2891 00000100:00100000:2.0:1787739058.379884:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c382940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665617280:0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739058.379896:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac5b6 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac5bd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.379904:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac5b6 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac5bd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.379926:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac5c4 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.379935:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac5c4 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.379943:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac5c4 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.379957:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac5cb lrc: 3/0,1 mode: --/EX res: [0x240000400:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.379969:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac585 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d71 expref: 10 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.379982:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac5cb lrc: 4/0,1 mode: --/EX res: [0x240000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.380004:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac585 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d71 expref: 10 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.380019:0:7944:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac585 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d71 expref: 10 pid: 6525 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739058.380030:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306005e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665617408:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739058.380102:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739058.393575:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306005e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665617408:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739058.393604:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac5cb lrc: 3/0,1 mode: --/EX res: [0x240000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.397632:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739058.397851:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652448896 00000100:00100000:0.0:1787739058.397861:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652448896 00010000:00010000:0.0:1787739058.397904:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac585 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d71 expref: 11 pid: 6525 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739058.397947:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1452 00000100:00100000:0.0:1787739058.397958:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2406:x1874579652448896:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739058.397972:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739058.397979:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac585 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d71 expref: 11 pid: 6525 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.397992:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac585 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d71 expref: 11 pid: 6525 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.398014:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac585 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d71 expref: 11 pid: 6525 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.398029:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xbc:0x0].0x0 (ffffa07305d56500) 00010000:00010000:0.0:1787739058.398035:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac5cb lrc: 3/0,1 mode: --/EX res: [0x240000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.398047:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac5cb lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.398064:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac5cb lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.398097:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739058.398144:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac5cb lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739058.398153:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2406:x1874579652448896:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 195us (528us total) trans 0 rc 0/0 00010000:00010000:3.0:1787739058.398158:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac5cb lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739058.398166:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1452 00010000:00010000:3.0:1787739058.398209:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ac5d2 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.398218:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ac5d2 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.398230:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ac5d2 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.398250:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ec3c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665617536:0@lo:101:mdt00_003.0 00000100:00100000:3.0:1787739058.398292:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739058.398332:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074062f3600 going to sleep for 11 seconds 00000100:00100000:0.0:1787739058.398434:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665617536 00000100:00100000:0.0:1787739058.398439:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665617536 00000100:00100000:0.0:1787739058.398473:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2892 00000100:00100000:0.0:1787739058.398481:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156efc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665617536:12345-0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739058.398497:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739058.398511:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac5d9 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac5d2 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.398532:0:18535:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac593 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d78 expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.398541:0:18535:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac5d9 lrc: 4/0,0 mode: --/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac5d2 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.398561:0:18535:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac593 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d78 expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.398574:0:18535:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac593 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d78 expref: 13 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739058.398585:0:18535:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7a940 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:18535:1874579665617664:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739058.398641:0:18535:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a5f6e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739058.414575:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739058.414610:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652449024 00000100:00100000:0.0:1787739058.414618:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652449024 00010000:00010000:0.0:1787739058.414656:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac593 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d78 expref: 14 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739058.414821:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1453 00000100:00100000:0.0:1787739058.414833:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2408:x1874579652449024:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739058.414848:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739058.414856:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac593 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d78 expref: 14 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.414871:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac593 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d78 expref: 14 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.414894:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbc:0x0].0x0 (ffffa073068a9100) 00010000:00010000:0.0:1787739058.414901:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac5d9 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac5d2 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.414913:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac5d9 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac5d2 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.414923:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac5d9 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac5d2 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.414955:0:6509:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac5d9 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac5d2 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739058.414968:0:6509:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a31e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665617792:0@lo:105: 00000100:00100000:0.0:1787739058.415013:0:6509:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.415060:0:6509:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a5f66c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739058.415090:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665617792 00000100:00100000:0.0:1787739058.415094:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665617792 00000100:00100000:0.0:1787739058.415123:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1111 00000100:00100000:0.0:1787739058.415130:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a330c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665617792:12345-0@lo:105: 00010000:00010000:0.0:1787739058.415141:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ac5d2 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac5d9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.415151:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ac5d2 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac5d9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.415159:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ac5d2 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac5d9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.415166:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ac5d2 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac5d9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.415172:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ac5d2 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac5d9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.415179:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ac5d2 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ac5d9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.415197:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069aa800) 00000100:00100000:0.0:1787739058.415233:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a330c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665617792:12345-0@lo:105: Request processed in 102us (217us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.415240:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1111 00000100:00100000:0.0:1787739058.415277:0:6509:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a31e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665617792:0@lo:105: 00010000:00010000:0.0:1787739058.415857:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739058.415906:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2408:x1874579652449024:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 1074us (1339us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.415915:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1453 00010000:00010000:0.0:1787739058.416060:0:18535:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac593 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xbc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d78 expref: 13 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739058.416078:0:18535:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7a940 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:18535:1874579665617664:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739058.416096:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac5d9 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac5d2 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.416104:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbc:0x0].0x0 (ffffa073068a9100) 00010000:00010000:0.0:1787739058.416107:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739058.416110:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a400, rc 0) 00000100:00100000:0.0:1787739058.416151:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156efc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665617536:12345-0@lo:101:mdt00_003.0 Request processed in 17671us (17859us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.416158:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2892 00000100:00100000:0.0:1787739058.416200:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ec3c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665617536:0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739058.416215:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ac5d2 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac5d9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739058.421440:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a31e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665617920:0@lo:403:mdt00_003.0 00000100:00100000:0.0:1787739058.421510:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.421542:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a5f6e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739058.421566:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665617920 00000100:00100000:0.0:1787739058.421572:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665617920 00000100:00100000:0.0:1787739058.421615:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2893 00000100:00100000:0.0:1787739058.421622:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665617920:12345-0@lo:403:mdt00_003.0 00000100:00100000:0.0:1787739058.432638:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665617920:12345-0@lo:403:mdt00_003.0 Request processed in 11006us (11117us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.432652:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2893 00000100:00100000:0.0:1787739058.432732:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a31e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665617920:0@lo:403:mdt00_003.0 00000040:00080000:0.0:1787739058.433685:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].344, 664 off523536 00000040:00080000:0.0:1787739058.433856:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].345, 744 off572904 00000004:00080000:0.0:1787739058.433862:0:7944:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0740a5f6e40 with version 657 00000004:00080000:0.0:1787739058.444466:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 657 rpc_version 657 00000020:00100000:0.0:1787739058.444510:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739058.444523:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303a31e00 x1874579665618176/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1808/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739058.444548:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a31e00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665618176:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739058.444616:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.444649:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355edf00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739058.446438:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665618176 00000100:00100000:3.0:1787739058.446452:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665618176 00000100:00100000:3.0:1787739058.446508:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2894 00000100:00100000:3.0:1787739058.446519:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a321c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665618176:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739058.447845:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a321c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665618176:12345-0@lo:1000:osp_up0-1.0 Request processed in 1323us (3223us total) trans 4294983353 rc 0/0 00000100:00100000:3.0:1787739058.447857:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2894 00000100:00100000:0.0:1787739058.452504:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a31e00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665618176:0@lo:1000:osp_up0-1.0 00000004:00200000:1.0:1787739058.458540:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x16b:0x0]: returning size 4096 00010000:00010000:1.0:1787739058.458572:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac5cb lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.458583:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac5cb lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.458590:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac5cb lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.458597:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739058.458602:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac5cb lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.458610:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac5cb lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.458626:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xbc:0x0].0x0 (ffffa07305d56500) 00010000:00010000:1.0:1787739058.458629:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739058.458633:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac5cb lrc: 2/0,0 mode: --/EX res: [0x240000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.458639:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac5cb lrc: 0/0,0 mode: --/EX res: [0x240000400:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.458653:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ac5d2 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac5d9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.458661:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ac5d2 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac5d9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.458668:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ac5d2 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac5d9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.458694:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739058.458713:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xbc:0x0] 00010000:00010000:1.0:1787739058.458720:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac5c4 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.458726:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac5c4 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.458732:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac5c4 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.458736:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739058.458739:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac5c4 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.458745:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac5c4 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.458753:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16b:0x0].0x0 (ffffa07305d56400) 00010000:00010000:1.0:1787739058.458755:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739058.458758:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac5c4 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.458764:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac5c4 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.458773:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac5b6 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac5bd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.458780:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac5b6 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac5bd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.458787:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac5b6 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac5bd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.458810:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:1.0:1787739058.458821:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739058.458826:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac59a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac5a1 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.458833:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac59a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac5a1 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.458840:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac59a lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac5a1 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.458858:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:1.0:1787739058.458861:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739058.458866:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ac5a8 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac5af expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.458874:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ac5a8 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac5af expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739058.458961:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c380000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17512:x1874579652448768:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 82920us (83087us total) trans 4294968343 rc 0/0 00000100:00100000:1.0:1787739058.458970:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23319 00010000:00010000:0.0:1787739058.459094:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1787739058.459102:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ac5d2 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac5d9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.459110:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739058.459115:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ac5d2 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac5d9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739058.459124:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ac5d2 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac5d9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.459154:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ac5d2 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac5d9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.459159:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739058.459191:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ac5d2 lrc: 1/0,0 mode: --/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac5d9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.459199:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1ac5d2 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac5d9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.459216:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787739058.459219:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac5b6 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac5bd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.459225:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739058.459229:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac5b6 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac5bd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739058.459235:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac5b6 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac5bd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.459254:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac5b6 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac5bd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.459260:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739058.459266:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac5b6 lrc: 1/0,0 mode: --/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac5bd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.459272:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac5b6 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac5bd expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.459282:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739058.459285:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac59a lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac5a1 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.459291:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739058.459294:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac59a lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac5a1 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739058.459300:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac59a lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac5a1 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.459315:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac59a lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac5a1 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.459321:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739058.459334:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac59a lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac5a1 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.459345:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac59a lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac5a1 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.459357:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739058.459359:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739058.459360:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739058.459375:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a31a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665618432:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739058.459459:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.459502:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787739058.459508:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665618304:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739058.459531:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.459553:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787739058.459561:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a30780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665618560:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739058.459583:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.459610:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665618432 00000100:00100000:0.0:1787739058.459616:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665618432 00000100:00100000:0.0:1787739058.459655:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665618304 00000100:00100000:0.0:1787739058.459658:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665618304 00000100:00100000:0.0:1787739058.459677:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665618560 00000100:00100000:0.0:1787739058.459679:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665618560 00000100:00100000:0.0:1787739058.459694:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1260 00000100:00100000:0.0:1787739058.459701:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+12:3653:x1874579665618432:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739058.459713:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739058.459730:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac5bd lrc: 0/0,0 mode: --/EX res: [0x240000402:0x16b:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac5b6 expref: 12 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.459743:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16b:0x0].0x0 (ffffa074419d5800) 00010000:00010000:0.0:1787739058.459753:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739058.459782:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+11:3653:x1874579665618432:12345-0@lo:103:ldlm_bl.0 Request processed in 82us (324us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.459788:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1260 00000100:00100000:0.0:1787739058.459805:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1261 00000100:00100000:0.0:1787739058.459810:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074184c6050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+10:3653:x1874579665618304:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739058.459817:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739058.459827:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac5d9 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xbc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac5d2 expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.459838:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbc:0x0].0x0 (ffffa073068a9100) 00010000:00010000:0.0:1787739058.459844:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739058.459865:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074184c6050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+9:3653:x1874579665618304:12345-0@lo:103:ldlm_bl.0 Request processed in 55us (333us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.459871:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1261 00000100:00100000:0.0:1787739058.459910:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1262 00000100:00100000:0.0:1787739058.459917:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+8:3651:x1874579665618560:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739058.459924:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739058.459934:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac5a1 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac59a expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.459944:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5200) 00010000:00010000:0.0:1787739058.459950:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739058.459976:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+7:3651:x1874579665618560:12345-0@lo:103:ldlm_bl.0 Request processed in 60us (391us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.459995:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1262 00000100:00100000:0.0:1787739058.460029:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a30780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665618560:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739058.460570:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665618304:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739058.460582:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a31a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665618432:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739058.461099:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739058.461485:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739058.493819:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739058.493886:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652449152 00000100:00100000:2.0:1787739058.493896:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652449152 00000100:00100000:2.0:1787739058.493954:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23320 00000100:00100000:2.0:1787739058.493965:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a31a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17513:x1874579652449152:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739058.493988:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739058.494009:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac5e0 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094d7f expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.494085:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac5e7 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.494102:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac5e7 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.494113:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac5e7 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.494128:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac5ee lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.494135:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac5ee lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.494141:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac5ee lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.494181:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac5e7 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.494188:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac5e7 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.494195:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac5e7 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.494201:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac5e7 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.494207:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739058.494211:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac5e7 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.494220:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac5e7 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.494232:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9e00) 00010000:00010000:2.0:1787739058.494235:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739058.494239:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac5e7 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.494246:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac5e7 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.494258:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac5ee lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.494264:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac5ee lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.494269:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac5ee lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.494274:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739058.494277:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac5ee lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.494283:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac5ee lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.494290:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x3873ad72 (ffffa074419d5300) 00010000:00010000:2.0:1787739058.494292:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739058.494295:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac5ee lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.494301:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac5ee lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.494337:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac5e0 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d7f expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.494347:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9e00) 00010000:00010000:2.0:1787739058.494350:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739058.494354:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac5e0 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d7f expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.494363:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d400, rc 0) 00000100:00100000:2.0:1787739058.494442:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a31a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17513:x1874579652449152:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 479us (630us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739058.494450:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23320 00010000:00010000:1.0:1787739058.499824:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739058.511876:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739058.511953:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652449280 00000100:00100000:2.0:1787739058.511962:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652449280 00000100:00100000:2.0:1787739058.512021:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23321 00000100:00100000:2.0:1787739058.512031:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17513:x1874579652449280:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739058.512065:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739058.512088:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac5f5 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094d86 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.512150:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac5fc lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.512163:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac5fc lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.512175:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac5fc lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739058.512223:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:2.0:1787739058.512228:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739058.512263:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac5f5 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d86 expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.512281:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac5fc lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d86 expref: 14 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.512289:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5300) 00010000:00010000:2.0:1787739058.512292:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739058.512295:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d200, rc 0) 00000100:00100000:2.0:1787739058.512352:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17513:x1874579652449280:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 322us (485us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.512359:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23321 00000100:00100000:0.0:1787739058.542355:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739058.542494:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652449536 00000100:00100000:1.0:1787739058.542504:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652449536 00000100:00100000:1.0:1787739058.542551:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23322 00000100:00100000:1.0:1787739058.542561:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17513:x1874579652449536:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:1.0:1787739058.542662:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665618688:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.542712:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739058.542755:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ea80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.542774:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665618688 00000100:00100000:1.0:1787739058.542777:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665618688 00000100:00100000:1.0:1787739058.542804:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2895 00000100:00100000:1.0:1787739058.542810:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665618688:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.542876:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665618688:12345-0@lo:1000:mdt00_003.0 Request processed in 65us (163us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.542887:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2895 00000100:00100000:1.0:1787739058.542926:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665618688:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.542967:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665618816:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.542991:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739058.543005:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ea80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739058.543244:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665618816 00000100:00100000:3.0:1787739058.543249:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665618816 00000100:00100000:3.0:1787739058.543285:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2896 00000100:00100000:3.0:1787739058.543293:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665618816:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739058.543370:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665618816:12345-0@lo:1000:mdt00_003.0 Request processed in 78us (377us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739058.543377:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2896 00000100:00100000:3.0:1787739058.543629:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665618816:0@lo:1000:mdt00_003.0 00010000:00010000:3.0:1787739058.543680:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ac603 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.543690:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ac603 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.543700:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ac603 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.543712:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665618944:0@lo:101:mdt00_003.0 00000100:00100000:3.0:1787739058.543737:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739058.543764:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404c00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739058.543777:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665618944 00000100:00100000:3.0:1787739058.543781:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665618944 00000100:00100000:3.0:1787739058.543802:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2897 00000100:00100000:3.0:1787739058.543808:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665618944:12345-0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739058.543817:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739058.543832:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ac60a lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac603 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.543855:0:17834:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac515 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d5c expref: 9 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.543864:0:17834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ac60a lrc: 4/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac603 expref: 11 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.543885:0:17834:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac515 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d5c expref: 9 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.543898:0:17834:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac515 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d5c expref: 9 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.543908:0:17834:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ec000 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:17834:1874579665619072:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739058.543961:0:17834:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739058.561456:0:17834:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ec000 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:17834:1874579665619072:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739058.561491:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ac60a lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac603 expref: 11 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.561505:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9e00) 00010000:00010000:3.0:1787739058.561508:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739058.561513:0:17834:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ac60a lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac603 expref: 11 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.561524:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa200, rc 0) 00000100:00100000:3.0:1787739058.561582:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:7944:x1874579665618944:12345-0@lo:101:mdt00_003.0 Request processed in 17773us (17843us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739058.561590:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2897 00000100:00100000:1.0:1787739058.561826:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665618944:0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739058.561842:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ac603 lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac60a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.561852:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ac603 lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac60a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739058.569245:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739058.569283:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652449664 00000100:00100000:2.0:1787739058.569292:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652449664 00010000:00010000:2.0:1787739058.569328:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac515 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d5c expref: 10 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739058.569363:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1454 00000100:00100000:2.0:1787739058.569372:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306005e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2409:x1874579652449664:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739058.569388:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739058.569394:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac515 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d5c expref: 10 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.569540:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac515 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d5c expref: 10 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.569560:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac515 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d5c expref: 10 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.569572:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9e00) 00010000:00010000:2.0:1787739058.569578:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ac60a lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac603 expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.569588:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ac60a lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac603 expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.569597:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ac60a lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac603 expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.569628:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ac60a lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac603 expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739058.569641:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060070c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665619200:0@lo:105: 00000100:00100000:2.0:1787739058.569688:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739058.569719:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068bae40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.569782:0:6506:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787739058.569795:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665619200 00000100:00100000:1.0:1787739058.569798:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665619200 00000100:00100000:1.0:1787739058.569836:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1112 00000100:00100000:1.0:1787739058.569843:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f4d050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665619200:12345-0@lo:105: 00010000:00010000:1.0:1787739058.569853:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ac603 lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac60a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.569862:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ac603 lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac60a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.569869:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ac603 lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac60a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.569880:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ac603 lrc: 5/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac60a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.569887:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ac603 lrc: 6/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac60a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.569893:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ac603 lrc: 6/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ac60a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.569909:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069ab800) 00000100:00100000:1.0:1787739058.569943:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f4d050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665619200:12345-0@lo:105: Request processed in 102us (255us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.569949:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1112 00010000:00010000:1.0:1787739058.569995:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ac603 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac60a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.570003:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ac603 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac60a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739058.570024:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739058.570066:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1ac611 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.570073:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1ac611 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.570081:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1ac611 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739058.570098:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665619328:0@lo:101:mdt00_003.0 00000100:00100000:1.0:1787739058.570122:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739058.570140:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.570152:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665619328 00000100:00100000:1.0:1787739058.570155:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665619328 00000100:00100000:1.0:1787739058.570179:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2898 00000100:00100000:1.0:1787739058.570184:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:7944:x1874579665619328:12345-0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739058.570199:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739058.570211:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac618 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac611 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.570228:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac618 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac611 expref: 12 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.570238:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac618 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac611 expref: 12 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.570246:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x3873ad72 (ffffa07305eae400) 00010000:00010000:1.0:1787739058.570249:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739058.570251:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c000, rc 0) 00000100:00100000:1.0:1787739058.570279:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:7944:x1874579665619328:12345-0@lo:101:mdt00_003.0 Request processed in 94us (155us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.570285:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2898 00000100:00100000:1.0:1787739058.570319:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665619328:0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739058.570331:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1ac611 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac618 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.570344:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1ac611 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac618 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739058.570354:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:1.0:1787739058.570381:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665619456:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.570444:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739058.570459:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.570471:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665619456 00000100:00100000:1.0:1787739058.570475:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665619456 00000100:00100000:1.0:1787739058.570497:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2899 00000100:00100000:1.0:1787739058.570503:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:7944:x1874579665619456:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.570559:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:7944:x1874579665619456:12345-0@lo:1000:mdt00_003.0 Request processed in 56us (155us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.570565:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2899 00000100:00100000:1.0:1787739058.570598:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665619456:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.570630:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665619584:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.570652:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739058.570666:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.570676:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665619584 00000100:00100000:1.0:1787739058.570680:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665619584 00000100:00100000:1.0:1787739058.570698:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2900 00000100:00100000:1.0:1787739058.570702:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:7944:x1874579665619584:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.570766:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:7944:x1874579665619584:12345-0@lo:1000:mdt00_003.0 Request processed in 62us (112us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739058.570772:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2900 00000100:00100000:1.0:1787739058.570795:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665619584:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.570855:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665619712:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.570875:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739058.570886:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.570896:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665619712 00000100:00100000:1.0:1787739058.570899:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665619712 00000100:00100000:1.0:1787739058.570915:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2901 00000100:00100000:1.0:1787739058.570919:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:7944:x1874579665619712:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.570963:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:7944:x1874579665619712:12345-0@lo:1000:mdt00_003.0 Request processed in 43us (86us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739058.570968:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2901 00000100:00100000:1.0:1787739058.570990:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665619712:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.571026:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665619840:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.571045:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739058.571068:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.571079:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665619840 00000100:00100000:1.0:1787739058.571082:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665619840 00000100:00100000:1.0:1787739058.571100:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2902 00000100:00100000:1.0:1787739058.571106:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157ead00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:7944:x1874579665619840:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.571148:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157ead00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:7944:x1874579665619840:12345-0@lo:1000:mdt00_003.0 Request processed in 41us (100us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.571153:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2902 00000100:00100000:1.0:1787739058.571175:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665619840:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.571208:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157eb0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665619968:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.571228:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739058.571241:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.571251:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665619968 00000100:00100000:1.0:1787739058.571254:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665619968 00000100:00100000:1.0:1787739058.571274:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2903 00000100:00100000:1.0:1787739058.571279:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157ea940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:7944:x1874579665619968:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.571332:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157ea940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:7944:x1874579665619968:12345-0@lo:1000:mdt00_003.0 Request processed in 52us (102us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.571338:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2903 00000100:00100000:1.0:1787739058.571360:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074157eb0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665619968:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.571390:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157eb0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665620096:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.571432:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739058.571445:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.571454:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665620096 00000100:00100000:1.0:1787739058.571457:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665620096 00000100:00100000:1.0:1787739058.571478:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2904 00000100:00100000:1.0:1787739058.571484:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157e9a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:7944:x1874579665620096:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.571528:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157e9a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:7944:x1874579665620096:12345-0@lo:1000:mdt00_003.0 Request processed in 43us (115us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.571534:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2904 00000100:00100000:1.0:1787739058.571556:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074157eb0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665620096:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.571582:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157eb0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665620224:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.571601:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739058.571613:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.571622:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665620224 00000100:00100000:1.0:1787739058.571625:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665620224 00000100:00100000:1.0:1787739058.571641:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2905 00000100:00100000:1.0:1787739058.571645:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157e92c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:7944:x1874579665620224:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739058.571682:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157e92c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:7944:x1874579665620224:12345-0@lo:1000:mdt00_003.0 Request processed in 35us (79us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.571688:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2905 00000100:00100000:1.0:1787739058.571711:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074157eb0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665620224:0@lo:1000:mdt00_003.0 00080000:02000000:1.0:1787739058.571856:0:7944:0:(osd_handler.c:2218:osd_trans_start()) *** cfs_fail_loc=19a, val=0*** 00000004:00080000:1.0:1787739058.571870:0:7944:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730610e9c0 with version 0 00000020:00080000:1.0:1787739058.571876:0:7944:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa0730610e300 refcount 3 committed 0 result -5 batchid 362 00000020:00080000:1.0:1787739058.571879:0:7944:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2900 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result -5 sub_th ffffa0730610e9c0 00000020:00080000:1.0:1787739058.571882:0:7944:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2300 obd lustre-MDT0000-osd committed 1 started 0 stopped 1 result 0 sub_th ffffa073040e9800 00000004:00080000:1.0:1787739058.571898:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739058.571906:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ac603 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac60a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.571916:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ac603 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac60a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.571923:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ac603 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac60a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.571942:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739058.571947:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739058.571950:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1ac611 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac618 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.571958:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1ac611 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac618 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.571970:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1ac611 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac618 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.571988:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1787739058.572042:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17513:x1874579652449536:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 29481us (29695us total) trans 0 rc -5/-5 00000100:00100000:1.0:1787739058.572061:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23322 00010000:00010000:3.0:1787739058.572253:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787739058.572261:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ac603 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac60a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.572270:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739058.572275:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ac603 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac60a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739058.572284:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ac603 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac60a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.572310:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ac603 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac60a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.572321:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739058.572344:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ac603 lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac60a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.572352:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ac603 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac60a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.572370:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739058.572373:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1ac611 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac618 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.572378:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ca00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739058.572381:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1ac611 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac618 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739058.572388:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1ac611 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac618 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.572480:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1ac611 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac618 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.572487:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739058.572501:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1ac611 lrc: 1/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac618 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.572508:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ca00/0x7ee4453e4a1ac611 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac618 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.572525:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739058.572527:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739058.572529:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739058.572644:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787739058.572655:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665620480:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739058.572686:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000020:00080000:2.0:1787739058.572921:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 360 00000100:00100000:2.0:1787739058.572947:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060070c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665619200:0@lo:105: 00010000:00010000:2.0:1787739058.572960:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739058.572999:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306005e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:2409:x1874579652449664:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 3627us (3759us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.573008:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1454 00000100:00100000:2.0:1787739058.573026:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665620480 00000100:00100000:2.0:1787739058.573030:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665620480 00000100:00100000:2.0:1787739058.573072:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1263 00000100:00100000:2.0:1787739058.573078:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+12:3653:x1874579665620480:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739058.573088:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739058.573104:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac618 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac611 expref: 12 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.573116:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x3873ad72 (ffffa07305eae400) 00010000:00010000:2.0:1787739058.573123:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739058.573152:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+11:3653:x1874579665620480:12345-0@lo:103:ldlm_bl.0 Request processed in 75us (466us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.573158:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1263 00000100:00100000:2.0:1787739058.573185:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665620480:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739058.573394:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739058.574617:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ec000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665620352:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739058.574665:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739058.574827:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665620352 00000100:00100000:2.0:1787739058.574834:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665620352 00000100:00100000:2.0:1787739058.574879:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1264 00000100:00100000:2.0:1787739058.574887:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157eb0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+11:3651:x1874579665620352:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739058.574898:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739058.574917:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ac60a lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac603 expref: 11 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.574932:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9e00) 00010000:00010000:2.0:1787739058.574940:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739058.574985:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157eb0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+10:3651:x1874579665620352:12345-0@lo:103:ldlm_bl.0 Request processed in 99us (318us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.574992:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1264 00000100:00100000:1.0:1787739058.579845:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ec000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665620352:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739058.620540:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739058.620735:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652449920 00000100:00100000:1.0:1787739058.620745:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652449920 00000100:00100000:1.0:1787739058.620798:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23323 00000100:00100000:1.0:1787739058.620809:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17515:x1874579652449920:12345-192.168.203.15@tcp:36:mkdir.0 00010000:00010000:1.0:1787739058.620923:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac61f lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.620940:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac61f lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.620952:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac61f lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.620965:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac626 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.620973:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac626 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.620981:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac626 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.621463:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac62d lrc: 3/0,1 mode: --/PW res: [0x240000402:0x16c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.621473:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac62d lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x16c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.621481:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac62d lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x16c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739058.621493:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a493f97a reply state 00000000d148d2a3 transno 4294968344 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac62d lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x16c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.621503:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac62d lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x16c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.621510:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16c:0x0].0x0 (ffffa07305eae200) 00010000:00010000:1.0:1787739058.621512:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739058.621515:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:1.0:1787739058.621519:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x16c:0x0]: returning size 4096 00010000:00010000:1.0:1787739058.621526:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac61f lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.621533:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac61f lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.621539:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac61f lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.621546:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac61f lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.621553:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739058.621557:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac61f lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.621564:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac61f lrc: 5/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.621573:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305eae900) 00010000:00010000:1.0:1787739058.621576:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739058.621579:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac61f lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.621585:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac61f lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739058.621600:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a493f97a reply state 00000000d148d2a3 transno 4294968344 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac626 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.621607:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac626 lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.621612:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x3873ad72 (ffffa07305eaef00) 00010000:00010000:1.0:1787739058.621614:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739058.621616:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000100:00100000:1.0:1787739058.621679:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17515:x1874579652449920:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 871us (1147us total) trans 4294968344 rc 0/0 00000100:00100000:1.0:1787739058.621687:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23323 00000100:00100000:3.0:1787739058.628209:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ec000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665620608:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1787739058.628277:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739058.628346:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665620608 00000100:00100000:3.0:1787739058.628352:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665620608 00000100:00100000:3.0:1787739058.628402:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 354 00000100:00100000:3.0:1787739058.628426:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ecf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874579665620608:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1787739058.628456:0:8441:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8646080 pending 0 free 3923529728 avail 3692007424 00000100:00100000:3.0:1787739058.628501:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ecf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874579665620608:12345-0@lo:13:osp-pre-0-1.0 Request processed in 91us (223us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739058.628508:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 354 00000100:00080000:3.0:1787739058.628528:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787739058.628563:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ec000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665620608:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1787739058.629095:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787739058.629106:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665620736:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787739058.629136:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.629208:0:10361:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665620736 00000100:00100000:0.0:1787739058.629212:0:10361:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665620736 00000100:00100000:0.0:1787739058.629254:0:10361:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 355 00000100:00100000:0.0:1787739058.629261:0:10361:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665620736:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787739058.629273:0:10361:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8646080 pending 0 free 3923529728 avail 3692007424 00000100:00100000:0.0:1787739058.629312:0:10361:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665620736:12345-0@lo:13:osp-pre-0-0.0 Request processed in 51us (174us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.629318:0:10361:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 355 00000100:00100000:0.0:1787739058.629356:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665620736:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1787739058.629405:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787739058.660252:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739058.660319:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652450048 00000100:00100000:3.0:1787739058.660328:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652450048 00000100:00100000:3.0:1787739058.660387:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23324 00000100:00100000:3.0:1787739058.660398:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17520:x1874579652450048:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739058.660443:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739058.660463:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ac634 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094d8d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660525:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac63b lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660538:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac63b lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660547:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac63b lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660556:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac642 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660562:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac642 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660567:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac642 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660630:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ac649 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x16c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660638:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ac649 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660645:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ac649 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739058.660683:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x16c:0x0]: returning size 4096 00000004:00200000:3.0:1787739058.660687:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787739058.660704:0:13793:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ac649 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660717:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac63b lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660723:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac63b lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660728:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac63b lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660734:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac63b lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660740:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739058.660744:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac63b lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660752:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac63b lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660763:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305d56e00) 00010000:00010000:3.0:1787739058.660766:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739058.660770:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac63b lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660776:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac63b lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660787:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac642 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660797:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac642 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660803:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac642 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660807:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739058.660810:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac642 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660816:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac642 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660822:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x3873ad72 (ffffa07305eaef00) 00010000:00010000:3.0:1787739058.660825:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739058.660827:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac642 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660832:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac642 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660856:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ac634 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d8d expref: 13 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660872:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ac649 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x16c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d8d expref: 12 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.660879:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16c:0x0].0x0 (ffffa07305eae200) 00010000:00010000:3.0:1787739058.660881:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787739058.660883:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa800, rc 0) 00000100:00100000:3.0:1787739058.660936:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:17520:x1874579652450048:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 541us (695us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739058.660943:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23324 00000100:00100000:3.0:1787739058.666517:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739058.671915:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652450304 00000100:00100000:1.0:1787739058.671931:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652450304 00000100:00100000:1.0:1787739058.672003:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23325 00000100:00100000:1.0:1787739058.672014:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:17520:x1874579652450304:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:1.0:1787739058.672081:0:7944:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739058.672110:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ac649 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x16c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d8d expref: 12 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672126:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16c:0x0].0x0 (ffffa07305eae200) 00010000:00010000:1.0:1787739058.672131:0:7944:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739058.672157:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac650 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672170:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac650 lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672181:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac650 lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672193:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac657 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672201:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac657 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672208:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac657 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672281:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac65e lrc: 3/0,1 mode: --/EX res: [0x240000402:0x16c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672290:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac65e lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x16c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672298:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac65e lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x16c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739058.672535:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x16c:0x0]: returning size 4096 00010000:00010000:1.0:1787739058.672554:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac65e lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x16c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672560:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac65e lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x16c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672566:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac65e lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672572:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739058.672575:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac65e lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672582:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac65e lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x16c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672590:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16c:0x0].0x0 (ffffa07305eae200) 00010000:00010000:1.0:1787739058.672593:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739058.672596:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac65e lrc: 2/0,0 mode: --/EX res: [0x240000402:0x16c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672601:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac65e lrc: 0/0,0 mode: --/EX res: [0x240000402:0x16c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672616:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac650 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672621:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac650 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672630:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac650 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672635:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac650 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672640:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739058.672642:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac650 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672648:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac650 lrc: 5/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672656:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305eae900) 00010000:00010000:1.0:1787739058.672658:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739058.672661:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac650 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672666:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac650 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739058.672693:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000057c8865d reply state 000000008bb724bc transno 4294968345 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac657 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672964:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac657 lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.672972:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x3873ad72 (ffffa07305eaef00) 00010000:00010000:1.0:1787739058.672974:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739058.672977:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787739058.673261:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17520:x1874579652450304:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 1247us (6750us total) trans 4294968345 rc 0/0 00000100:00100000:1.0:1787739058.673270:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23325 00000100:00100000:3.0:1787739058.700389:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739058.700749:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652450432 00000100:00100000:0.0:1787739058.700759:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652450432 00000100:00100000:0.0:1787739058.700813:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23326 00000100:00100000:0.0:1787739058.700822:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17525:x1874579652450432:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739058.700841:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739058.700859:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac665 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094d94 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.700917:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac66c lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.700931:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac66c lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.700941:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac66c lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.700954:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ac673 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.700961:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ac673 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.700967:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ac673 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.701007:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac66c lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.701014:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac66c lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.701020:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac66c lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.701026:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac66c lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.701032:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609aa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739058.701036:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac66c lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.701044:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac66c lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.701240:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5300) 00010000:00010000:0.0:1787739058.701243:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739058.701248:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac66c lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.701253:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac66c lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.701266:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ac673 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.701271:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ac673 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.701276:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ac673 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.701281:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739058.701283:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ac673 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.701288:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ac673 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.701294:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x8102366f (ffffa073068a9100) 00010000:00010000:0.0:1787739058.701296:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739058.701298:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ac673 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.701304:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ac673 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.701336:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac665 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d94 expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.701346:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5300) 00010000:00010000:0.0:1787739058.701348:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739058.701351:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac665 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d94 expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.701359:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b600, rc 0) 00000100:00100000:0.0:1787739058.701564:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17525:x1874579652450432:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 741us (1183us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739058.701571:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23326 00000100:00100000:2.0:1787739058.708975:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739058.709115:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652450560 00000100:00100000:0.0:1787739058.709124:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652450560 00000100:00100000:0.0:1787739058.709169:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23327 00000100:00100000:0.0:1787739058.709177:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060070c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17525:x1874579652450560:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739058.709198:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739058.709219:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac67a lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094d9b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.709273:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac681 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.709286:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac681 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.709296:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac681 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739058.709343:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x5:0x0]: returning size 4096 00000004:00200000:0.0:1787739058.709347:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739058.709382:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac67a lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d9b expref: 15 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.709400:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac681 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d9b expref: 14 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.709439:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9800) 00010000:00010000:0.0:1787739058.709443:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739058.709447:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609bc00, rc 0) 00000100:00100000:0.0:1787739058.709498:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060070c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17525:x1874579652450560:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 320us (530us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.709505:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23327 00000100:00100000:3.0:1787739058.723499:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739058.725847:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652450816 00000100:00100000:0.0:1787739058.725862:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652450816 00000100:00100000:0.0:1787739058.725974:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23328 00000100:00100000:0.0:1787739058.725985:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073040003c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17525:x1874579652450816:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:0.0:1787739058.726106:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665620864:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739058.726171:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.726225:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739058.726282:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665620864 00000100:00100000:2.0:1787739058.726286:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665620864 00000100:00100000:2.0:1787739058.726322:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2906 00000100:00100000:2.0:1787739058.726330:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665620864:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739058.726402:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665620864:12345-0@lo:1000:mdt00_000.0 Request processed in 72us (230us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.726409:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2906 00000100:00100000:0.0:1787739058.727580:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665620864:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739058.727639:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665620992:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739058.727689:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.727725:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739058.727826:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665620992 00000100:00100000:2.0:1787739058.727833:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665620992 00000100:00100000:2.0:1787739058.727874:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2907 00000100:00100000:2.0:1787739058.727882:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a312c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665620992:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739058.727966:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a312c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665620992:12345-0@lo:1000:mdt00_000.0 Request processed in 85us (276us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739058.727974:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2907 00000100:00100000:0.0:1787739058.728807:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665620992:0@lo:1000:mdt00_000.0 00010000:00010000:0.0:1787739058.728865:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1ac688 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.728875:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1ac688 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.728884:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1ac688 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739058.728898:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665621120:0@lo:101:mdt00_000.0 00000100:00100000:0.0:1787739058.728931:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.728964:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739058.729019:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665621120 00000100:00100000:2.0:1787739058.729024:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665621120 00000100:00100000:2.0:1787739058.729067:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2908 00000100:00100000:2.0:1787739058.729073:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665621120:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739058.729084:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739058.729099:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac68f lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac688 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.729121:0:8448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac5fc lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d86 expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.729134:0:8448:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac68f lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac688 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.729157:0:8448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac5fc lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d86 expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.729170:0:8448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac5fc lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d86 expref: 13 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739058.729181:0:8448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306007480 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8448:1874579665621248:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739058.729237:0:8448:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068bae40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739058.733359:0:8448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306007480 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8448:1874579665621248:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739058.733396:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac68f lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac688 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.733445:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5300) 00010000:00010000:3.0:1787739058.733449:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739058.733453:0:8448:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac68f lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac688 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.733465:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633de00, rc 0) 00000100:00100000:3.0:1787739058.733534:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665621120:12345-0@lo:101:mdt00_000.0 Request processed in 4461us (4601us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739058.733542:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2908 00000100:00100000:0.0:1787739058.733644:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665621120:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739058.733663:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1ac688 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac68f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.733684:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1ac688 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac68f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.737746:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739058.737998:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652450944 00000100:00100000:0.0:1787739058.738007:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652450944 00010000:00010000:0.0:1787739058.738058:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac5fc lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d86 expref: 14 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739058.738097:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1455 00000100:00100000:0.0:1787739058.738107:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304003c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2408:x1874579652450944:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739058.738123:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739058.738131:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac5fc lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d86 expref: 14 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.738143:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac5fc lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d86 expref: 14 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.738165:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ac5fc lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d86 expref: 14 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.738183:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5300) 00010000:00010000:0.0:1787739058.738189:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac68f lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac688 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.738201:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac68f lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac688 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.738211:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac68f lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac688 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.738244:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac68f lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac688 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739058.738258:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a330c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665621376:0@lo:105: 00000100:00100000:0.0:1787739058.738305:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.738348:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739058.738365:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665621376 00000100:00100000:0.0:1787739058.738369:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665621376 00000100:00100000:0.0:1787739058.738397:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1113 00000100:00100000:0.0:1787739058.738404:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665621376:12345-0@lo:105: 00010000:00010000:0.0:1787739058.738441:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1ac688 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac68f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.738451:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1ac688 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac68f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.738459:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1ac688 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac68f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.738467:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1ac688 lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac68f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.738474:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1ac688 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac68f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.738483:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1ac688 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ac68f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.738501:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609a600) 00000100:00100000:0.0:1787739058.738537:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665621376:12345-0@lo:105: Request processed in 133us (230us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.738545:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1113 00000100:00100000:0.0:1787739058.738589:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a330c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665621376:0@lo:105: 00010000:00010000:0.0:1787739058.738604:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739058.738652:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304003c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:2408:x1874579652450944:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 546us (914us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.738659:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1455 00010000:00010000:0.0:1787739058.738796:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1ac688 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac68f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.738806:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1ac688 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac68f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739058.738825:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739058.738856:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac696 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.738864:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac696 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.738873:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac696 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739058.738888:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c0000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665621504:0@lo:101:mdt00_000.0 00000100:00100000:0.0:1787739058.738917:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.738937:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739058.738950:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665621504 00000100:00100000:0.0:1787739058.738954:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665621504 00000100:00100000:0.0:1787739058.738981:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2909 00000100:00100000:0.0:1787739058.738988:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c12c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665621504:12345-0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739058.739003:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739058.739016:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ac69d lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac696 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.739035:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ac69d lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac696 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.739063:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ac69d lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac696 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.739070:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x8102366f (ffffa073068a9600) 00010000:00010000:0.0:1787739058.739073:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739058.739076:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b200, rc 0) 00000100:00100000:0.0:1787739058.739113:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c12c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665621504:12345-0@lo:101:mdt00_000.0 Request processed in 124us (194us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.739119:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2909 00000100:00100000:0.0:1787739058.739150:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c0000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665621504:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739058.739161:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac696 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac69d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.739170:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac696 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac69d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739058.739182:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:0.0:1787739058.739213:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c0000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665621632:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739058.739239:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.739255:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739058.739271:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665621632 00000100:00100000:0.0:1787739058.739274:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665621632 00000100:00100000:0.0:1787739058.739298:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2910 00000100:00100000:0.0:1787739058.739305:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665621632:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739058.739363:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665621632:12345-0@lo:1000:mdt00_000.0 Request processed in 58us (124us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.739370:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2910 00000100:00100000:0.0:1787739058.739394:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c0000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665621632:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739058.739465:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a330c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665621760:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739058.739488:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.739503:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739058.739528:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665621760 00000100:00100000:0.0:1787739058.739531:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665621760 00000100:00100000:0.0:1787739058.739553:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2911 00000100:00100000:0.0:1787739058.739559:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665621760:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739058.739622:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665621760:12345-0@lo:1000:mdt00_000.0 Request processed in 63us (133us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739058.739628:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2911 00000100:00100000:0.0:1787739058.739655:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a330c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665621760:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739058.740886:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a330c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665621888:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739058.740931:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.741003:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739058.741329:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665621888 00000100:00100000:2.0:1787739058.741336:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665621888 00000100:00100000:2.0:1787739058.741429:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2912 00000100:00100000:2.0:1787739058.741439:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665621888:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739058.741531:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665621888:12345-0@lo:1000:mdt00_000.0 Request processed in 93us (599us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739058.741539:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2912 00000100:00100000:3.0:1787739058.741606:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a330c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665621888:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739058.741664:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304002d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665622016:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739058.741712:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739058.741996:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739058.742032:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665622016 00000100:00100000:0.0:1787739058.742037:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665622016 00000100:00100000:0.0:1787739058.742117:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2913 00000100:00100000:0.0:1787739058.742125:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304001680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665622016:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739058.742181:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304001680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665622016:12345-0@lo:1000:mdt00_000.0 Request processed in 58us (469us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.742188:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2913 00000100:00100000:3.0:1787739058.742252:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304002d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665622016:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739058.742304:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304002d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665622144:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739058.742329:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739058.742345:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739058.742357:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665622144 00000100:00100000:3.0:1787739058.742360:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665622144 00000100:00100000:3.0:1787739058.742380:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2914 00000100:00100000:3.0:1787739058.742386:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304000b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665622144:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739058.742472:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304000b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665622144:12345-0@lo:1000:mdt00_000.0 Request processed in 84us (140us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739058.742480:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2914 00000100:00100000:3.0:1787739058.742509:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304002d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665622144:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739058.742549:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304002d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665622272:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739058.742573:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739058.742591:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739058.742605:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665622272 00000100:00100000:3.0:1787739058.742608:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665622272 00000100:00100000:3.0:1787739058.742626:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2915 00000100:00100000:3.0:1787739058.742631:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073040012c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665622272:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739058.742689:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073040012c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665622272:12345-0@lo:1000:mdt00_000.0 Request processed in 58us (115us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739058.742700:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2915 00000100:00100000:3.0:1787739058.742724:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304002d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665622272:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739058.742761:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304002d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665622400:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739058.742787:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739058.742804:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739058.742935:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665622400 00000100:00100000:3.0:1787739058.742940:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665622400 00000100:00100000:3.0:1787739058.742960:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2916 00000100:00100000:3.0:1787739058.742967:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304000780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665622400:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739058.743017:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304000780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665622400:12345-0@lo:1000:mdt00_000.0 Request processed in 50us (229us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739058.743023:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2916 00000100:00100000:3.0:1787739058.743055:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304002d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665622400:0@lo:1000:mdt00_000.0 00000040:00080000:3.0:1787739058.744198:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].346, 2168 off526456 00000040:00080000:3.0:1787739058.744295:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].346, 2368 off575272 00000004:00080000:3.0:1787739058.744302:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07410404240 with version 658 00000004:00080000:0.0:1787739058.744439:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 658 rpc_version 658 00000020:00100000:0.0:1787739058.744476:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739058.744487:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07305f7b480 x1874579665622656/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739058.744508:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7b480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665622656:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739058.744577:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.744598:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739058.744617:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665622656 00000100:00100000:0.0:1787739058.744622:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665622656 00000100:00100000:0.0:1787739058.744656:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2917 00000100:00100000:0.0:1787739058.744663:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665622656:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739058.746039:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665622656:12345-0@lo:1000:osp_up0-1.0 Request processed in 1372us (1457us total) trans 4294983354 rc 0/0 00000100:00100000:0.0:1787739058.746059:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2917 00000100:00100000:0.0:1787739058.746172:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7b480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665622656:0@lo:1000:osp_up0-1.0 00010000:00010000:3.0:1787739058.746297:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ac6a4 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x16d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.746313:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ac6a4 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x16d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.746566:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ac6a4 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x16d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.746591:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac6ab lrc: 3/0,1 mode: --/PW res: [0x240000400:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.746599:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac6ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.746607:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac6ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.746638:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac6b2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.746646:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac6b2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.746655:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac6b2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.746672:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304002d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665622784:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739058.746899:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739058.746970:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074104049c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739058.747013:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665622784 00000100:00100000:0.0:1787739058.747019:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665622784 00000100:00100000:0.0:1787739058.747106:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2918 00000100:00100000:0.0:1787739058.747115:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304000000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665622784:12345-0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739058.747132:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739058.747148:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac6b9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac6b2 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.747167:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac6b9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac6b2 expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.747179:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac6b9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac6b2 expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.747193:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbd:0x0].0x0 (ffffa073068a9d00) 00010000:00010000:0.0:1787739058.747196:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739058.747199:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609aa00, rc 0) 00000100:00100000:0.0:1787739058.747240:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304000000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665622784:12345-0@lo:101:mdt00_000.0 Request processed in 127us (342us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.747246:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2918 00000100:00100000:3.0:1787739058.747310:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304002d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665622784:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739058.747323:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac6b2 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac6b9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.747333:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac6b2 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac6b9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739058.747352:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c3cf8569 reply state 00000000dea58f05 transno 4294968346 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac6ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.747364:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac6ab lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.747370:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xbd:0x0].0x0 (ffffa07305d56100) 00010000:00010000:3.0:1787739058.747372:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739058.747375:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787739058.747381:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac6b2 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac6b9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.747388:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac6b2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac6b9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739058.747397:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xbd:0x0] 00000004:00010000:3.0:1787739058.747402:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c3cf8569 reply state 00000000dea58f05 transno 4294968346 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ac6a4 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x16d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.747410:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ac6a4 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x16d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.747431:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16d:0x0].0x0 (ffffa07305d56e00) 00010000:00010000:3.0:1787739058.747433:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739058.747435:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:3.0:1787739058.747440:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x16d:0x0]: returning size 4096 00000004:00080000:3.0:1787739058.747446:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739058.747454:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1ac688 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac68f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.747462:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1ac688 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac68f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.747469:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1ac688 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac68f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.747486:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739058.747490:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739058.747494:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac696 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac69d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.747501:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac696 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac69d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.747574:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073040003c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17525:x1874579652450816:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 21590us (24083us total) trans 4294968346 rc 0/0 00000100:00100000:3.0:1787739058.747581:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23328 00010000:00010000:2.0:1787739058.749032:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739058.749044:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1ac688 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac68f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.749063:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739058.749069:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1ac688 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac68f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739058.749083:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1ac688 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac68f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.749116:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1ac688 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac68f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.749123:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739058.749146:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1ac688 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac68f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.749154:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1ac688 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac68f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.749172:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739058.749174:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739058.749175:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739058.749189:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306006d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665622912:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739058.749240:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.749340:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665622912 00000100:00100000:0.0:1787739058.749346:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665622912 00000100:00100000:0.0:1787739058.749392:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1265 00000100:00100000:0.0:1787739058.749401:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+11:3651:x1874579665622912:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739058.749432:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739058.749451:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac68f lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac688 expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.749464:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5300) 00010000:00010000:0.0:1787739058.749472:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739058.749506:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+10:3651:x1874579665622912:12345-0@lo:103:ldlm_bl.0 Request processed in 106us (265us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.749512:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1265 00000100:00100000:2.0:1787739058.749596:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306006d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665622912:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739058.778277:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739058.778371:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652451200 00000100:00100000:0.0:1787739058.778379:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652451200 00000100:00100000:0.0:1787739058.778473:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23329 00000100:00100000:0.0:1787739058.778484:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304002d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17526:x1874579652451200:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:0.0:1787739058.778635:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304002d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17526:x1874579652451200:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 152us (370us total) trans 0 rc -17/-17 00000100:00100000:0.0:1787739058.778646:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23329 00000100:00100000:3.0:1787739058.845534:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739058.845708:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652451328 00000100:00100000:1.0:1787739058.845718:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652451328 00000100:00100000:1.0:1787739058.845773:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23330 00000100:00100000:1.0:1787739058.845784:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a330c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17527:x1874579652451328:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739058.845808:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739058.845830:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac6c0 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094da2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.845886:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac6c7 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.845899:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac6c7 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.845910:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac6c7 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.845921:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac6ce lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.845932:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ac69d lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac696 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.846002:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac6ce lrc: 4/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.846034:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ac69d lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ac696 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739058.846061:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157e9e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665623040:0@lo:104: 00000100:00100000:1.0:1787739058.846108:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739058.846216:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e900 going to sleep for 11 seconds 00000100:00100000:2.0:1787739058.846697:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665623040 00000100:00100000:2.0:1787739058.846703:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665623040 00000100:00100000:2.0:1787739058.846775:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1114 00000100:00100000:2.0:1787739058.846784:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665623040:12345-0@lo:104: 00010000:00010000:2.0:1787739058.846796:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac696 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac69d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.846870:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739058.846877:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665623040:12345-0@lo:104: Request processed in 94us (768us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.846884:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1114 00000100:00100000:1.0:1787739058.846955:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074157e9e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665623040:0@lo:104: 00010000:00010000:1.0:1787739058.846977:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac6ce lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.847027:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739058.847035:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac696 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac69d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.847043:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739058.847058:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac696 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ac69d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739058.847066:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac696 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ac69d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.847092:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac696 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac69d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.847099:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739058.847127:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac696 lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac69d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.847135:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ac696 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac69d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.847152:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739058.847154:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739058.847156:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739058.847163:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787739058.847173:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c416940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665623168:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739058.847205:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739058.847245:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665623168 00000100:00100000:3.0:1787739058.847250:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665623168 00010000:00010000:3.0:1787739058.847268:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ac69d lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ac696 expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.847291:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1456 00000100:00100000:3.0:1787739058.847297:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074184c7050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+10:3653:x1874579665623168:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739058.847317:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:3.0:1787739058.853580:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983354 is committed 00010000:00010000:1.0:1787739058.853721:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ac69d lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ac696 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.853741:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x8102366f (ffffa073068a9600) 00010000:00010000:1.0:1787739058.853746:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac6ce lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.853756:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac6ce lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.853763:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac6ce lrc: 4/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.853795:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739058.853874:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074184c7050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+7:3653:x1874579665623168:12345-0@lo:103:ldlm_bl.0 Request processed in 6575us (6666us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.853883:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1456 00010000:00010000:2.0:1787739058.854131:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac6ce lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.854140:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac6ce lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.854143:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c416940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665623168:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739058.854252:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306006d00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665623296:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739058.854321:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739058.854348:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba000 going to sleep for 11 seconds 00000100:00100000:0.0:1787739058.856653:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665623296 00000100:00100000:0.0:1787739058.856666:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665623296 00000100:00100000:0.0:1787739058.856725:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2919 00000100:00100000:0.0:1787739058.856734:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306006940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665623296:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739058.856844:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306006940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665623296:12345-0@lo:1000:mdt00_003.0 Request processed in 111us (2535us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739058.856851:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2919 00000100:00100000:2.0:1787739058.856893:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306006d00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665623296:0@lo:1000:mdt00_003.0 00010000:00010000:2.0:1787739058.857193:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac6d5 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x16d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.857210:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac6d5 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.857223:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac6d5 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739058.857247:0:7944:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac6d5 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.857262:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac6c7 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.857269:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac6c7 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.857275:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac6c7 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.857282:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac6c7 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.857289:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739058.857293:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac6c7 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.857300:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac6c7 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.857312:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae900) 00010000:00010000:2.0:1787739058.857316:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739058.857319:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac6c7 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.857325:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac6c7 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.857338:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac6ce lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.857344:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac6ce lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.857350:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac6ce lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.857355:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739058.857358:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac6ce lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.857366:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac6ce lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.857373:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x8102366f (ffffa073068a9600) 00010000:00010000:2.0:1787739058.857376:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739058.857379:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac6ce lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.857384:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac6ce lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.858132:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac6c0 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094da2 expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.858157:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac6d5 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x16d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094da2 expref: 14 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.858165:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16d:0x0].0x0 (ffffa074419d5500) 00010000:00010000:2.0:1787739058.858168:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739058.858170:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c600, rc 0) 00000100:00100000:2.0:1787739058.858276:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a330c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17527:x1874579652451328:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 12493us (12750us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.858284:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23330 00000100:00100000:2.0:1787739058.864691:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739058.864880:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652451456 00000100:00100000:3.0:1787739058.865019:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652451456 00000100:00100000:3.0:1787739058.865089:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23331 00000100:00100000:3.0:1787739058.865100:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306006d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17527:x1874579652451456:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739058.865123:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739058.865143:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ac6dc lrc: 2/0,0 mode: --/CR res: [0x240000402:0x16d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094da9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.865203:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ac6e3 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x16d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.865216:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ac6e3 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.865231:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ac6e3 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739058.865334:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x16d:0x0]: returning size 4096 00000004:00200000:3.0:1787739058.865339:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739058.865376:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ac6dc lrc: 0/0,0 mode: --/CR res: [0x240000402:0x16d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094da9 expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.865393:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ac6e3 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x16d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094da9 expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.865402:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16d:0x0].0x0 (ffffa07305d56e00) 00010000:00010000:3.0:1787739058.865404:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739058.865409:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aaa00, rc 0) 00000100:00100000:3.0:1787739058.865511:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306006d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17527:x1874579652451456:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 413us (828us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739058.865519:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23331 00000100:00100000:3.0:1787739058.872229:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739058.872332:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652451584 00000100:00100000:2.0:1787739058.872342:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652451584 00000100:00100000:2.0:1787739058.872400:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23332 00000100:00100000:2.0:1787739058.872448:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c416940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17527:x1874579652451584:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739058.872472:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739058.872493:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac6ea lrc: 2/0,0 mode: --/CR res: [0x240000400:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094db0 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.872554:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac6f1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.872568:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac6f1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.872580:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac6f1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739058.872630:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xbd:0x0]: returning size 4096 00000004:00200000:2.0:1787739058.872635:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739058.872668:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac6ea lrc: 0/0,0 mode: --/CR res: [0x240000400:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094db0 expref: 19 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.872687:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac6f1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094db0 expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.872696:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xbd:0x0].0x0 (ffffa07305d56100) 00010000:00010000:2.0:1787739058.872699:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739058.872702:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c400, rc 0) 00000100:00100000:2.0:1787739058.872771:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c416940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17527:x1874579652451584:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 361us (549us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.872778:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23332 00000100:00100000:2.0:1787739058.878278:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739058.878620:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652451712 00000100:00100000:1.0:1787739058.878640:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652451712 00000100:00100000:0.0:1787739058.878862:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23333 00000100:00100000:0.0:1787739058.878875:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306005e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17527:x1874579652451712:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739058.878895:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739058.878915:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac6f8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094db7 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.878974:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac6ff lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.878988:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac6b9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac6b2 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.878998:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac6ff lrc: 4/1,0 mode: --/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.879031:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac6b9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ac6b2 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739058.879055:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fe940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665623424:0@lo:104: 00000100:00100000:0.0:1787739058.879099:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.879143:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739058.879307:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665623424 00000100:00100000:2.0:1787739058.879313:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665623424 00000100:00100000:2.0:1787739058.879353:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1115 00000100:00100000:2.0:1787739058.879361:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fcb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665623424:12345-0@lo:104: 00010000:00010000:2.0:1787739058.879375:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac6b2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac6b9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.879463:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739058.879469:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fcb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665623424:12345-0@lo:104: Request processed in 109us (368us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.879475:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1115 00010000:00010000:3.0:1787739058.879539:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739058.879547:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac6b2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac6b9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.879556:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739058.879561:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac6b2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ac6b9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739058.879571:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac6b2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ac6b9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.879601:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac6b2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac6b9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.879608:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739058.879635:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac6b2 lrc: 1/0,0 mode: --/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac6b9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.879643:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ac6b2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac6b9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.879659:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739058.879661:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739058.879664:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739058.879672:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787739058.879683:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c414b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665623552:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739058.879729:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739058.879770:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665623552 00000100:00100000:3.0:1787739058.879773:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665623552 00010000:00010000:3.0:1787739058.879791:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac6b9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ac6b2 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.879811:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1457 00000100:00100000:3.0:1787739058.879829:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c4152c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1874579665623552:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739058.879838:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739058.880763:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbd:0x0].0x0 (ffffa073068a9d00) 00010000:00010000:3.0:1787739058.880772:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac6ff lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.880786:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac6ff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.880793:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac6ff lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.880819:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739058.880943:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac6b9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ac6b2 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739058.880963:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fe940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665623424:0@lo:104: 00010000:00010000:1.0:1787739058.880980:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac6ff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.880989:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac6ff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.880995:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac6ff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739058.881055:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xbd:0x0]: returning size 4096 00000004:00200000:1.0:1787739058.881061:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739058.881103:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac6f8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094db7 expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.881118:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac6ff lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094db7 expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.881126:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbd:0x0].0x0 (ffffa073068a9d00) 00010000:00010000:1.0:1787739058.881128:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739058.881131:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a600, rc 0) 00000100:00100000:1.0:1787739058.881181:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306005e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17527:x1874579652451712:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 2308us (2910us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.881189:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23333 00000100:00100000:2.0:1787739058.881388:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c414b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665623552:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739058.882002:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c4152c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+6:3652:x1874579665623552:12345-0@lo:103:ldlm_bl.0 Request processed in 2170us (2269us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739058.882012:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1457 00000100:00100000:0.0:1787739058.896505:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739058.896598:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652451840 00000100:00100000:2.0:1787739058.896609:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652451840 00000100:00100000:2.0:1787739058.896678:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1266 00000100:00100000:2.0:1787739058.896690:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061ff840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2409:x1874579652451840:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:2.0:1787739058.896770:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739058.896798:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac6d5 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x16d:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094da2 expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.897082:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16d:0x0].0x0 (ffffa074419d5500) 00010000:00010000:2.0:1787739058.897095:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739058.897156:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061ff840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652451840:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 468us (659us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.897165:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1266 00000100:00100000:3.0:1787739058.906184:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739058.906570:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652452096 00000100:00100000:0.0:1787739058.906582:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652452096 00000100:00100000:0.0:1787739058.906838:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23334 00000100:00100000:0.0:1787739058.906851:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c414b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17527:x1874579652452096:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:0.0:1787739058.906917:0:6523:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739058.906951:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ac6e3 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x16d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094da9 expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.906971:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16d:0x0].0x0 (ffffa07305d56e00) 00010000:00010000:0.0:1787739058.906976:0:6523:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739058.907021:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac706 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.907030:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac706 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.907039:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac706 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739058.907070:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fed00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665623680:0@lo:101:mdt00_000.0 00000100:00100000:0.0:1787739058.907121:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.907161:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.908795:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665623680 00000100:00100000:1.0:1787739058.908806:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665623680 00000100:00100000:1.0:1787739058.908862:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2920 00000100:00100000:1.0:1787739058.908871:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:6523:x1874579665623680:12345-0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739058.908888:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739058.908908:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac70d lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac706 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.908936:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac70d lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac706 expref: 7 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.908951:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac70d lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac706 expref: 7 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.908958:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eaeb00) 00010000:00010000:1.0:1787739058.908961:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739058.908964:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1de00, rc 0) 00000100:00100000:1.0:1787739058.909032:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665623680:12345-0@lo:101:mdt00_000.0 Request processed in 162us (1909us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.909039:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2920 00000100:00100000:0.0:1787739058.909075:0:6523:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983354 gen 2 00000004:00080000:0.0:1787739058.909082:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983353 committed transno 4294983354 00000004:00080000:0.0:1787739058.909105:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983354 committed transno 4294983354 00000100:00100000:0.0:1787739058.909125:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fed00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665623680:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739058.909143:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac706 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac70d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.909154:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac706 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac70d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739058.909174:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739058.909204:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac714 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.909210:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac714 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739058.909217:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac714 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739058.909233:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fed00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665623808:0@lo:101:mdt00_000.0 00000100:00100000:0.0:1787739058.909265:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.909296:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.909686:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665623808 00000100:00100000:1.0:1787739058.909701:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665623808 00000100:00100000:1.0:1787739058.909731:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2921 00000100:00100000:1.0:1787739058.909738:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fcf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665623808:12345-0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739058.909748:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739058.909761:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac71b lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac714 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.909778:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac71b lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac714 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.909794:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac71b lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac714 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.909801:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x8102366f (ffffa07305eae600) 00010000:00010000:1.0:1787739058.909803:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739058.909806:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1ce00, rc 0) 00000100:00100000:1.0:1787739058.909849:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fcf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665623808:12345-0@lo:101:mdt00_000.0 Request processed in 111us (582us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.909855:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2921 00000100:00100000:3.0:1787739058.911892:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fed00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665623808:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739058.911921:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac714 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac71b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.911935:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac714 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac71b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739058.912017:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:3.0:1787739058.912081:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f712c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665623936:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739058.912140:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739058.912195:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.912213:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665623936 00000100:00100000:1.0:1787739058.912222:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665623936 00000100:00100000:1.0:1787739058.912275:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2922 00000100:00100000:1.0:1787739058.912283:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f71680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665623936:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739058.912367:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f71680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665623936:12345-0@lo:1000:mdt00_000.0 Request processed in 84us (227us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.912375:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2922 00000100:00100000:1.0:1787739058.912405:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f712c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665623936:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739058.912488:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a31e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665624064:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739058.912536:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739058.912553:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e240 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.912566:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665624064 00000100:00100000:1.0:1787739058.912569:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665624064 00000100:00100000:1.0:1787739058.912591:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2923 00000100:00100000:1.0:1787739058.912597:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665624064:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739058.912667:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665624064:12345-0@lo:1000:mdt00_000.0 Request processed in 69us (130us total) trans 0 rc 1/1 00000100:00100000:1.0:1787739058.912674:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2923 00000100:00100000:1.0:1787739058.912697:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a31e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665624064:0@lo:1000:mdt00_000.0 00010000:00010000:1.0:1787739058.912744:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ac722 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.912752:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ac722 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.912761:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ac722 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739058.912772:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a31e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665624192:0@lo:101:mdt00_000.0 00000100:00100000:1.0:1787739058.912794:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739058.912807:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ecc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.912818:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665624192 00000100:00100000:1.0:1787739058.912821:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665624192 00000100:00100000:1.0:1787739058.912839:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2924 00000100:00100000:1.0:1787739058.912845:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665624192:12345-0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739058.912853:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739058.912867:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac729 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac722 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.912886:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac729 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac722 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.912897:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac729 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac722 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.912910:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16d:0x0].0x0 (ffffa07305eaee00) 00010000:00010000:1.0:1787739058.912913:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739058.912917:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1da00, rc 0) 00000100:00100000:1.0:1787739058.912945:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665624192:12345-0@lo:101:mdt00_000.0 Request processed in 100us (150us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.912950:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2924 00000100:00100000:1.0:1787739058.912976:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a31e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665624192:0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739058.912988:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ac722 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac729 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.912997:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ac722 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac729 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.913019:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac730 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.913027:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac730 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.913035:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac730 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.913061:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac737 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.913072:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac6f1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094db0 expref: 17 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.913095:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac737 lrc: 4/0,1 mode: --/EX res: [0x240000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.913118:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac6f1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094db0 expref: 17 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.913131:0:6523:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac6f1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094db0 expref: 17 pid: 6525 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739058.913141:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a31e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665624320:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739058.913196:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ecc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.933005:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739058.933056:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652452224 00000100:00100000:1.0:1787739058.933065:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652452224 00010000:00010000:1.0:1787739058.933103:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac6f1 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094db0 expref: 18 pid: 6525 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739058.933138:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1458 00000100:00100000:1.0:1787739058.933147:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304003c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2408:x1874579652452224:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739058.933161:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739058.933168:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac6f1 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094db0 expref: 18 pid: 6525 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.933181:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac6f1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094db0 expref: 18 pid: 6525 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.933208:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xbd:0x0].0x0 (ffffa07305d56100) 00010000:00010000:1.0:1787739058.933213:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac737 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.933223:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac737 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.933230:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac737 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.933246:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739058.933288:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304003c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2408:x1874579652452224:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 143us (293us total) trans 0 rc 0/0 00010000:00010000:3.0:1787739058.938508:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac6f1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094db0 expref: 18 pid: 6525 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.938532:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a31e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665624320:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739058.938553:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac737 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.938563:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac737 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.938570:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac737 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.938612:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aba00/0x7ee4453e4a1ac73e lrc: 3/0,1 mode: --/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.938619:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aba00/0x7ee4453e4a1ac73e lrc: 3/0,1 mode: --/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.938631:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aba00/0x7ee4453e4a1ac73e lrc: 3/0,1 mode: --/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.938650:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f72580 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665624448:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739058.938704:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739058.938738:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.938925:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1458 00000100:00100000:1.0:1787739058.939075:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665624448 00000100:00100000:1.0:1787739058.939081:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665624448 00000100:00100000:1.0:1787739058.939125:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2925 00000100:00100000:1.0:1787739058.939132:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f73c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665624448:12345-0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739058.939147:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739058.939163:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac745 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac73e expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.939188:0:17834:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac6ff lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094db7 expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.939197:0:17834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac745 lrc: 4/0,0 mode: --/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac73e expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.939224:0:17834:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac6ff lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094db7 expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.939236:0:17834:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac6ff lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094db7 expref: 13 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739058.939248:0:17834:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157eb0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:17834:1874579665624576:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739058.939383:0:17834:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e240 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.946660:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739058.947406:0:17834:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074157eb0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:17834:1874579665624576:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739058.947475:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac745 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac73e expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.947488:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbd:0x0].0x0 (ffffa073068a9d00) 00010000:00010000:2.0:1787739058.947492:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739058.947496:0:17834:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac745 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac73e expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.947506:0:17834:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac745 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac73e expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.947514:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d000, rc 0) 00000100:00100000:2.0:1787739058.947571:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f73c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665624448:12345-0@lo:101:mdt00_000.0 Request processed in 8439us (8867us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.947579:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2925 00000100:00100000:1.0:1787739058.949492:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f72580 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665624448:0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739058.949519:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aba00/0x7ee4453e4a1ac73e lrc: 4/0,1 mode: --/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac745 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.949532:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aba00/0x7ee4453e4a1ac73e lrc: 4/0,1 mode: --/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac745 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.953605:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652452352 00000100:00100000:3.0:1787739058.953618:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652452352 00010000:00010000:2.0:1787739058.959255:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac6ff lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094db7 expref: 14 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739058.959302:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1459 00000100:00100000:2.0:1787739058.959313:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157e9e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2407:x1874579652452352:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739058.959330:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739058.959337:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac6ff lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094db7 expref: 14 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.959351:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac6ff lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094db7 expref: 14 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.959373:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac6ff lrc: 0/0,0 mode: --/PR res: [0x200000401:0xbd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094db7 expref: 14 pid: 6524 timeout: 745 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.959388:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbd:0x0].0x0 (ffffa073068a9d00) 00010000:00010000:2.0:1787739058.959395:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac745 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac73e expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.959404:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac745 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac73e expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.959436:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac745 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac73e expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.959470:0:6508:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac745 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac73e expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739058.959483:0:6508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004f00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665624704:0@lo:105: 00000100:00100000:2.0:1787739058.959545:0:6508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739058.959590:0:6508:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba000 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.959717:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665624704 00000100:00100000:1.0:1787739058.959723:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665624704 00000100:00100000:1.0:1787739058.959763:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1116 00000100:00100000:1.0:1787739058.959770:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306007480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665624704:12345-0@lo:105: 00010000:00010000:1.0:1787739058.959782:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aba00/0x7ee4453e4a1ac73e lrc: 5/0,1 mode: --/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac745 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.959791:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aba00/0x7ee4453e4a1ac73e lrc: 5/0,1 mode: --/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac745 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.959803:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aba00/0x7ee4453e4a1ac73e lrc: 5/0,1 mode: --/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac745 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.959810:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aba00/0x7ee4453e4a1ac73e lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac745 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.959816:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aba00/0x7ee4453e4a1ac73e lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac745 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.959822:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aba00/0x7ee4453e4a1ac73e lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ac745 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.959841:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069aba00) 00000100:00100000:1.0:1787739058.959891:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306007480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665624704:12345-0@lo:105: Request processed in 121us (345us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.959898:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1116 00000100:00100000:2.0:1787739058.960162:0:6508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004f00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665624704:0@lo:105: 00010000:00010000:2.0:1787739058.960177:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739058.960223:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157e9e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:2407:x1874579652452352:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 911us (13571us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.960231:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1459 00010000:00010000:3.0:1787739058.964569:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aba00/0x7ee4453e4a1ac73e lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac745 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.964588:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aba00/0x7ee4453e4a1ac73e lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac745 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739058.965284:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f72580 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665624832:0@lo:403:mdt00_000.0 00000100:00100000:3.0:1787739058.965354:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739058.965410:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074104049c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.965754:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665624832 00000100:00100000:1.0:1787739058.965761:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665624832 00000100:00100000:1.0:1787739058.965813:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2926 00000100:00100000:1.0:1787739058.965823:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f70b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665624832:12345-0@lo:403:mdt00_000.0 00000100:00100000:1.0:1787739058.968629:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f70b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665624832:12345-0@lo:403:mdt00_000.0 Request processed in 2798us (3265us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.968644:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2926 00000100:00100000:2.0:1787739058.969163:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f72580 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665624832:0@lo:403:mdt00_000.0 00000040:00080000:1.0:1787739058.970651:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].347, 664 off527120 00000040:00080000:1.0:1787739058.971986:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].347, 744 off576016 00000004:00080000:1.0:1787739058.971995:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730610e480 with version 659 00000004:00080000:0.0:1787739058.972016:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 659 rpc_version 659 00000020:00100000:0.0:1787739058.972061:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739058.972072:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073061fed00 x1874579665625088/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1808/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739058.972094:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fed00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665625088:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739058.972162:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739058.972198:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e28300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739058.972217:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665625088 00000100:00100000:1.0:1787739058.972224:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665625088 00000100:00100000:1.0:1787739058.972270:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2927 00000100:00100000:1.0:1787739058.972278:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07436d8e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665625088:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739058.972646:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07436d8e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:7467:x1874579665625088:12345-0@lo:1000:osp_up0-1.0 Request processed in 368us (483us total) trans 4294983355 rc 0/0 00000100:00100000:1.0:1787739058.972654:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2927 00000100:00100000:3.0:1787739058.973055:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fed00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665625088:0@lo:1000:osp_up0-1.0 00000004:00200000:1.0:1787739058.973133:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x16d:0x0]: returning size 4096 00010000:00010000:1.0:1787739058.973153:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac737 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.973163:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac737 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.973171:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac737 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.973178:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739058.973183:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac737 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.973191:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac737 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.973206:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xbd:0x0].0x0 (ffffa07305d56100) 00010000:00010000:1.0:1787739058.973209:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739058.973214:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac737 lrc: 2/0,0 mode: --/EX res: [0x240000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.973220:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac737 lrc: 0/0,0 mode: --/EX res: [0x240000400:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.973236:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aba00/0x7ee4453e4a1ac73e lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac745 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.973244:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aba00/0x7ee4453e4a1ac73e lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac745 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.973251:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aba00/0x7ee4453e4a1ac73e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac745 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.973276:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739058.973285:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xbd:0x0] 00010000:00010000:1.0:1787739058.973291:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac730 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.973298:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac730 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.973304:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac730 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.973309:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1cc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739058.973312:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac730 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.973317:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac730 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.973325:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16d:0x0].0x0 (ffffa07305d56e00) 00010000:00010000:1.0:1787739058.973327:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739058.973331:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac730 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.973336:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac730 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.973346:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ac722 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac729 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.973354:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ac722 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac729 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.973360:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ac722 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac729 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.973372:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1787739058.973379:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00000004:00080000:1.0:1787739058.973381:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739058.973384:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aba00/0x7ee4453e4a1ac73e lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac745 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.973385:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac706 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac70d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.973392:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aba00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739058.973393:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac706 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac70d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.973397:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aba00/0x7ee4453e4a1ac73e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac745 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.973400:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac706 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac70d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739058.973405:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aba00/0x7ee4453e4a1ac73e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac745 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.973427:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:1.0:1787739058.973431:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739058.973435:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac714 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac71b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.973446:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac714 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac71b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.973449:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aba00/0x7ee4453e4a1ac73e lrc: 2/0,0 mode: --/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac745 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.973455:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739058.973469:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aba00/0x7ee4453e4a1ac73e lrc: 1/0,0 mode: --/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac745 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.973476:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aba00/0x7ee4453e4a1ac73e lrc: 0/0,0 mode: --/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac745 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.973489:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787739058.973492:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ac722 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac729 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.973498:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739058.973501:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ac722 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac729 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739058.973503:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c414b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17527:x1874579652452096:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 66654us (67328us total) trans 4294968347 rc 0/0 00000100:00100000:1.0:1787739058.973511:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23334 00000004:00010000:3.0:1787739058.973513:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ac722 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac729 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.973528:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ac722 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac729 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.973534:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739058.973540:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ac722 lrc: 1/0,0 mode: --/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac729 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.973546:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ac722 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac729 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.973558:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739058.973561:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac706 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac70d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.973566:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739058.973569:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac706 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac70d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739058.973575:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac706 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac70d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.973588:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac706 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac70d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.973594:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739058.973610:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac706 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac70d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.973616:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a400/0x7ee4453e4a1ac706 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac70d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739058.973628:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739058.973630:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739058.973631:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739058.973637:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739058.973836:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f70b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665625472:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739058.974528:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739058.974540:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f71680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665625216:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739058.974586:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739058.974615:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787739058.974620:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f73c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665625344:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739058.974636:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739058.974665:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665625216 00000100:00100000:2.0:1787739058.974671:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665625216 00000100:00100000:2.0:1787739058.974704:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665625344 00000100:00100000:2.0:1787739058.974706:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665625344 00000100:00100000:2.0:1787739058.974725:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1267 00000100:00100000:2.0:1787739058.974732:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+12:3650:x1874579665625216:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739058.974743:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739058.974761:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ac745 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xbd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac73e expref: 12 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.974776:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbd:0x0].0x0 (ffffa073068a9d00) 00010000:00010000:2.0:1787739058.974784:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739058.974810:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+11:3650:x1874579665625216:12345-0@lo:103:ldlm_bl.0 Request processed in 78us (224us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.974816:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1267 00000100:00100000:2.0:1787739058.974829:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1268 00000100:00100000:2.0:1787739058.974834:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306006580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+10:3650:x1874579665625344:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739058.974840:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739058.974848:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac729 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x16d:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac722 expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739058.974857:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16d:0x0].0x0 (ffffa07305eaee00) 00010000:00010000:2.0:1787739058.974863:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739058.974881:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306006580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3650:x1874579665625344:12345-0@lo:103:ldlm_bl.0 Request processed in 47us (244us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739058.974890:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1268 00000100:00100000:2.0:1787739058.974918:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f73c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665625344:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739058.974926:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f71680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665625216:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739058.977860:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739058.978007:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665625472 00000100:00100000:1.0:1787739058.978015:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665625472 00000100:00100000:1.0:1787739058.978084:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1269 00000100:00100000:1.0:1787739058.978095:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074184c4450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+9:3652:x1874579665625472:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739058.978111:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739058.978134:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac70d lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac706 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739058.978151:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eaeb00) 00010000:00010000:1.0:1787739058.978161:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739058.978210:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074184c4450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+8:3652:x1874579665625472:12345-0@lo:103:ldlm_bl.0 Request processed in 118us (358us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739058.978218:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1269 00000100:00100000:3.0:1787739058.978331:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f70b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665625472:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739059.021271:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739059.021347:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652452480 00000100:00100000:2.0:1787739059.021356:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652452480 00000100:00100000:2.0:1787739059.021439:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23335 00000100:00100000:2.0:1787739059.021450:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157e9e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17528:x1874579652452480:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739059.021468:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739059.021488:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac74c lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094dbe expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.021552:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac753 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.021567:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac753 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.021579:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac753 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.021592:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac75a lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.021599:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac75a lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.021606:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac75a lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.021647:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac753 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.021655:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac753 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.021662:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac753 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.021669:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac753 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.021675:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739059.021679:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac753 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.021687:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac753 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.021700:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5500) 00010000:00010000:2.0:1787739059.021702:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739059.021707:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac753 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.021714:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac753 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.021727:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac75a lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.021734:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac75a lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.021740:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac75a lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.021746:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739059.021749:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac75a lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.021755:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac75a lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.021762:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xb6ebd3c3 (ffffa074419d5600) 00010000:00010000:2.0:1787739059.021765:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739059.021768:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac75a lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.021773:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac75a lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.021802:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac74c lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094dbe expref: 14 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.021812:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5500) 00010000:00010000:2.0:1787739059.021815:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739059.021818:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac74c lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094dbe expref: 14 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.021830:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633cc00, rc 0) 00000100:00100000:2.0:1787739059.021879:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157e9e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17528:x1874579652452480:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 431us (613us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739059.021890:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23335 00000100:00080000:1.0:1787739059.021956:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787739059.027178:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739059.027259:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652452608 00000100:00100000:0.0:1787739059.027270:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652452608 00000100:00100000:0.0:1787739059.027341:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23336 00000100:00100000:0.0:1787739059.027351:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306005680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17528:x1874579652452608:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739059.027370:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739059.027391:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ac761 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094dc5 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.027490:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ac768 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.027504:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ac768 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.027516:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ac768 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.027529:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac76f lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.027536:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac76f lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.027541:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac76f lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.027578:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ac768 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.027585:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ac768 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.027591:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ac768 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.027602:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ac768 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.027608:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739059.027612:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ac768 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.027618:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ac768 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.027629:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9b00) 00010000:00010000:0.0:1787739059.027633:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739059.027637:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ac768 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.027643:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ac768 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.027653:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac76f lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.027659:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac76f lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.027665:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac76f lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.027670:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739059.027673:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac76f lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.027683:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac76f lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.027690:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xb6ebd3c3 (ffffa073068a9000) 00010000:00010000:0.0:1787739059.027693:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739059.027695:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac76f lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.027701:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac76f lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.027729:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ac761 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094dc5 expref: 14 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.027740:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9b00) 00010000:00010000:0.0:1787739059.027742:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739059.027745:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ac761 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094dc5 expref: 14 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.027754:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a800, rc 0) 00000100:00100000:0.0:1787739059.027820:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306005680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17528:x1874579652452608:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 470us (651us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739059.027827:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23336 00000100:00100000:1.0:1787739059.043886:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739059.043974:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652452736 00000100:00100000:3.0:1787739059.043983:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652452736 00000100:00100000:0.0:1787739059.044130:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23337 00000100:00100000:0.0:1787739059.044144:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157ead00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17528:x1874579652452736:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739059.044165:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739059.044187:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ac776 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094dcc expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.044248:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac77d lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.044262:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac77d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.044274:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac77d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739059.044317:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:0.0:1787739059.044322:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739059.044357:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ac776 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094dcc expref: 15 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.044375:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac77d lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094dcc expref: 14 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.044384:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9b00) 00010000:00010000:0.0:1787739059.044387:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739059.044390:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a000, rc 0) 00000100:00100000:0.0:1787739059.044483:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157ead00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17528:x1874579652452736:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 343us (607us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739059.044489:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23337 00000100:00100000:3.0:1787739059.058748:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739059.058822:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652452992 00000100:00100000:2.0:1787739059.058832:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652452992 00000100:00100000:2.0:1787739059.058906:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23338 00000100:00100000:2.0:1787739059.058916:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f70b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17528:x1874579652452992:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:2.0:1787739059.058972:0:7944:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739059.058999:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac77d lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094dcc expref: 14 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.059016:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9b00) 00010000:00010000:2.0:1787739059.059025:0:7944:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739059.059092:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac784 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.059105:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac784 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.059114:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac784 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.059127:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac78b lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.059133:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac78b lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.059139:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac78b lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1787739059.059952:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].360, 2168 off552520 00000040:00080000:2.0:1787739059.060034:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].360, 2368 off598704 00000004:00080000:2.0:1787739059.060042:0:7944:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa073068bae40 with version 687 00000004:00080000:2.0:1787739059.060076:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 687 rpc_version 687 00000020:00100000:2.0:1787739059.060110:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739059.060124:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306004000 x1874579665625728/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 4232/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739059.060149:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665625728:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739059.060212:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739059.060263:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46cc00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739059.060283:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665625728 00000100:00100000:2.0:1787739059.060288:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665625728 00000100:00100000:2.0:1787739059.060322:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2928 00000100:00100000:2.0:1787739059.060330:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060061c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665625728:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739059.060752:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060061c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:7740:x1874579665625728:12345-0@lo:1000:osp_up1-0.0 Request processed in 421us (539us total) trans 4294968348 rc 0/0 00000100:00100000:2.0:1787739059.060760:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2928 00000100:00100000:2.0:1787739059.060817:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665625728:0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1787739059.060906:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac792 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x150b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.060919:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac792 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x150b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.060930:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac792 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x150b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.060952:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac799 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.060959:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac799 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.060966:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac799 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.060993:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ac7a0 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.061000:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ac7a0 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.061009:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ac7a0 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739059.061021:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004f00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665625856:0@lo:101:mdt00_003.0 00000100:00100000:2.0:1787739059.061062:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739059.061080:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46cc00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739059.061094:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665625856 00000100:00100000:2.0:1787739059.061098:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665625856 00000100:00100000:2.0:1787739059.061124:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2929 00000100:00100000:2.0:1787739059.061132:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306006580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:7944:x1874579665625856:12345-0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739059.061145:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739059.061159:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac7a7 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac7a0 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.061177:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac7a7 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac7a0 expref: 12 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.061191:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac7a7 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac7a0 expref: 12 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.061199:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc0:0x0].0x0 (ffffa074419d5a00) 00010000:00010000:2.0:1787739059.061201:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739059.061204:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d600, rc 0) 00000100:00100000:2.0:1787739059.061238:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306006580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:7944:x1874579665625856:12345-0@lo:101:mdt00_003.0 Request processed in 107us (175us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.061244:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2929 00000100:00100000:2.0:1787739059.061273:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004f00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665625856:0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739059.061284:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ac7a0 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac7a7 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.061293:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ac7a0 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac7a7 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739059.061312:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000574d0089 reply state 000000008ec8fe13 transno 4294983356 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac799 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.061324:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac799 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.061330:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc0:0x0].0x0 (ffffa074419d5700) 00010000:00010000:2.0:1787739059.061333:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739059.061336:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787739059.061342:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ac7a0 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac7a7 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.061354:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ac7a0 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac7a7 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739059.061362:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xc0:0x0] 00000004:00010000:2.0:1787739059.061368:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000574d0089 reply state 000000008ec8fe13 transno 4294983356 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac792 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x150b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.061374:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac792 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x150b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.061379:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x150b:0x0].0x0 (ffffa074419d5e00) 00010000:00010000:2.0:1787739059.061381:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739059.061383:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:2.0:1787739059.061387:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x150b:0x0]: returning size 4096 00010000:00010000:2.0:1787739059.061393:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac784 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.061398:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac784 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.061402:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac784 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.061408:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac784 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.061431:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633cc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739059.061435:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac784 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.061442:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac784 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.061451:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5500) 00010000:00010000:2.0:1787739059.061453:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739059.061456:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac784 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.061461:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac784 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739059.061479:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000574d0089 reply state 000000008ec8fe13 transno 4294983356 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac78b lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.061486:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac78b lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.061492:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xb6ebd3c3 (ffffa074419d5600) 00010000:00010000:2.0:1787739059.061494:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739059.061495:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 2 00000100:00100000:2.0:1787739059.061557:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f70b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17528:x1874579652452992:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 2641us (2815us total) trans 4294983356 rc 0/0 00000100:00100000:2.0:1787739059.061565:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23338 00000100:00100000:3.0:1787739059.084090:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739059.084138:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652453248 00000100:00100000:3.0:1787739059.084147:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652453248 00000100:00100000:3.0:1787739059.084199:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23339 00000100:00100000:3.0:1787739059.084209:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f72580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17529:x1874579652453248:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:3.0:1787739059.084391:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f72580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17529:x1874579652453248:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 182us (308us total) trans 0 rc -17/-17 00000100:00100000:3.0:1787739059.085581:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23339 00000100:00100000:1.0:1787739059.127670:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739059.129960:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652453376 00000100:00100000:2.0:1787739059.129976:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652453376 00000100:00100000:2.0:1787739059.130082:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23340 00000100:00100000:2.0:1787739059.130093:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157ea940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17530:x1874579652453376:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739059.130113:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739059.130136:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac7ae lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094dd3 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130195:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac7b5 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130209:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac7b5 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130220:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac7b5 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130230:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac7bc lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130238:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac7bc lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130245:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac7bc lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130313:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac7c3 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x150b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130320:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac7c3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x150b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130326:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac7c3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x150b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739059.130397:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x150b:0x0]: returning size 4096 00000004:00200000:2.0:1787739059.130402:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:2.0:1787739059.130438:0:6523:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac7c3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x150b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130451:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac7b5 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130461:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac7b5 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130466:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac7b5 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130472:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac7b5 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130479:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739059.130483:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac7b5 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130490:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac7b5 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130500:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5500) 00010000:00010000:2.0:1787739059.130503:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739059.130507:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac7b5 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130513:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac7b5 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130525:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac7bc lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130532:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac7bc lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130538:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac7bc lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130543:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739059.130546:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac7bc lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130551:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac7bc lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130558:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xb6ebd3c3 (ffffa074419d5600) 00010000:00010000:2.0:1787739059.130561:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739059.130564:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac7bc lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130570:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac7bc lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130593:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac7ae lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094dd3 expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130610:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac7c3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x150b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094dd3 expref: 16 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.130617:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x150b:0x0].0x0 (ffffa074419d5e00) 00010000:00010000:2.0:1787739059.130620:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787739059.130623:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633da00, rc 0) 00000100:00100000:2.0:1787739059.131872:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157ea940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17530:x1874579652453376:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 599us (3030us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.131944:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23340 00000100:00100000:2.0:1787739059.142038:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665625984:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787739059.142116:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739059.142180:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665625984 00000100:00100000:2.0:1787739059.142185:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665625984 00000100:00100000:2.0:1787739059.142222:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2930 00000100:00100000:2.0:1787739059.142230:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:3653:x1874579665625984:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1787739059.142259:0:8447:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1445130240 avail 1298890752 00000100:00100000:2.0:1787739059.142297:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:3653:x1874579665625984:12345-0@lo:41:osp-pre-1-0.0 Request processed in 68us (180us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.142304:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2930 00000100:00100000:2.0:1787739059.142346:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665625984:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787739059.145076:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739059.145660:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652453504 00000100:00100000:1.0:1787739059.145671:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652453504 00000100:00100000:1.0:1787739059.145962:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23341 00000100:00100000:1.0:1787739059.145973:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17530:x1874579652453504:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739059.145992:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739059.146014:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac7ca lrc: 2/0,0 mode: --/CR res: [0x200000400:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094dda expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.146079:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac7d1 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.146093:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac7d1 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.146104:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac7d1 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739059.146145:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xc0:0x0]: returning size 4096 00000004:00200000:1.0:1787739059.146149:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739059.146183:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ac7ca lrc: 0/0,0 mode: --/CR res: [0x200000400:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094dda expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.146199:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac7d1 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094dda expref: 17 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.146208:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc0:0x0].0x0 (ffffa074419d5700) 00010000:00010000:1.0:1787739059.146211:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739059.146214:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d800, rc 0) 00000100:00100000:1.0:1787739059.146268:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17530:x1874579652453504:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 298us (1199us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.146280:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23341 00000100:00100000:1.0:1787739059.151238:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739059.151307:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652453632 00000100:00100000:3.0:1787739059.151317:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652453632 00000100:00100000:3.0:1787739059.151385:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23342 00000100:00100000:3.0:1787739059.151395:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157e9a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17530:x1874579652453632:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739059.151449:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739059.151471:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ac7d8 lrc: 2/0,0 mode: --/CR res: [0x240000401:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094de1 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.151535:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac7df lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.151551:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac7a7 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac7a0 expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.151561:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac7df lrc: 4/1,0 mode: --/PR res: [0x240000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.151589:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac7a7 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ac7a0 expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.151603:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f712c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665626112:0@lo:104: 00000100:00100000:3.0:1787739059.151644:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.152026:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.152041:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665626112 00000100:00100000:3.0:1787739059.152054:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665626112 00000100:00100000:3.0:1787739059.152089:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1117 00000100:00100000:3.0:1787739059.152096:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f71680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6525:x1874579665626112:12345-0@lo:104: 00010000:00010000:3.0:1787739059.152107:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ac7a0 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac7a7 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.152153:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739059.152158:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f71680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6525:x1874579665626112:12345-0@lo:104: Request processed in 63us (513us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.152165:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1117 00000100:00100000:3.0:1787739059.152196:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f712c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665626112:0@lo:104: 00010000:00010000:3.0:1787739059.152214:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac7df lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.152228:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739059.152233:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ac7a0 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac7a7 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.152241:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633ce00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739059.152246:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ac7a0 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xc0:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ac7a7 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739059.152255:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ac7a0 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xc0:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ac7a7 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.152278:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ac7a0 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xc0:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac7a7 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.152285:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739059.152304:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ac7a0 lrc: 1/0,0 mode: --/PW res: [0x240000401:0xc0:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac7a7 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.152311:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ac7a0 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xc0:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac7a7 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.152329:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739059.152331:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739059.152333:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739059.152343:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f712c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665626240:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739059.152369:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.152406:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665626240 00000100:00100000:3.0:1787739059.152410:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665626240 00010000:00010000:3.0:1787739059.152447:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac7a7 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ac7a0 expref: 12 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.152467:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1460 00000100:00100000:3.0:1787739059.152472:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f73c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+12:3650:x1874579665626240:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739059.152481:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00080000:1.0:1787739059.152857:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:2.0:1787739059.159627:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac714 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac71b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739059.159650:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac714 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac71b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.159711:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac714 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac71b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.159718:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1787739059.159782:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00010000:00010000:2.0:1787739059.159959:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ac714 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac71b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.159983:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ac5a8 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac5af expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739059.159991:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ac5a8 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac5af expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.160011:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ac5a8 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac5af expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.160018:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739059.160052:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ac5a8 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac5af expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787739059.160067:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968348 is committed 00000001:00080000:2.0:1787739059.160086:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968347 is committed 00000020:00080000:2.0:1787739059.160094:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07410404a80 refcount 2 committed 0 result 0 batchid 353 00000020:00080000:2.0:1787739059.160097:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa074356c7600 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07410404240 00000020:00080000:2.0:1787739059.160101:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].346 00000020:00080000:2.0:1787739059.160103:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07302ea6300 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306aade00 00000020:00080000:2.0:1787739059.160105:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].346 00000020:00080000:2.0:1787739059.160117:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa0740a5f66c0 refcount 2 committed 0 result 0 batchid 352 00000020:00080000:2.0:1787739059.160120:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2600 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa0740a5f6e40 00000020:00080000:2.0:1787739059.160122:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].345 00000020:00080000:2.0:1787739059.160124:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2c00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa0730541e800 00000020:00080000:2.0:1787739059.160127:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].344 00000020:00080000:2.0:1787739059.160131:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 350 batchid updated 00000020:00080000:2.0:1787739059.160141:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa073068ba480 refcount 2 committed 0 result 0 batchid 351 00000020:00080000:2.0:1787739059.160144:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88200 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa073068ba3c0 00000020:00080000:2.0:1787739059.160147:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].344 00000020:00080000:2.0:1787739059.160149:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88500 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa0730541e000 00000020:00080000:2.0:1787739059.160152:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].343 00000100:00100000:2.0:1787739059.160163:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787739059.160180:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306006940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665626496:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:2.0:1787739059.160243:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739059.160503:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665626496 00000100:00100000:2.0:1787739059.160510:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665626496 00000100:00100000:2.0:1787739059.160558:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1270 00000100:00100000:2.0:1787739059.160567:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c381680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+8:3653:x1874579665626496:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:2.0:1787739059.160580:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739059.160596:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac5af lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x83604406 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac5a8 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.160610:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x83604406 (ffffa07305eaed00) 00010000:00010000:2.0:1787739059.160618:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739059.160650:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c381680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+7:3653:x1874579665626496:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 84us (407us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.160657:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1270 00000100:00100000:2.0:1787739059.160702:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306006940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665626496:0@lo:103:jbd2/dm-1-8.0 00010000:00010000:3.0:1787739059.161011:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ac7a7 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ac7a0 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.161024:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc0:0x0].0x0 (ffffa074419d5a00) 00010000:00010000:3.0:1787739059.161029:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac7df lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.161039:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac7df lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.161057:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac7df lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.161077:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739059.161127:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac7df lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.161136:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac7df lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739059.161189:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xc0:0x0]: returning size 4096 00000004:00200000:2.0:1787739059.161194:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739059.161232:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ac7d8 lrc: 0/0,0 mode: --/CR res: [0x240000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094de1 expref: 13 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.161247:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac7df lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094de1 expref: 12 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.161255:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc0:0x0].0x0 (ffffa074419d5a00) 00010000:00010000:2.0:1787739059.161258:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739059.161260:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab000, rc 0) 00000100:00100000:2.0:1787739059.161309:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157e9a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:17530:x1874579652453632:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 9916us (10081us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.161316:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23342 00000100:00100000:1.0:1787739059.161827:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665626368:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787739059.161863:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739059.161992:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665626368 00000100:00100000:2.0:1787739059.161998:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665626368 00000100:00100000:2.0:1787739059.162038:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1271 00000100:00100000:2.0:1787739059.162055:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157e92c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+7:3650:x1874579665626368:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:2.0:1787739059.162065:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739059.162083:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac71b lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x8102366f bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac714 expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.162096:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x8102366f (ffffa07305eae600) 00010000:00010000:2.0:1787739059.162104:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739059.162582:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac538 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x16b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162592:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac538 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x16b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162603:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac538 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x16b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162609:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739059.162613:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac538 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x16b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162621:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac538 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x16b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162631:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16b:0x0].0x0 (ffffa07305d56400) 00010000:00010000:0.0:1787739059.162634:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739059.162638:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac538 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x16b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162644:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ac538 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x16b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162658:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac53f lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162664:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac53f lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162669:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac53f lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162674:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739059.162677:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac53f lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162685:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac53f lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162704:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xbc:0x0].0x0 (ffffa07305d56500) 00010000:00010000:0.0:1787739059.162707:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739059.162710:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac53f lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162714:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ac53f lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xbc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162729:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac626 lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162735:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac626 lrc: 2/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162740:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac626 lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162746:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739059.162748:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac626 lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162754:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac626 lrc: 4/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162761:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x3873ad72 (ffffa07305eaef00) 00010000:00010000:0.0:1787739059.162763:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739059.162767:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac626 lrc: 2/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162773:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac626 lrc: 0/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162783:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac62d lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x16c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162789:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac62d lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x16c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162794:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac62d lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x16c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162799:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739059.162802:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac62d lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x16c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162807:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac62d lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x16c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162825:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16c:0x0].0x0 (ffffa07305eae200) 00010000:00010000:0.0:1787739059.162828:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739059.162830:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac62d lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x16c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162836:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac62d lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x16c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162849:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac657 lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162855:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac657 lrc: 2/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162860:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac657 lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162865:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ca00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739059.162868:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac657 lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162874:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac657 lrc: 4/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162881:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x3873ad72 (ffffa07305eaef00) 00010000:00010000:0.0:1787739059.162892:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739059.162896:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac657 lrc: 2/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162901:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac657 lrc: 0/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0x3873ad72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162913:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ac6a4 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x16d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162918:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ac6a4 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x16d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162923:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ac6a4 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x16d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162927:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739059.162930:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ac6a4 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x16d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162938:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ac6a4 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x16d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162945:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16d:0x0].0x0 (ffffa07305d56e00) 00010000:00010000:0.0:1787739059.162947:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739059.162950:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ac6a4 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x16d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162956:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ac6a4 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x16d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162967:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac6ab lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162972:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac6ab lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162978:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac6ab lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162983:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739059.162985:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac6ab lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162992:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac6ab lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.162999:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xbd:0x0].0x0 (ffffa07305d56100) 00010000:00010000:0.0:1787739059.163001:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739059.163003:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac6ab lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.163018:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ac6ab lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xbd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739059.163531:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157e92c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+6:3650:x1874579665626368:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 1482us (1664us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.163540:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1271 00000100:00100000:1.0:1787739059.163665:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f712c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665626240:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739059.163788:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665626368:0@lo:103:jbd2/dm-1-8.0 00000020:00080000:1.0:1787739059.163820:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 353 committed batchid 350 00000004:00080000:1.0:1787739059.164060:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730610e840 with version 660 00000020:00080000:1.0:1787739059.164079:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 349 cancel update log [0x200000401:0x3:0x0].343: rc = 0 00000020:00080000:1.0:1787739059.164123:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 349 cancel update log [0x240000400:0x3:0x0].342: rc = 0 00000020:00080000:1.0:1787739059.164133:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 350 00000004:00080000:0.0:1787739059.164183:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 660 rpc_version 660 00000020:00100000:0.0:1787739059.164211:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739059.164219:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073061fe940 x1874579665626752/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739059.164240:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fe940 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665626752:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739059.164286:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739059.164312:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355eda80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739059.164361:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665626752 00000100:00100000:2.0:1787739059.164367:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665626752 00000100:00100000:2.0:1787739059.164438:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2931 00000100:00100000:2.0:1787739059.164449:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061ff840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:7467:x1874579665626752:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739059.164570:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061ff840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665626752:12345-0@lo:1000:osp_up0-1.0 Request processed in 124us (284us total) trans 4294983357 rc 0/0 00000100:00100000:2.0:1787739059.164579:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2931 00000100:00100000:0.0:1787739059.164795:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fe940 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665626752:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739059.165186:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f73c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+5:3650:x1874579665626240:12345-0@lo:103:ldlm_bl.0 Request processed in 12711us (12812us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.165195:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1460 00000100:00100000:1.0:1787739059.169615:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739059.169963:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652453888 00000100:00100000:2.0:1787739059.169973:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652453888 00000100:00100000:2.0:1787739059.170131:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23343 00000100:00100000:2.0:1787739059.170143:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157eb0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17530:x1874579652453888:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:2.0:1787739059.170196:0:6523:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739059.170224:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac7c3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x150b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094dd3 expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.170245:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x150b:0x0].0x0 (ffffa074419d5e00) 00010000:00010000:2.0:1787739059.170250:0:6523:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739059.170275:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac7e6 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.170288:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac7e6 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.170297:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac7e6 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.170309:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac7ed lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.170318:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac7ed lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.170326:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac7ed lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.170407:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac7f4 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x150b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.170458:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac7f4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x150b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.170466:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac7f4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x150b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.170484:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac7fb lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.170494:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac7d1 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094dda expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.170502:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac7fb lrc: 4/0,1 mode: --/EX res: [0x200000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.170533:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac7d1 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094dda expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.170548:0:6523:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac7d1 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094dda expref: 16 pid: 6524 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739059.170562:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c382940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665626880:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739059.170631:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46c0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739059.173370:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c382940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665626880:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739059.173397:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac7fb lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739059.174519:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739059.174680:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652454016 00000100:00100000:2.0:1787739059.174689:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652454016 00010000:00010000:2.0:1787739059.174958:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac7d1 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094dda expref: 17 pid: 6524 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739059.175030:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1461 00000100:00100000:2.0:1787739059.175053:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c4152c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2408:x1874579652454016:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739059.175069:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739059.175077:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac7d1 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094dda expref: 17 pid: 6524 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.175090:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac7d1 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094dda expref: 17 pid: 6524 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.175113:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac7d1 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094dda expref: 17 pid: 6524 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.175129:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc0:0x0].0x0 (ffffa074419d5700) 00010000:00010000:2.0:1787739059.175134:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac7fb lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.175144:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac7fb lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.175152:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac7fb lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.175179:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739059.175227:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c4152c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:2408:x1874579652454016:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 189us (713us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.175239:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1461 00010000:00010000:3.0:1787739059.175300:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac7fb lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.175310:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac7fb lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.175365:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1ac802 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.175373:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1ac802 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.175382:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1ac802 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.175401:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f712c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665627008:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739059.175460:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.175512:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404000 going to sleep for 11 seconds 00000100:00100000:1.0:1787739059.177143:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665627008 00000100:00100000:1.0:1787739059.177154:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665627008 00000100:00100000:1.0:1787739059.177210:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2932 00000100:00100000:1.0:1787739059.177218:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+5:6523:x1874579665627008:12345-0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739059.177235:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739059.177252:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac809 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac802 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.177279:0:8447:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac7df lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094de1 expref: 7 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.177304:0:8447:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac809 lrc: 4/0,0 mode: --/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac802 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.177333:0:8447:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac7df lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094de1 expref: 7 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.177347:0:8447:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac7df lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094de1 expref: 7 pid: 6525 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739059.177360:0:8447:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415700b40 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8447:1874579665627136:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739059.177462:0:8447:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ef00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739059.181825:0:8447:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415700b40 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8447:1874579665627136:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739059.181862:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac809 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac802 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.181875:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc0:0x0].0x0 (ffffa074419d5a00) 00010000:00010000:2.0:1787739059.181878:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739059.181893:0:8447:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac809 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac802 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.181904:0:8447:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac809 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac802 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.181911:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1de00, rc 0) 00000100:00100000:2.0:1787739059.181979:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6523:x1874579665627008:12345-0@lo:101:mdt00_000.0 Request processed in 4761us (6518us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.181989:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2932 00000100:00100000:3.0:1787739059.182187:0:6523:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968348 gen 2 00000004:00080000:3.0:1787739059.182193:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968337 committed transno 4294968348 00000020:00080000:3.0:1787739059.182199:0:6523:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07302e89780 refcount 2 committed 0 result 0 batchid 361 00000020:00080000:3.0:1787739059.182203:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730424d500 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07302e89f00 00000020:00080000:3.0:1787739059.182205:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].359 00000020:00080000:3.0:1787739059.182207:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730424df00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304cf5400 00000020:00080000:3.0:1787739059.182210:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].359 00000004:00080000:3.0:1787739059.182236:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968339 committed transno 4294968348 00000004:00080000:3.0:1787739059.182247:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968340 committed transno 4294968348 00000004:00080000:3.0:1787739059.182254:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968341 committed transno 4294968348 00000004:00080000:3.0:1787739059.182261:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968342 committed transno 4294968348 00000004:00080000:3.0:1787739059.182269:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968348 committed transno 4294968348 00000100:00100000:3.0:1787739059.182281:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f712c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665627008:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739059.182297:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1ac802 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac809 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.182313:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1ac802 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac809 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739059.182364:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 362 committed batchid 360 00000020:00080000:2.0:1787739059.182453:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 360 00000100:00100000:1.0:1787739059.194119:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739059.194506:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652454144 00000100:00100000:0.0:1787739059.194517:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652454144 00010000:00010000:0.0:1787739059.194559:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac7df lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094de1 expref: 8 pid: 6525 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739059.194674:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1462 00000100:00100000:2.0:1787739059.194687:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415701680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2408:x1874579652454144:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739059.194703:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739059.194711:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac7df lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094de1 expref: 8 pid: 6525 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.194725:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac7df lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094de1 expref: 8 pid: 6525 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.194749:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac7df lrc: 0/0,0 mode: --/PR res: [0x240000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094de1 expref: 8 pid: 6525 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.194765:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc0:0x0].0x0 (ffffa074419d5a00) 00010000:00010000:2.0:1787739059.194772:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac809 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac802 expref: 5 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.194782:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac809 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac802 expref: 5 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.194791:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac809 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac802 expref: 5 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.194824:0:6508:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac809 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac802 expref: 5 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739059.194838:0:6508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c382940 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001_UUID:6508:1874579665627264:0@lo:105: 00000100:00100000:2.0:1787739059.194962:0:6508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739059.195005:0:6508:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46c0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739059.195068:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665627264 00000100:00100000:0.0:1787739059.195072:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665627264 00000100:00100000:0.0:1787739059.195105:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1118 00000100:00100000:0.0:1787739059.195111:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c380f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6508:x1874579665627264:12345-0@lo:105: 00010000:00010000:0.0:1787739059.195122:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1ac802 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac809 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.195131:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1ac802 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac809 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.195140:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1ac802 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac809 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.195149:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1ac802 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac809 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.195156:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1ac802 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac809 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.195164:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1ac802 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ac809 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.195187:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069ab600) 00000100:00100000:0.0:1787739059.195230:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c380f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6508:x1874579665627264:12345-0@lo:105: Request processed in 118us (268us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739059.195236:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1118 00010000:00010000:2.0:1787739059.195344:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1ac802 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac809 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.195354:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1ac802 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac809 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739059.195444:0:6508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c382940 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001_UUID:6508:1874579665627264:0@lo:105: 00010000:00010000:1.0:1787739059.195456:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739059.195492:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415701680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652454144:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 808us (1383us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.195499:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1462 00000100:00100000:2.0:1787739059.198168:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c382940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665627392:0@lo:403:mdt00_000.0 00000100:00100000:2.0:1787739059.198238:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739059.198279:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739059.200348:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665627392 00000100:00100000:1.0:1787739059.200361:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665627392 00000100:00100000:1.0:1787739059.200462:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2933 00000100:00100000:1.0:1787739059.200473:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f712c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6523:x1874579665627392:12345-0@lo:403:mdt00_000.0 00000100:00100000:1.0:1787739059.210329:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f712c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6523:x1874579665627392:12345-0@lo:403:mdt00_000.0 Request processed in 9848us (12080us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.210344:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2933 00000100:00100000:3.0:1787739059.211594:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c382940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665627392:0@lo:403:mdt00_000.0 00000040:00080000:3.0:1787739059.212722:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].361, 664 off553184 00000040:00080000:3.0:1787739059.217009:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].361, 744 off599448 00000004:00080000:3.0:1787739059.217028:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07410404000 with version 688 00000004:00080000:2.0:1787739059.217276:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 688 rpc_version 688 00000020:00100000:2.0:1787739059.217313:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739059.217324:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306004000 x1874579665627648/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1448/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739059.217348:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665627648:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739059.218039:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739059.218087:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413600 going to sleep for 11 seconds 00000100:00100000:0.0:1787739059.218795:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665627648 00000100:00100000:0.0:1787739059.218806:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665627648 00000100:00100000:0.0:1787739059.218866:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2934 00000100:00100000:0.0:1787739059.218876:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306007c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665627648:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739059.219219:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306007c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665627648:12345-0@lo:1000:osp_up1-0.0 Request processed in 344us (1181us total) trans 4294968349 rc 0/0 00000100:00100000:0.0:1787739059.219227:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2934 00000100:00100000:1.0:1787739059.219579:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665627648:0@lo:1000:osp_up1-0.0 00000004:00200000:0.0:1787739059.224003:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x150b:0x0]: returning size 4096 00010000:00010000:0.0:1787739059.224034:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac7fb lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224056:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac7fb lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224069:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac7fb lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224077:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739059.224081:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac7fb lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224089:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac7fb lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224103:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc0:0x0].0x0 (ffffa074419d5700) 00010000:00010000:0.0:1787739059.224106:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739059.224110:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac7fb lrc: 2/0,0 mode: --/EX res: [0x200000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224117:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac7fb lrc: 0/0,0 mode: --/EX res: [0x200000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224132:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1ac802 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac809 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224140:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1ac802 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac809 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224148:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1ac802 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac809 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224174:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739059.224182:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xc0:0x0] 00010000:00010000:0.0:1787739059.224189:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac7f4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x150b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224196:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac7f4 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x150b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224203:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac7f4 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x150b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224208:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739059.224211:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac7f4 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x150b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224217:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac7f4 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x150b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224224:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x150b:0x0].0x0 (ffffa074419d5e00) 00010000:00010000:0.0:1787739059.224226:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739059.224229:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac7f4 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x150b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224235:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac7f4 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x150b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224250:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac7e6 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224256:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac7e6 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224262:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac7e6 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224268:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac7e6 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224274:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633cc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739059.224277:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac7e6 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224283:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac7e6 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224289:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78600) 00010000:00010000:0.0:1787739059.224291:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739059.224293:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac7e6 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224298:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac7e6 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739059.224312:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009b44d0e7 reply state 00000000f093fbd4 transno 4294983358 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac7ed lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224323:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac7ed lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.224331:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xb6ebd3c3 (ffffa074419d5600) 00010000:00010000:0.0:1787739059.224333:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739059.224336:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787739059.224434:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157eb0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17530:x1874579652453888:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 54270us (54805us total) trans 4294983358 rc 0/0 00000100:00100000:0.0:1787739059.224442:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23343 00010000:00010000:2.0:1787739059.224692:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739059.224700:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1ac802 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac809 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.224707:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739059.224711:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1ac802 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac809 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739059.224718:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1ac802 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac809 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.224748:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1ac802 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac809 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.224754:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739059.224779:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1ac802 lrc: 1/0,0 mode: --/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac809 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.224787:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab600/0x7ee4453e4a1ac802 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac809 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.224809:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739059.224811:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739059.224813:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739059.225212:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665627776:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739059.225264:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739059.225355:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665627776 00000100:00100000:2.0:1787739059.225362:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665627776 00000100:00100000:2.0:1787739059.225454:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1272 00000100:00100000:2.0:1787739059.225464:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+7:3653:x1874579665627776:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739059.225480:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739059.225499:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ac809 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac802 expref: 7 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.225512:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc0:0x0].0x0 (ffffa074419d5a00) 00010000:00010000:2.0:1787739059.225519:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739059.225554:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+6:3653:x1874579665627776:12345-0@lo:103:ldlm_bl.0 Request processed in 92us (289us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.225561:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1272 00000100:00100000:1.0:1787739059.225701:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665627776:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739059.247250:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739059.247323:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652454272 00000100:00100000:0.0:1787739059.247332:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652454272 00000100:00100000:0.0:1787739059.247393:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23344 00000100:00100000:0.0:1787739059.247431:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157012c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17531:x1874579652454272:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739059.247587:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739059.247612:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac810 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094de8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.247680:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac817 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.247748:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac817 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.247761:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac817 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.247776:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac81e lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.247783:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac81e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.247790:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac81e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.247830:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac817 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.247837:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac817 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.247847:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac817 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.247854:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac817 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.247859:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609be00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739059.247864:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac817 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.247870:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac817 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.247881:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9000) 00010000:00010000:0.0:1787739059.247957:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739059.247962:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac817 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.247968:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ac817 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.247982:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac81e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.247987:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac81e lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.247993:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac81e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.247998:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739059.248001:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac81e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.248006:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac81e lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.248014:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xee50ff8a (ffffa073068a9f00) 00010000:00010000:0.0:1787739059.248016:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739059.248018:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac81e lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.248024:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ac81e lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.248062:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac810 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094de8 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.248071:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9000) 00010000:00010000:0.0:1787739059.248073:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739059.248077:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ac810 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094de8 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.248088:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609ac00, rc 0) 00000100:00100000:0.0:1787739059.248261:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157012c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17531:x1874579652454272:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 850us (1017us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739059.248269:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23344 00000100:00100000:2.0:1787739059.266171:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739059.270544:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652454400 00000100:00100000:3.0:1787739059.270557:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652454400 00000100:00100000:3.0:1787739059.270616:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23345 00000100:00100000:3.0:1787739059.270625:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306006940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17531:x1874579652454400:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739059.270643:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739059.270664:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac825 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094def expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.270729:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac82c lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.270743:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac82c lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.270754:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac82c lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.270767:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac833 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.270775:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac833 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.270782:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac833 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.270836:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac82c lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.270844:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac82c lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.270851:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac82c lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.270858:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac82c lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.270865:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739059.270869:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac82c lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.270876:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac82c lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.270895:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074039f2100) 00010000:00010000:3.0:1787739059.270898:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739059.270901:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac82c lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.270908:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac82c lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.270921:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac833 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.270929:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac833 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.270935:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac833 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.270939:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739059.270942:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac833 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.270948:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac833 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.270954:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xee50ff8a (ffffa074039f2c00) 00010000:00010000:3.0:1787739059.270956:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739059.270959:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac833 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.270964:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac833 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.270991:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac825 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094def expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.271000:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074039f2100) 00010000:00010000:3.0:1787739059.271002:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739059.271005:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac825 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094def expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.271015:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa000, rc 0) 00000100:00100000:3.0:1787739059.271087:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306006940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17531:x1874579652454400:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 463us (4926us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739059.271096:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23345 00000100:00100000:1.0:1787739059.282650:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739059.282805:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652454528 00000100:00100000:3.0:1787739059.282814:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652454528 00000100:00100000:3.0:1787739059.282866:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23346 00000100:00100000:3.0:1787739059.282876:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415702580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17531:x1874579652454528:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739059.282994:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739059.283017:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ac83a lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094df6 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.283084:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac841 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.283098:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac841 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.283109:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac841 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739059.283155:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:3.0:1787739059.283161:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739059.283194:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ac83a lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094df6 expref: 19 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.283211:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac841 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094df6 expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.283219:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074039f2100) 00010000:00010000:3.0:1787739059.283223:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739059.283226:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa000, rc 0) 00000100:00100000:3.0:1787739059.283280:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415702580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17531:x1874579652454528:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 406us (639us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.283288:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23346 00000100:00100000:3.0:1787739059.301139:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739059.301208:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652454784 00000100:00100000:2.0:1787739059.301217:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652454784 00000100:00100000:2.0:1787739059.301273:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23347 00000100:00100000:2.0:1787739059.301282:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a321c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17531:x1874579652454784:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:2.0:1787739059.301392:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004b40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665627904:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739059.301482:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739059.301533:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739059.301601:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665627904 00000100:00100000:1.0:1787739059.301606:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665627904 00000100:00100000:1.0:1787739059.301641:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2935 00000100:00100000:1.0:1787739059.301648:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304261a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665627904:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739059.301714:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304261a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665627904:12345-0@lo:1000:mdt00_004.0 Request processed in 67us (231us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.301721:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2935 00000100:00100000:2.0:1787739059.301794:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004b40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665627904:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739059.301839:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042612c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665628032:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739059.301862:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739059.301900:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739059.301988:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665628032 00000100:00100000:1.0:1787739059.301992:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665628032 00000100:00100000:1.0:1787739059.302020:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2936 00000100:00100000:1.0:1787739059.302026:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304263840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665628032:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739059.302099:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304263840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665628032:12345-0@lo:1000:mdt00_004.0 Request processed in 73us (235us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739059.302106:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2936 00000100:00100000:2.0:1787739059.302163:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042612c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665628032:0@lo:1000:mdt00_004.0 00010000:00010000:2.0:1787739059.302208:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac848 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.302218:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac848 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.302228:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac848 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739059.302238:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042612c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665628160:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1787739059.302260:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739059.302281:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739059.302318:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665628160 00000100:00100000:1.0:1787739059.302322:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665628160 00000100:00100000:1.0:1787739059.302351:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2937 00000100:00100000:1.0:1787739059.302357:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042621c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665628160:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739059.302365:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739059.302381:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac84f lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac848 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.302405:0:17834:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac841 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094df6 expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.302445:0:17834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac84f lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac848 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.302469:0:17834:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac841 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094df6 expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.302482:0:17834:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac841 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094df6 expref: 17 pid: 6523 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739059.302494:0:17834:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415701e00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:17834:1874579665628288:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739059.302542:0:17834:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739059.308345:0:17834:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415701e00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:17834:1874579665628288:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739059.308384:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac84f lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac848 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.309155:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074039f2100) 00010000:00010000:0.0:1787739059.309159:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739059.309165:0:17834:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac84f lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac848 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.309176:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c000, rc 0) 00000100:00100000:0.0:1787739059.309257:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042621c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665628160:12345-0@lo:101:mdt00_004.0 Request processed in 6899us (6994us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739059.309267:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2937 00000100:00100000:2.0:1787739059.309386:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042612c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665628160:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739059.309443:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac848 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac84f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.309455:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac848 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac84f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.311651:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739059.311784:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652454912 00000100:00100000:0.0:1787739059.311793:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652454912 00010000:00010000:0.0:1787739059.311830:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac841 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094df6 expref: 18 pid: 6523 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739059.311859:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1463 00000100:00100000:0.0:1787739059.311868:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074184c1450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2409:x1874579652454912:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739059.311895:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739059.311902:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac841 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094df6 expref: 18 pid: 6523 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.311914:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac841 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094df6 expref: 18 pid: 6523 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.311937:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac841 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094df6 expref: 18 pid: 6523 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.311956:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074039f2100) 00010000:00010000:0.0:1787739059.311963:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac84f lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac848 expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.311973:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac84f lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac848 expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.311983:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac84f lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac848 expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.312016:0:10340:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac84f lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac848 expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739059.312030:0:10340:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f78b40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:10340:1874579665628416:0@lo:105: 00000100:00100000:0.0:1787739059.312088:0:10340:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739059.312122:0:10340:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d840 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.312339:0:6507:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:3.0:1787739059.312352:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665628416 00000100:00100000:3.0:1787739059.312356:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665628416 00000100:00100000:3.0:1787739059.312397:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1119 00000100:00100000:3.0:1787739059.312405:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41a850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:10340:x1874579665628416:12345-0@lo:105: 00010000:00010000:3.0:1787739059.312440:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac848 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac84f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.312450:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac848 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac84f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.312458:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac848 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac84f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.312465:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac848 lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac84f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.312472:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac848 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac84f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.312480:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac848 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ac84f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.312506:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633dc00) 00000100:00100000:3.0:1787739059.312556:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41a850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:10340:x1874579665628416:12345-0@lo:105: Request processed in 152us (466us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.312562:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1119 00000100:00100000:3.0:1787739059.312620:0:10340:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f78b40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:10340:1874579665628416:0@lo:105: 00010000:00010000:3.0:1787739059.312637:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739059.312674:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074184c1450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652454912:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 806us (1032us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.312682:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1463 00010000:00010000:1.0:1787739059.312838:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac848 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac84f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.312849:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac848 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac84f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739059.312869:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739059.312916:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac856 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.312925:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac856 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.312934:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac856 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739059.312949:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665628544:0@lo:101:mdt00_004.0 00000100:00100000:1.0:1787739059.312981:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739059.313006:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e780 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.313168:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665628544 00000100:00100000:3.0:1787739059.313173:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665628544 00000100:00100000:3.0:1787739059.313208:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2938 00000100:00100000:3.0:1787739059.313215:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415701680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665628544:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739059.313228:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739059.313241:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac85d lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac856 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.313260:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac85d lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac856 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.313272:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac85d lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac856 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.313280:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xee50ff8a (ffffa074039f2c00) 00010000:00010000:3.0:1787739059.313283:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739059.313287:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa600, rc 0) 00000100:00100000:3.0:1787739059.313319:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415701680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665628544:12345-0@lo:101:mdt00_004.0 Request processed in 105us (337us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.313325:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2938 00000100:00100000:3.0:1787739059.313350:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665628544:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739059.313361:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac856 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac85d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.313369:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac856 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac85d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739059.313380:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:3.0:1787739059.313410:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a30000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665628672:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739059.313452:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.313468:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404300 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.313480:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665628672 00000100:00100000:3.0:1787739059.313484:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665628672 00000100:00100000:3.0:1787739059.313505:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2939 00000100:00100000:3.0:1787739059.313510:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a312c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665628672:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739059.313559:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a312c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665628672:12345-0@lo:1000:mdt00_004.0 Request processed in 49us (106us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.313565:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2939 00000100:00100000:3.0:1787739059.313586:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a30000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665628672:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739059.313617:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a30000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665628800:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739059.313636:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.313649:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074104046c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.313659:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665628800 00000100:00100000:3.0:1787739059.313662:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665628800 00000100:00100000:3.0:1787739059.313678:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2940 00000100:00100000:3.0:1787739059.313683:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665628800:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739059.313737:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665628800:12345-0@lo:1000:mdt00_004.0 Request processed in 53us (99us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739059.313743:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2940 00000100:00100000:3.0:1787739059.313764:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a30000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665628800:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739059.313820:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a30000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665628928:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739059.313841:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.313854:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404300 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.313863:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665628928 00000100:00100000:3.0:1787739059.313866:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665628928 00000100:00100000:3.0:1787739059.314038:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2941 00000100:00100000:3.0:1787739059.314056:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665628928:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739059.314111:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665628928:12345-0@lo:1000:mdt00_004.0 Request processed in 58us (269us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739059.314151:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a30000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665628928:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739059.314193:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7b0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665629056:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739059.314217:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739059.314225:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d840 going to sleep for 11 seconds 00000100:00100000:1.0:1787739059.317173:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665629056 00000100:00100000:1.0:1787739059.317188:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665629056 00000100:00100000:1.0:1787739059.317225:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2942 00000100:00100000:1.0:1787739059.317234:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665629056:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739059.317354:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665629056:12345-0@lo:1000:mdt00_004.0 Request processed in 121us (3135us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.317361:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2942 00000100:00100000:1.0:1787739059.317509:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7b0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665629056:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739059.317578:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415701e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665629184:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739059.317610:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739059.317629:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ef00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739059.317641:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665629184 00000100:00100000:1.0:1787739059.317644:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665629184 00000100:00100000:1.0:1787739059.317660:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2943 00000100:00100000:1.0:1787739059.317665:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415700b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665629184:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739059.317725:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415700b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665629184:12345-0@lo:1000:mdt00_004.0 Request processed in 60us (115us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.317732:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2943 00000100:00100000:1.0:1787739059.317757:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415701e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665629184:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739059.317795:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415701e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665629312:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739059.317819:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739059.317833:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ef00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739059.317844:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665629312 00000100:00100000:1.0:1787739059.317848:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665629312 00000100:00100000:1.0:1787739059.317863:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2944 00000100:00100000:1.0:1787739059.317869:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415701a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665629312:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739059.317987:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2941 00000100:00100000:1.0:1787739059.318258:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415701a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665629312:12345-0@lo:1000:mdt00_004.0 Request processed in 388us (437us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.318266:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2944 00000100:00100000:0.0:1787739059.320220:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415701e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665629312:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739059.320286:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7b480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665629440:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739059.320343:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739059.320443:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739059.324131:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665629440 00000100:00100000:2.0:1787739059.324144:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665629440 00000100:00100000:2.0:1787739059.324199:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2945 00000100:00100000:2.0:1787739059.324208:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665629440:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739059.324321:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665629440:12345-0@lo:1000:mdt00_004.0 Request processed in 114us (3977us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.324329:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2945 00000100:00100000:2.0:1787739059.324381:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7b480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665629440:0@lo:1000:mdt00_004.0 00000040:00080000:2.0:1787739059.325817:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].348, 2168 off529288 00000040:00080000:2.0:1787739059.327001:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].348, 2368 off578384 00000004:00080000:2.0:1787739059.327011:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0743c413c00 with version 661 00000004:00080000:3.0:1787739059.333617:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 661 rpc_version 661 00000020:00100000:3.0:1787739059.333667:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739059.333678:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303a33480 x1874579665629696/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739059.333702:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665629696:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739059.333776:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.333823:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739059.334504:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665629696 00000100:00100000:1.0:1787739059.334513:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665629696 00000100:00100000:1.0:1787739059.334558:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2946 00000100:00100000:1.0:1787739059.334566:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665629696:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739059.335298:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665629696:12345-0@lo:1000:osp_up0-1.0 Request processed in 729us (1519us total) trans 4294983359 rc 0/0 00000100:00100000:1.0:1787739059.335308:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2946 00000100:00100000:3.0:1787739059.335528:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665629696:0@lo:1000:osp_up0-1.0 00010000:00010000:2.0:1787739059.336687:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac864 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x16e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.336706:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac864 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x16e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.336717:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac864 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x16e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.336739:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ac86b lrc: 3/0,1 mode: --/PW res: [0x240000400:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.336746:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ac86b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.336753:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ac86b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.336787:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d200/0x7ee4453e4a1ac872 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.336794:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d200/0x7ee4453e4a1ac872 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.336804:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d200/0x7ee4453e4a1ac872 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739059.336828:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304260b40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665629824:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1787739059.336876:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739059.337151:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c4139c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.339637:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665629824 00000100:00100000:3.0:1787739059.339649:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665629824 00000100:00100000:3.0:1787739059.339707:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2947 00000100:00100000:3.0:1787739059.339717:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304260f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665629824:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739059.339742:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739059.339765:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac879 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac872 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.339794:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac879 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac872 expref: 11 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.339808:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac879 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac872 expref: 11 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.339816:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbe:0x0].0x0 (ffffa074039f2e00) 00010000:00010000:3.0:1787739059.339819:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739059.339823:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abc00, rc 0) 00000100:00100000:2.0:1787739059.340009:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304260b40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665629824:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739059.340028:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d200/0x7ee4453e4a1ac872 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac879 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.340039:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d200/0x7ee4453e4a1ac872 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac879 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739059.340075:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007e5e9aa7 reply state 000000005afbc87c transno 4294968350 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ac86b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.340087:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ac86b lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.340093:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xbe:0x0].0x0 (ffffa07303c78000) 00010000:00010000:2.0:1787739059.340096:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739059.340099:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787739059.340105:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d200/0x7ee4453e4a1ac872 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac879 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.340113:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d200/0x7ee4453e4a1ac872 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac879 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739059.340125:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xbe:0x0] 00000004:00010000:2.0:1787739059.340132:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007e5e9aa7 reply state 000000005afbc87c transno 4294968350 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac864 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x16e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.340146:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac864 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x16e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.340152:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16e:0x0].0x0 (ffffa07303c78a00) 00010000:00010000:2.0:1787739059.340154:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739059.340156:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:2.0:1787739059.340161:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x16e:0x0]: returning size 4096 00000004:00080000:2.0:1787739059.340170:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739059.340180:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac848 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac84f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.340187:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac848 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac84f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.340193:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac848 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac84f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.340212:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739059.340217:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739059.340221:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac856 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac85d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.340228:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac856 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac85d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739059.341900:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a321c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17531:x1874579652454784:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 40611us (40761us total) trans 4294968350 rc 0/0 00000100:00100000:2.0:1787739059.341912:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23347 00000100:00100000:3.0:1787739059.343072:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304260f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:13793:x1874579665629824:12345-0@lo:101:mdt00_004.0 Request processed in 3350us (6189us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.343083:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2947 00010000:00010000:3.0:1787739059.343118:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739059.343126:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac848 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac84f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.343137:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633dc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739059.343144:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac848 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac84f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739059.343154:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac848 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac84f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.343197:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac848 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac84f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.343205:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739059.343241:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac848 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac84f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.343250:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac848 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac84f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.343267:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739059.343269:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739059.343271:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739059.345007:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a30000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665629952:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739059.345088:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739059.345183:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665629952 00000100:00100000:1.0:1787739059.345190:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665629952 00000100:00100000:1.0:1787739059.345246:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1273 00000100:00100000:1.0:1787739059.345255:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304260b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3650:x1874579665629952:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739059.345269:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739059.345291:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac84f lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac848 expref: 11 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.345306:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074039f2100) 00010000:00010000:1.0:1787739059.345314:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739059.345352:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304260b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+10:3650:x1874579665629952:12345-0@lo:103:ldlm_bl.0 Request processed in 98us (264us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.345359:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1273 00000100:00100000:2.0:1787739059.346570:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a30000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665629952:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739059.390200:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739059.390302:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652455168 00000100:00100000:1.0:1787739059.390311:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652455168 00000100:00100000:1.0:1787739059.390373:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23348 00000100:00100000:1.0:1787739059.390383:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17532:x1874579652455168:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:1.0:1787739059.390579:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17532:x1874579652455168:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 197us (388us total) trans 0 rc -17/-17 00000100:00100000:1.0:1787739059.390586:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23348 00000100:00100000:0.0:1787739059.427166:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739059.427362:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652455296 00000100:00100000:2.0:1787739059.427371:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652455296 00000100:00100000:2.0:1787739059.427464:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23349 00000100:00100000:2.0:1787739059.427475:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f78b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17533:x1874579652455296:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739059.427495:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739059.427520:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac880 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094dfd expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.427587:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac887 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.427601:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac887 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.427613:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac887 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.427625:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac88e lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.427635:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac85d lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac856 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.427644:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac88e lrc: 4/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.427675:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac85d lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ac856 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739059.427896:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304262580 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665630080:0@lo:104: 00000100:00100000:2.0:1787739059.427948:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739059.428001:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c4130c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739059.428117:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665630080 00000100:00100000:0.0:1787739059.428122:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665630080 00000100:00100000:0.0:1787739059.428165:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1120 00000100:00100000:0.0:1787739059.428173:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6523:x1874579665630080:12345-0@lo:104: 00010000:00010000:0.0:1787739059.428186:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac856 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac85d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.428241:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787739059.428247:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6523:x1874579665630080:12345-0@lo:104: Request processed in 76us (299us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739059.428255:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1120 00000100:00100000:0.0:1787739059.428293:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304262580 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665630080:0@lo:104: 00010000:00010000:0.0:1787739059.428311:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac88e lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.428327:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739059.428333:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac856 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac85d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.428341:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1de00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739059.428346:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac856 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ac85d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739059.428356:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac856 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ac85d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.428384:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac856 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac85d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.428392:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739059.428443:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac856 lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac85d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.428450:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ac856 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac85d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.428468:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739059.428470:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739059.428471:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739059.428482:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665630208:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739059.428513:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739059.428547:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665630208 00000100:00100000:0.0:1787739059.428550:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665630208 00010000:00010000:0.0:1787739059.428567:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac85d lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ac856 expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739059.428589:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1464 00000100:00100000:0.0:1787739059.428595:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+10:3651:x1874579665630208:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739059.428604:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:0.0:1787739059.431772:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983359 is committed 00000001:00080000:0.0:1787739059.431794:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983358 is committed 00000020:00080000:0.0:1787739059.431802:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 362 batchid updated 00000020:00080000:0.0:1787739059.431815:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa073068ba3c0 refcount 2 committed 0 result 0 batchid 363 00000020:00080000:0.0:1787739059.431820:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730415f400 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa073068bae40 00000020:00080000:0.0:1787739059.431823:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].360 00000020:00080000:0.0:1787739059.431826:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730415fc00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306940800 00000020:00080000:0.0:1787739059.431829:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].360 00010000:00010000:0.0:1787739059.432072:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ac85d lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ac856 expref: 7 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.432089:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xee50ff8a (ffffa074039f2c00) 00010000:00010000:0.0:1787739059.432094:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac88e lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.432103:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac88e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.432110:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac88e lrc: 4/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.432137:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739059.432209:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+6:3651:x1874579665630208:12345-0@lo:103:ldlm_bl.0 Request processed in 3612us (3693us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739059.432219:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1464 00010000:00010000:0.0:1787739059.432254:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac88e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.432261:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac88e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739059.432369:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665630336:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739059.432508:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739059.432532:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739059.432569:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665630208:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739059.432663:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac78b lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.432672:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac78b lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.432679:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac78b lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.432797:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739059.432803:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac78b lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.432812:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac78b lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.432823:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xb6ebd3c3 (ffffa074419d5600) 00010000:00010000:1.0:1787739059.432825:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739059.432829:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac78b lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.432834:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac78b lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.432845:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac792 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x150b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.432852:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac792 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x150b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.432857:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac792 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x150b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.432862:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739059.432864:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac792 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x150b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.432870:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac792 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x150b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.432876:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x150b:0x0].0x0 (ffffa074419d5e00) 00010000:00010000:1.0:1787739059.432878:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739059.432955:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac792 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x150b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.432963:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac792 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x150b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.432980:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac799 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.432986:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac799 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.432992:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac799 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.432997:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739059.433000:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac799 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433006:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac799 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433013:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc0:0x0].0x0 (ffffa074419d5700) 00010000:00010000:1.0:1787739059.433015:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739059.433018:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac799 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433024:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac799 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433038:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac7ed lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433056:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac7ed lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433061:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac7ed lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433066:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739059.433069:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac7ed lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433075:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac7ed lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433082:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xb6ebd3c3 (ffffa074419d5600) 00010000:00010000:1.0:1787739059.433084:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739059.433088:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac7ed lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433093:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ac7ed lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xb6ebd3c3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739059.433118:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665630336 00000100:00100000:1.0:1787739059.433125:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665630336 00000100:00100000:1.0:1787739059.433177:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2948 00000100:00100000:1.0:1787739059.433186:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6523:x1874579665630336:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739059.433279:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6523:x1874579665630336:12345-0@lo:1000:mdt00_000.0 Request processed in 93us (770us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.433286:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2948 00000100:00100000:1.0:1787739059.433326:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665630336:0@lo:1000:mdt00_000.0 00010000:00010000:1.0:1787739059.433370:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac895 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x16e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433381:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac895 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433390:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac895 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739059.433559:0:6523:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac895 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433574:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac887 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433581:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac887 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433590:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac887 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433596:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac887 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433601:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739059.433604:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac887 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433609:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac887 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433617:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78500) 00010000:00010000:1.0:1787739059.433619:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739059.433622:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac887 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433627:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac887 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433637:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac88e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433642:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac88e lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433647:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac88e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433651:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633cc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739059.433654:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac88e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433659:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac88e lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433665:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xee50ff8a (ffffa074039f2c00) 00010000:00010000:1.0:1787739059.433668:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739059.433670:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac88e lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433676:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ac88e lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433876:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac880 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094dfd expref: 15 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433962:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac895 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x16e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094dfd expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.433971:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16e:0x0].0x0 (ffffa07305eae800) 00010000:00010000:1.0:1787739059.433973:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739059.433975:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c000, rc 0) 00000100:00100000:1.0:1787739059.434068:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f78b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17533:x1874579652455296:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 6595us (6912us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.434075:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23349 00000020:00080000:2.0:1787739059.434637:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 363 committed batchid 362 00080000:02000000:2.0:1787739059.434656:0:6555:0:(osd_handler.c:2218:osd_trans_start()) *** cfs_fail_loc=19a, val=0*** 00000004:00080000:2.0:1787739059.435461:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0743c413000 with version 689 00000020:00080000:2.0:1787739059.435474:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 361 cancel update log [0x240000401:0x3:0x0].359: rc = 0 00000020:00080000:2.0:1787739059.435540:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 361 cancel update log [0x200000400:0x3:0x0].359: rc = 0 00000020:00080000:2.0:1787739059.435553:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 363 committed batchid 362 00000004:00080000:1.0:1787739059.435611:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 689 rpc_version 689 00000020:00100000:1.0:1787739059.435641:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739059.435650:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07415701e00 x1874579665630592/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739059.435677:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415701e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665630592:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739059.435749:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739059.435782:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ed80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739059.435991:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665630592 00000100:00100000:2.0:1787739059.435998:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665630592 00000100:00100000:2.0:1787739059.436062:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2949 00000100:00100000:2.0:1787739059.436072:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665630592:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739059.436202:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665630592:12345-0@lo:1000:osp_up1-0.0 Request processed in 131us (453us total) trans 4294968351 rc 0/0 00000100:00100000:2.0:1787739059.436210:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2949 00000100:00100000:2.0:1787739059.436256:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415701e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665630592:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739059.449343:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739059.449447:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652455424 00000100:00100000:2.0:1787739059.449457:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652455424 00000100:00100000:2.0:1787739059.449509:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23350 00000100:00100000:2.0:1787739059.449519:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17533:x1874579652455424:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739059.449543:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739059.449563:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac89c lrc: 2/0,0 mode: --/CR res: [0x240000402:0x16e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094e04 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.449630:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac8a3 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x16e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.449643:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac8a3 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.449654:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac8a3 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739059.449975:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x16e:0x0]: returning size 4096 00000004:00200000:2.0:1787739059.449981:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739059.450019:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac89c lrc: 0/0,0 mode: --/CR res: [0x240000402:0x16e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e04 expref: 11 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.450038:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac8a3 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x16e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e04 expref: 10 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.450059:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16e:0x0].0x0 (ffffa07303c78a00) 00010000:00010000:2.0:1787739059.450062:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739059.450066:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d400, rc 0) 00000100:00100000:2.0:1787739059.450126:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17533:x1874579652455424:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 609us (790us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.450134:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23350 00000100:00100000:3.0:1787739059.456826:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739059.457036:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652455552 00000100:00100000:0.0:1787739059.457055:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652455552 00000100:00100000:0.0:1787739059.457115:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23351 00000100:00100000:0.0:1787739059.457125:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157e92c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17533:x1874579652455552:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739059.457145:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739059.457165:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac8aa lrc: 2/0,0 mode: --/CR res: [0x240000400:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094e0b expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.457221:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac8b1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.457234:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac8b1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.457245:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac8b1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739059.457292:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xbe:0x0]: returning size 4096 00000004:00200000:0.0:1787739059.457297:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739059.457329:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac8aa lrc: 0/0,0 mode: --/CR res: [0x240000400:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e0b expref: 12 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.457344:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac8b1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e0b expref: 11 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.457352:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xbe:0x0].0x0 (ffffa07303c78000) 00010000:00010000:0.0:1787739059.457355:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739059.457358:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b800, rc 0) 00000100:00100000:0.0:1787739059.457444:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157e92c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17533:x1874579652455552:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 321us (627us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739059.457451:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23351 00000100:00100000:0.0:1787739059.473304:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739059.473387:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652455680 00000100:00100000:2.0:1787739059.473406:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652455680 00000100:00100000:2.0:1787739059.473484:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23352 00000100:00100000:2.0:1787739059.473494:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17533:x1874579652455680:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739059.473517:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739059.473536:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac8b8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094e12 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.473597:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac8bf lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.473612:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac879 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac872 expref: 5 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.473621:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac8bf lrc: 4/1,0 mode: --/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.473650:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac879 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ac872 expref: 5 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739059.473665:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042630c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665630720:0@lo:104: 00000100:00100000:2.0:1787739059.473723:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739059.473757:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413840 going to sleep for 11 seconds 00000100:00100000:0.0:1787739059.473900:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665630720 00000100:00100000:0.0:1787739059.473904:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665630720 00000100:00100000:0.0:1787739059.473936:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1121 00000100:00100000:0.0:1787739059.473942:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6523:x1874579665630720:12345-0@lo:104: 00010000:00010000:0.0:1787739059.473953:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d200/0x7ee4453e4a1ac872 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac879 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.473998:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787739059.474003:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6523:x1874579665630720:12345-0@lo:104: Request processed in 62us (279us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739059.474010:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1121 00010000:00010000:3.0:1787739059.474182:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739059.474189:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d200/0x7ee4453e4a1ac872 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac879 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.474198:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739059.474203:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d200/0x7ee4453e4a1ac872 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ac879 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739059.474212:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d200/0x7ee4453e4a1ac872 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ac879 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.474238:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d200/0x7ee4453e4a1ac872 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac879 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.474249:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739059.474278:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d200/0x7ee4453e4a1ac872 lrc: 1/0,0 mode: --/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac879 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.474285:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d200/0x7ee4453e4a1ac872 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac879 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.474303:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739059.474305:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739059.474306:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739059.474315:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787739059.474324:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c4152c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665630848:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739059.474358:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.474397:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665630848 00000100:00100000:3.0:1787739059.474401:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665630848 00010000:00010000:3.0:1787739059.474440:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac879 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ac872 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.474463:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1465 00000100:00100000:3.0:1787739059.474470:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fcf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+6:3653:x1874579665630848:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739059.474480:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1787739059.474806:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042630c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665630720:0@lo:104: 00010000:00010000:2.0:1787739059.474836:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac8bf lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:0.0:1787739059.479274:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 363 batchid updated 00000020:00080000:2.0:1787739059.479319:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 363 00000004:00080000:2.0:1787739059.479382:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0743c413840 with version 690 00000020:00080000:2.0:1787739059.480557:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 363 cancel update log [0x240000401:0x3:0x0].360: rc = 0 00000020:00080000:2.0:1787739059.480659:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 363 cancel update log [0x200000400:0x3:0x0].360: rc = 0 00000020:00080000:2.0:1787739059.480670:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 363 00010000:00010000:2.0:1787739059.482094:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac879 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ac872 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.482117:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbe:0x0].0x0 (ffffa074039f2e00) 00010000:00010000:2.0:1787739059.482122:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac8bf lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.482132:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac8bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.482140:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac8bf lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.482177:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739059.482257:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fcf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+5:3653:x1874579665630848:12345-0@lo:103:ldlm_bl.0 Request processed in 7786us (7896us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.482268:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1465 00010000:00010000:0.0:1787739059.482342:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac8bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.482352:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac8bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739059.482442:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xbe:0x0]: returning size 4096 00000004:00200000:0.0:1787739059.482447:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739059.482491:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ac8b8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e12 expref: 16 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.482507:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac8bf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e12 expref: 15 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.482519:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbe:0x0].0x0 (ffffa074039f2e00) 00010000:00010000:0.0:1787739059.482521:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739059.482523:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633ce00, rc 0) 00000100:00100000:0.0:1787739059.482568:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17533:x1874579652455680:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 9077us (9275us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739059.482575:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23352 00000100:00100000:2.0:1787739059.484568:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c4152c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665630848:0@lo:103:ldlm_bl.0 00000004:00080000:3.0:1787739059.485776:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 690 rpc_version 690 00000020:00100000:3.0:1787739059.485843:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739059.485852:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073061fc3c0 x1874579665631104/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739059.485876:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fc3c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665631104:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739059.485950:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.485989:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739059.486998:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665631104 00000100:00100000:0.0:1787739059.487006:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665631104 00000100:00100000:0.0:1787739059.487071:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2950 00000100:00100000:0.0:1787739059.487082:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fcb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665631104:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739059.487256:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fcb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665631104:12345-0@lo:1000:osp_up1-0.0 Request processed in 176us (1305us total) trans 4294968352 rc 0/0 00000100:00100000:0.0:1787739059.487264:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2950 00000100:00100000:3.0:1787739059.487874:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fc3c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665631104:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739059.488098:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739059.488142:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739059.488167:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652456064 00000100:00100000:1.0:1787739059.488172:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652456064 00000100:00100000:1.0:1787739059.488211:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23353 00000100:00100000:1.0:1787739059.488219:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17533:x1874579652456064:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:1.0:1787739059.488263:0:13793:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739059.488289:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ac8a3 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x16e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e04 expref: 11 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.488306:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16e:0x0].0x0 (ffffa07303c78a00) 00010000:00010000:1.0:1787739059.488311:0:13793:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739059.488352:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac8c6 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.488361:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac8c6 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.488370:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac8c6 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739059.488389:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252b0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665631232:0@lo:101:mdt00_004.0 00000100:00100000:1.0:1787739059.488466:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739059.488488:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ee40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739059.488503:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665631232 00000100:00100000:1.0:1787739059.488507:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665631232 00000100:00100000:1.0:1787739059.488536:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2951 00000100:00100000:1.0:1787739059.488542:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+5:13793:x1874579665631232:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739059.488550:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739059.488565:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac8cd lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac8c6 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.488586:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac8cd lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac8c6 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.488599:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac8cd lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac8c6 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.488606:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eaec00) 00010000:00010000:1.0:1787739059.488609:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739059.488611:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1da00, rc 0) 00000100:00100000:1.0:1787739059.488644:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:13793:x1874579665631232:12345-0@lo:101:mdt00_004.0 Request processed in 102us (177us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.488651:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2951 00000100:00100000:1.0:1787739059.488684:0:13793:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983359 gen 2 00000004:00080000:1.0:1787739059.488800:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983355 committed transno 4294983359 00000020:00080000:1.0:1787739059.488806:0:13793:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa0730610eb40 refcount 2 committed 0 result 0 batchid 354 00000020:00080000:1.0:1787739059.488809:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2e00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730610e480 00000020:00080000:1.0:1787739059.488812:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].347 00000020:00080000:1.0:1787739059.488815:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2c00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073040e8800 00000020:00080000:1.0:1787739059.488817:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].347 00000004:00080000:1.0:1787739059.488838:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983357 committed transno 4294983359 00000004:00080000:1.0:1787739059.488847:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983359 committed transno 4294983359 00000100:00100000:1.0:1787739059.488866:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252b0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665631232:0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739059.488880:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac8c6 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac8cd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.488962:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac8c6 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac8cd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739059.488977:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739059.488998:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ac8d4 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.489005:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ac8d4 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.489012:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ac8d4 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739059.489024:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252b0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665631360:0@lo:101:mdt00_004.0 00000100:00100000:1.0:1787739059.489061:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739059.489078:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ee40 going to sleep for 11 seconds 00000020:00080000:1.0:1787739059.489090:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 354 committed batchid 350 00000020:00080000:1.0:1787739059.489145:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 350 00000100:00100000:1.0:1787739059.489158:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665631360 00000100:00100000:1.0:1787739059.489162:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665631360 00000100:00100000:1.0:1787739059.489189:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2952 00000100:00100000:1.0:1787739059.489195:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:13793:x1874579665631360:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739059.489204:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739059.489218:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac8db lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac8d4 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.489237:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac8db lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac8d4 expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.489247:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac8db lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac8d4 expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.489254:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xee50ff8a (ffffa07305eae100) 00010000:00010000:1.0:1787739059.489257:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739059.489259:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1ce00, rc 0) 00000100:00100000:1.0:1787739059.489297:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665631360:12345-0@lo:101:mdt00_004.0 Request processed in 102us (236us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.489305:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2952 00000100:00100000:2.0:1787739059.489447:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252b0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665631360:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739059.489462:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ac8d4 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac8db expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.489473:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ac8d4 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac8db expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739059.489493:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:2.0:1787739059.489528:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042630c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665631488:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739059.489560:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739059.489582:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413600 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.489852:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652455808 00000100:00100000:3.0:1787739059.489858:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652455808 00000100:00100000:3.0:1787739059.489907:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1274 00000100:00100000:3.0:1787739059.489916:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2408:x1874579652455808:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:3.0:1787739059.489927:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739059.489942:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ac895 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x16e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094dfd expref: 15 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.489954:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16e:0x0].0x0 (ffffa07305eae800) 00010000:00010000:3.0:1787739059.489962:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739059.489999:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2408:x1874579652455808:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 85us (1903us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.490006:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1274 00000100:00100000:1.0:1787739059.491062:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665631488 00000100:00100000:1.0:1787739059.491070:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665631488 00000100:00100000:1.0:1787739059.491118:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2953 00000100:00100000:1.0:1787739059.491127:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042612c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665631488:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739059.491272:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042612c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665631488:12345-0@lo:1000:mdt00_004.0 Request processed in 145us (1711us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.491280:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2953 00000100:00100000:3.0:1787739059.495600:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042630c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665631488:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739059.495679:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fe940 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665631616:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739059.495860:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.496012:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068bae40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739059.497152:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665631616 00000100:00100000:0.0:1787739059.497165:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665631616 00000100:00100000:0.0:1787739059.497220:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2954 00000100:00100000:0.0:1787739059.497228:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665631616:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739059.497351:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665631616:12345-0@lo:1000:mdt00_004.0 Request processed in 123us (1491us total) trans 0 rc 1/1 00000100:00100000:0.0:1787739059.497358:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2954 00000100:00100000:2.0:1787739059.497503:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fe940 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665631616:0@lo:1000:mdt00_004.0 00010000:00010000:2.0:1787739059.497567:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac8e2 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.497577:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac8e2 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.497587:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac8e2 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739059.497602:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042630c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665631744:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1787739059.497636:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739059.497669:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c4139c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.497822:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665631744 00000100:00100000:3.0:1787739059.497827:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665631744 00000100:00100000:3.0:1787739059.497863:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2955 00000100:00100000:3.0:1787739059.497870:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c4152c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665631744:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739059.497890:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739059.497908:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac8e9 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac8e2 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.497931:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac8e9 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac8e2 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.497944:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac8e9 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac8e2 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.497952:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16e:0x0].0x0 (ffffa074039f2500) 00010000:00010000:3.0:1787739059.497955:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739059.497959:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aae00, rc 0) 00000100:00100000:3.0:1787739059.498003:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c4152c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665631744:12345-0@lo:101:mdt00_004.0 Request processed in 134us (366us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.498009:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2955 00000100:00100000:0.0:1787739059.498252:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042630c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665631744:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739059.498266:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac8e2 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac8e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.498276:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac8e2 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac8e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.498301:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac8f0 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.498309:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac8f0 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.498317:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac8f0 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.498333:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac8f7 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.498346:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac8b1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e0b expref: 10 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.498354:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac8f7 lrc: 4/0,1 mode: --/EX res: [0x240000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.498377:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac8b1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e0b expref: 10 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.498391:0:13793:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac8b1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e0b expref: 10 pid: 7944 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739059.498433:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561fc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665631872:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739059.498487:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074104046c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739059.506600:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739059.506678:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652456192 00000100:00100000:3.0:1787739059.506696:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652456192 00010000:00010000:3.0:1787739059.506734:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac8b1 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e0b expref: 11 pid: 7944 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.506775:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1466 00000100:00100000:3.0:1787739059.506784:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2406:x1874579652456192:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739059.506800:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739059.506806:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac8b1 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e0b expref: 11 pid: 7944 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.506818:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac8b1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e0b expref: 11 pid: 7944 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.506843:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xbe:0x0].0x0 (ffffa07303c78000) 00010000:00010000:3.0:1787739059.506848:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac8f7 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.506857:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac8f7 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.506864:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac8f7 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.507079:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739059.507140:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2406:x1874579652456192:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 357us (550us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.507148:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1466 00010000:00010000:2.0:1787739059.507621:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac8b1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e0b expref: 10 pid: 7944 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739059.507639:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561fc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665631872:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739059.507660:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac8f7 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.507671:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac8f7 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.507677:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac8f7 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.507732:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ac8fe lrc: 3/0,1 mode: --/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.507740:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ac8fe lrc: 3/0,1 mode: --/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.507750:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ac8fe lrc: 3/0,1 mode: --/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739059.507769:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306007480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665632000:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1787739059.507815:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739059.507850:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413600 going to sleep for 11 seconds 00000100:00100000:1.0:1787739059.508007:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665632000 00000100:00100000:1.0:1787739059.508013:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665632000 00000100:00100000:1.0:1787739059.508060:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2956 00000100:00100000:1.0:1787739059.508068:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665632000:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739059.508082:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739059.508097:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac905 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac8fe expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.508121:0:17834:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac8bf lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e12 expref: 13 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.508130:0:17834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac905 lrc: 4/0,0 mode: --/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac8fe expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.508150:0:17834:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac8bf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e12 expref: 13 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.508163:0:17834:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac8bf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e12 expref: 13 pid: 6523 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739059.508187:0:17834:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252b0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:17834:1874579665632128:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739059.508242:0:17834:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e840 going to sleep for 11 seconds 00000100:00100000:1.0:1787739059.511341:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739059.511586:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652456320 00000100:00100000:3.0:1787739059.511596:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652456320 00010000:00010000:3.0:1787739059.511635:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac8bf lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e12 expref: 14 pid: 6523 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.511678:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1467 00000100:00100000:3.0:1787739059.511698:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652456320:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739059.511713:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739059.511721:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac8bf lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e12 expref: 14 pid: 6523 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.511734:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac8bf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e12 expref: 14 pid: 6523 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.511762:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbe:0x0].0x0 (ffffa074039f2e00) 00010000:00010000:3.0:1787739059.511768:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac905 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac8fe expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.511779:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac905 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac8fe expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.511787:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac905 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac8fe expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.511830:0:10340:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac905 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac8fe expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.511845:0:10340:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fe940 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:10340:1874579665632256:0@lo:105: 00010000:00010000:2.0:1787739059.511851:0:17834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac8bf lrc: 0/0,0 mode: --/PR res: [0x200000401:0xbe:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e12 expref: 14 pid: 6523 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739059.511871:0:17834:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252b0c0 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:17834:1874579665632128:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739059.511894:0:10340:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787739059.511911:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac905 lrc: 5/0,0 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac8fe expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.511920:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbe:0x0].0x0 (ffffa074039f2e00) 00010000:00010000:2.0:1787739059.511924:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739059.511928:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1ca00, rc 0) 00000100:00100000:3.0:1787739059.511939:0:10340:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739059.511988:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665632000:12345-0@lo:101:mdt00_004.0 Request processed in 3920us (4171us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.511995:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2956 00000100:00100000:2.0:1787739059.512025:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665632256 00000100:00100000:2.0:1787739059.512030:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665632256 00000100:00100000:2.0:1787739059.512071:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1122 00000100:00100000:2.0:1787739059.512078:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f73c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:10340:x1874579665632256:12345-0@lo:105: 00010000:00010000:2.0:1787739059.512088:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ac8fe lrc: 4/0,1 mode: --/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac905 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.512098:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ac8fe lrc: 4/0,1 mode: --/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac905 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.512107:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ac8fe lrc: 4/0,1 mode: --/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac905 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.512114:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ac8fe lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac905 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.512120:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ac8fe lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac905 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.512128:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ac8fe lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ac905 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.512139:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633da00) 00000100:00100000:2.0:1787739059.512173:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f73c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:10340:x1874579665632256:12345-0@lo:105: Request processed in 95us (278us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.512179:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1122 00000100:00100000:2.0:1787739059.512209:0:10340:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fe940 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:10340:1874579665632256:0@lo:105: 00010000:00010000:2.0:1787739059.512220:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739059.512256:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:2409:x1874579652456320:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 569us (924us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.512262:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1467 00000100:00100000:3.0:1787739059.512390:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306007480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665632000:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739059.512406:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ac8fe lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac905 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739059.513510:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306007480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665632384:0@lo:403:mdt00_004.0 00000100:00100000:2.0:1787739059.513571:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739059.513599:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413780 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.514809:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665632384 00000100:00100000:3.0:1787739059.514829:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665632384 00000100:00100000:3.0:1787739059.514897:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2957 00000100:00100000:3.0:1787739059.514906:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fe940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665632384:12345-0@lo:403:mdt00_004.0 00000100:00100000:3.0:1787739059.516643:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fe940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665632384:12345-0@lo:403:mdt00_004.0 Request processed in 1732us (3065us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.516659:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2957 00000100:00100000:2.0:1787739059.516811:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306007480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665632384:0@lo:403:mdt00_004.0 00000040:00080000:2.0:1787739059.517840:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].349, 952 off530240 00000040:00080000:2.0:1787739059.518013:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].349, 1032 off579416 00000004:00080000:2.0:1787739059.518019:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0743c413780 with version 662 00000004:00080000:3.0:1787739059.520343:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 662 rpc_version 662 00000020:00100000:3.0:1787739059.520463:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739059.520475:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303f71680 x1874579665632640/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739059.520562:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f71680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665632640:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739059.520630:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.520684:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068bae40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739059.520777:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665632640 00000100:00100000:2.0:1787739059.520785:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665632640 00000100:00100000:2.0:1787739059.520841:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2958 00000100:00100000:2.0:1787739059.520850:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c381680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665632640:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739059.526474:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f71680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665632640:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739059.526507:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c381680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665632640:12345-0@lo:1000:osp_up0-1.0 Request processed in 5540us (5758us total) trans 4294983360 rc 0/0 00000100:00100000:2.0:1787739059.526520:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2958 00000004:00200000:2.0:1787739059.526564:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x16e:0x0]: returning size 4096 00010000:00010000:2.0:1787739059.526583:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac8f7 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.526593:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac8f7 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.526601:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac8f7 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.526608:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739059.526612:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac8f7 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.526620:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac8f7 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.526634:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xbe:0x0].0x0 (ffffa07303c78000) 00010000:00010000:2.0:1787739059.526637:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739059.526641:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac8f7 lrc: 2/0,0 mode: --/EX res: [0x240000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.526647:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac8f7 lrc: 0/0,0 mode: --/EX res: [0x240000400:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.526660:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ac8fe lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac905 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.526668:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ac8fe lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac905 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.526675:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ac8fe lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac905 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.526701:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739059.526710:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xbe:0x0] 00010000:00010000:2.0:1787739059.526719:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac8f0 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.526726:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac8f0 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.526731:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac8f0 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.526737:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739059.526740:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac8f0 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.526746:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac8f0 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.526757:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16e:0x0].0x0 (ffffa07303c78a00) 00010000:00010000:2.0:1787739059.526759:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739059.526761:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac8f0 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.526767:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac8f0 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.526776:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac8e2 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac8e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.526784:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac8e2 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac8e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.526791:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac8e2 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac8e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.526803:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:2.0:1787739059.526812:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739059.526816:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac8c6 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac8cd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.526822:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac8c6 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac8cd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.526828:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac8c6 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac8cd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.526847:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:2.0:1787739059.526852:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739059.526857:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ac8d4 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac8db expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.526864:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ac8d4 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac8db expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739059.526931:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17533:x1874579652456064:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 38711us (38786us total) trans 4294968353 rc 0/0 00000100:00100000:2.0:1787739059.526939:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23353 00010000:00010000:3.0:1787739059.527348:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1787739059.527355:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ac8fe lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac905 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.527363:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633da00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739059.527367:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ac8fe lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac905 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739059.527375:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ac8fe lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac905 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.527452:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ac8fe lrc: 2/0,0 mode: --/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac905 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.527459:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739059.527478:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ac8fe lrc: 1/0,0 mode: --/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac905 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.527490:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1ac8fe lrc: 0/0,0 mode: --/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac905 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.527505:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787739059.527508:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac8e2 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac8e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.527514:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633dc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739059.527518:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac8e2 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac8e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739059.527524:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac8e2 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac8e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.527538:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac8e2 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac8e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.527544:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739059.527556:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac8e2 lrc: 1/0,0 mode: --/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac8e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.527562:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ac8e2 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac8e9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.527571:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739059.527574:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac8c6 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac8cd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.527584:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739059.527587:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac8c6 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac8cd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739059.527594:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac8c6 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac8cd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.527610:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac8c6 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac8cd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.527616:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739059.527622:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac8c6 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac8cd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.527628:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ac8c6 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac8cd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.527639:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739059.527641:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739059.527642:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739059.527648:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739059.527655:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787739059.527669:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c380f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665632768:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739059.527717:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.527760:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c382940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665632896:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739059.527784:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.527805:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787739059.527811:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f75680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665633024:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739059.527832:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.527863:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665632768 00000100:00100000:3.0:1787739059.527869:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665632768 00000100:00100000:3.0:1787739059.527904:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665632896 00000100:00100000:3.0:1787739059.527907:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665632896 00000100:00100000:3.0:1787739059.527921:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665633024 00000100:00100000:3.0:1787739059.527923:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665633024 00000100:00100000:3.0:1787739059.527937:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1275 00000100:00100000:3.0:1787739059.527944:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074184c0850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+12:3652:x1874579665632768:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739059.527956:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739059.527976:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac905 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xbe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac8fe expref: 12 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.527988:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbe:0x0].0x0 (ffffa074039f2e00) 00010000:00010000:3.0:1787739059.527995:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739059.528027:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074184c0850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+11:3652:x1874579665632768:12345-0@lo:103:ldlm_bl.0 Request processed in 84us (310us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.528034:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1275 00000100:00100000:3.0:1787739059.528080:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1276 00000100:00100000:3.0:1787739059.528087:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f743c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+10:3650:x1874579665632896:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739059.528094:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739059.528105:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac8e9 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x16e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac8e2 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.528114:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16e:0x0].0x0 (ffffa074039f2500) 00010000:00010000:3.0:1787739059.528120:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739059.528150:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f743c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+9:3650:x1874579665632896:12345-0@lo:103:ldlm_bl.0 Request processed in 65us (365us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.528155:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1276 00000100:00100000:3.0:1787739059.528167:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1277 00000100:00100000:3.0:1787739059.528173:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f76940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+8:3650:x1874579665633024:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739059.528180:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739059.528189:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ac8cd lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac8c6 expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.528199:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eaec00) 00010000:00010000:3.0:1787739059.528204:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739059.528226:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f76940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+7:3650:x1874579665633024:12345-0@lo:103:ldlm_bl.0 Request processed in 54us (393us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.528231:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1277 00000100:00100000:3.0:1787739059.528270:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f75680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665633024:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739059.528281:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c382940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665632896:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739059.528307:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c380f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665632768:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739059.528746:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739059.530565:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739059.554602:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739059.556572:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652456448 00000100:00100000:0.0:1787739059.556586:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652456448 00000100:00100000:2.0:1787739059.556722:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23354 00000100:00100000:2.0:1787739059.556734:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17543:x1874579652456448:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739059.556755:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739059.556779:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac90c lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094e19 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.556840:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac913 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.556854:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac913 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.556865:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac913 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.556878:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac91a lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.556886:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac91a lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.556893:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac91a lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.556933:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac913 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.556940:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac913 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.556947:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac913 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.556953:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac913 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.556960:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739059.556964:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac913 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.556971:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac913 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.556982:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78300) 00010000:00010000:2.0:1787739059.556986:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739059.556990:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac913 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.556998:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac913 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.557011:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac91a lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.557017:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac91a lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.557022:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac91a lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.557028:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739059.557030:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac91a lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.557037:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac91a lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.557057:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xb994c874 (ffffa07303c78f00) 00010000:00010000:2.0:1787739059.557060:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739059.557063:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac91a lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.557068:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ac91a lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.557098:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac90c lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e19 expref: 14 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.557108:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78300) 00010000:00010000:2.0:1787739059.557110:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739059.557114:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ac90c lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e19 expref: 14 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.557126:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633ce00, rc 0) 00000100:00100000:2.0:1787739059.557197:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fcf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17543:x1874579652456448:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 464us (2603us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739059.557205:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23354 00000100:00100000:0.0:1787739059.567171:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739059.567289:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652456576 00000100:00100000:1.0:1787739059.567299:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652456576 00000100:00100000:1.0:1787739059.567351:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23355 00000100:00100000:1.0:1787739059.567361:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17543:x1874579652456576:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739059.567380:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739059.567402:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac921 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094e20 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.567483:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac928 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.567500:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac928 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.567510:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac928 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.567523:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac92f lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.567529:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac92f lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.567536:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac92f lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.567571:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac928 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.567578:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac928 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.567584:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac928 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.567589:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac928 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.567595:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739059.567600:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac928 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.567607:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac928 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.567619:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eaea00) 00010000:00010000:1.0:1787739059.567621:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739059.567626:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac928 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.567632:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ac928 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.567646:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac92f lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.567651:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac92f lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.567657:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac92f lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.567662:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1cc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739059.567664:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac92f lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.567671:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac92f lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.567678:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xb994c874 (ffffa07305eae700) 00010000:00010000:1.0:1787739059.567680:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739059.567683:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac92f lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.567688:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ac92f lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.567723:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac921 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e20 expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.567734:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eaea00) 00010000:00010000:1.0:1787739059.567736:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739059.567740:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ac921 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e20 expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.567751:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c200, rc 0) 00000100:00100000:1.0:1787739059.567809:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17543:x1874579652456576:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 449us (646us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739059.567818:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23355 00000100:00100000:2.0:1787739059.579166:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739059.579210:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652456704 00000100:00100000:2.0:1787739059.579220:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652456704 00000100:00100000:2.0:1787739059.579271:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23356 00000100:00100000:2.0:1787739059.579280:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306007480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17543:x1874579652456704:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739059.579301:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739059.579323:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac936 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094e27 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.579379:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac93d lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.579396:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac93d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.579408:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac93d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739059.579469:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:2.0:1787739059.579475:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739059.579511:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac936 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e27 expref: 15 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.579528:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac93d lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e27 expref: 14 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.579543:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78c00) 00010000:00010000:2.0:1787739059.579546:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739059.579549:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c000, rc 0) 00000100:00100000:2.0:1787739059.579604:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306007480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17543:x1874579652456704:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 324us (445us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.579611:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23356 00000100:00100000:3.0:1787739059.610202:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739059.610255:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652456960 00000100:00100000:3.0:1787739059.610264:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652456960 00000100:00100000:3.0:1787739059.610316:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23357 00000100:00100000:3.0:1787739059.610326:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f75680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17543:x1874579652456960:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:3.0:1787739059.610381:0:6524:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739059.610406:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ac93d lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e27 expref: 14 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.610490:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78c00) 00010000:00010000:3.0:1787739059.610499:0:6524:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739059.610552:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac944 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.610565:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac944 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.610574:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac944 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.610584:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ac94b lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xb994c874 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.610590:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ac94b lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.610596:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ac94b lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787739059.611315:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].362, 2168 off555352 00000040:00080000:3.0:1787739059.611389:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].362, 2368 off601816 00000004:00080000:3.0:1787739059.611395:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa073068ba9c0 with version 691 00000004:00080000:3.0:1787739059.611444:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 691 rpc_version 691 00000020:00100000:3.0:1787739059.611474:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739059.611483:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303f75e00 x1874579665633280/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 3976/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739059.611504:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f75e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665633280:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739059.611561:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.611613:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba780 going to sleep for 11 seconds 00000100:00100000:2.0:1787739059.612647:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665633280 00000100:00100000:2.0:1787739059.612657:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665633280 00000100:00100000:2.0:1787739059.612703:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2959 00000100:00100000:2.0:1787739059.612712:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f77840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665633280:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739059.613197:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f77840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665633280:12345-0@lo:1000:osp_up1-0.0 Request processed in 484us (1633us total) trans 4294968354 rc 0/0 00000100:00100000:2.0:1787739059.613205:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2959 00000100:00100000:0.0:1787739059.614119:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f75e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665633280:0@lo:1000:osp_up1-0.0 00010000:00010000:3.0:1787739059.614273:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ac952 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x150e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.614291:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ac952 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x150e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.614303:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ac952 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x150e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.614324:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac959 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.614331:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac959 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.614338:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac959 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.614371:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ac960 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.614378:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ac960 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.614387:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ac960 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.614406:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74b40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665633408:0@lo:101:mdt00_001.0 00000100:00100000:3.0:1787739059.614490:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.614528:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba780 going to sleep for 11 seconds 00000100:00100000:2.0:1787739059.615572:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665633408 00000100:00100000:2.0:1787739059.615582:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665633408 00000100:00100000:2.0:1787739059.615666:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2960 00000100:00100000:2.0:1787739059.615676:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f75a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665633408:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739059.615698:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739059.615718:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac967 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac960 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.615741:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac967 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac960 expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.615755:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac967 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac960 expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.615768:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc1:0x0].0x0 (ffffa07303c78300) 00010000:00010000:2.0:1787739059.615772:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739059.615775:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633da00, rc 0) 00000100:00100000:2.0:1787739059.615841:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f75a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6524:x1874579665633408:12345-0@lo:101:mdt00_001.0 Request processed in 167us (1351us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.615848:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2960 00000100:00100000:0.0:1787739059.615921:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74b40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665633408:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739059.615938:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ac960 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac967 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.615949:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ac960 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac967 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739059.615973:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c27c269f reply state 00000000a0951399 transno 4294983361 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac959 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.615985:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac959 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.615991:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc1:0x0].0x0 (ffffa074039f2000) 00010000:00010000:0.0:1787739059.615993:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739059.615996:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739059.616001:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ac960 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac967 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.616007:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ac960 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac967 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739059.616018:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xc1:0x0] 00000004:00010000:0.0:1787739059.616025:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c27c269f reply state 00000000a0951399 transno 4294983361 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ac952 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x150e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.616032:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ac952 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x150e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.616038:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x150e:0x0].0x0 (ffffa074039f2f00) 00010000:00010000:0.0:1787739059.616051:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739059.616053:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787739059.616058:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x150e:0x0]: returning size 4096 00010000:00010000:0.0:1787739059.616068:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac944 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.616074:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac944 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.616080:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac944 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.616086:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac944 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.616091:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aae00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739059.616095:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac944 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.616101:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac944 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.616112:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074039f2500) 00010000:00010000:0.0:1787739059.616114:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739059.616117:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac944 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.616123:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ac944 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739059.616138:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c27c269f reply state 00000000a0951399 transno 4294983361 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ac94b lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.616146:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ac94b lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.616151:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xb994c874 (ffffa074039f2e00) 00010000:00010000:0.0:1787739059.616154:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739059.616155:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 2 00000100:00100000:0.0:1787739059.616231:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f75680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17543:x1874579652456960:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 5904us (6036us total) trans 4294983361 rc 0/0 00000100:00100000:0.0:1787739059.616240:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23357 00000100:00100000:1.0:1787739059.650619:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787739059.650644:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665633536:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1787739059.650743:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739059.650929:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665633536 00000100:00100000:0.0:1787739059.650937:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665633536 00000100:00100000:0.0:1787739059.650990:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2961 00000100:00100000:0.0:1787739059.651002:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:3653:x1874579665633536:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1787739059.651050:0:18535:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1444044800 avail 1297805312 00000100:00100000:0.0:1787739059.651106:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:3653:x1874579665633536:12345-0@lo:41:osp-pre-0-1.0 Request processed in 107us (362us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739059.651113:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2961 00000100:00100000:1.0:1787739059.651199:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665633536:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1787739059.653017:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739059.653134:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652457216 00000100:00100000:1.0:1787739059.653143:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652457216 00000100:00100000:1.0:1787739059.653207:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23358 00000100:00100000:1.0:1787739059.653216:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17544:x1874579652457216:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:1.0:1787739059.654258:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17544:x1874579652457216:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 1038us (1243us total) trans 0 rc -17/-17 00000100:00100000:1.0:1787739059.654268:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23358 00000100:00100000:2.0:1787739059.670654:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739059.672009:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652457344 00000100:00100000:3.0:1787739059.672022:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652457344 00000100:00100000:3.0:1787739059.672095:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23359 00000100:00100000:3.0:1787739059.672106:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17545:x1874579652457344:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739059.672126:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739059.672149:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac96e lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094e2e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672213:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac975 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672228:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac975 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672240:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac975 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672252:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac97c lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672259:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac97c lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672267:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac97c lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672350:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ac983 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x150e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672358:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ac983 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x150e:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672375:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ac983 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x150e:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739059.672471:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x150e:0x0]: returning size 4096 00000004:00200000:3.0:1787739059.672476:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:3.0:1787739059.672496:0:6525:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ac983 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x150e:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672509:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac975 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672515:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac975 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672521:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac975 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672527:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac975 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672532:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739059.672536:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac975 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672542:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac975 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672553:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074039f2300) 00010000:00010000:3.0:1787739059.672557:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739059.672562:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac975 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672568:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ac975 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672581:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac97c lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672587:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac97c lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672593:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac97c lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672597:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739059.672600:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac97c lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672605:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac97c lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672611:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xb994c874 (ffffa074039f2e00) 00010000:00010000:3.0:1787739059.672614:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739059.672617:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac97c lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672623:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac97c lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672643:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ac96e lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e2e expref: 17 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672658:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ac983 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x150e:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e2e expref: 16 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.672670:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x150e:0x0].0x0 (ffffa074039f2f00) 00010000:00010000:3.0:1787739059.672672:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787739059.672674:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739059.672752:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17545:x1874579652457344:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 650us (2103us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.672760:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23359 00000100:00100000:3.0:1787739059.682099:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739059.682179:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652457472 00000100:00100000:2.0:1787739059.682189:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652457472 00000100:00100000:2.0:1787739059.682245:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23360 00000100:00100000:2.0:1787739059.682256:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f770c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17545:x1874579652457472:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739059.682276:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739059.682296:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac98a lrc: 2/0,0 mode: --/CR res: [0x200000400:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094e35 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.682351:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac991 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.682365:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac991 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.682386:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac991 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739059.682453:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xc1:0x0]: returning size 4096 00000004:00200000:2.0:1787739059.682458:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739059.682492:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ac98a lrc: 0/0,0 mode: --/CR res: [0x200000400:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e35 expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.682512:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac991 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e35 expref: 17 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.682521:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc1:0x0].0x0 (ffffa074039f2000) 00010000:00010000:2.0:1787739059.682525:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739059.682528:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d000, rc 0) 00000100:00100000:2.0:1787739059.682596:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f770c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17545:x1874579652457472:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 330us (492us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.682604:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23360 00000100:00100000:1.0:1787739059.692811:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739059.692924:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652457600 00000100:00100000:0.0:1787739059.692935:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652457600 00000100:00100000:0.0:1787739059.693222:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23361 00000100:00100000:0.0:1787739059.693234:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17545:x1874579652457600:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739059.693258:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739059.693279:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac998 lrc: 2/0,0 mode: --/CR res: [0x240000401:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094e3c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.693337:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac99f lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.693354:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac967 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac960 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.693363:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac99f lrc: 4/1,0 mode: --/PR res: [0x240000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.693797:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac967 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ac960 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739059.693814:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561cf00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665633664:0@lo:104: 00000100:00100000:0.0:1787739059.693869:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739059.693957:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739059.694671:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665633664 00000100:00100000:1.0:1787739059.694679:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665633664 00000100:00100000:1.0:1787739059.694731:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1123 00000100:00100000:1.0:1787739059.694739:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561de00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665633664:12345-0@lo:104: 00010000:00010000:1.0:1787739059.694753:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ac960 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac967 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.694819:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787739059.694825:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561de00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665633664:12345-0@lo:104: Request processed in 88us (956us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.694832:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1123 00010000:00010000:1.0:1787739059.694856:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739059.694862:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ac960 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ac967 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.694871:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739059.694876:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ac960 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xc1:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ac967 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739059.694885:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ac960 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xc1:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ac967 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.694911:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ac960 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xc1:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac967 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.694918:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739059.694936:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ac960 lrc: 1/0,0 mode: --/PW res: [0x240000401:0xc1:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac967 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.694943:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ac960 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xc1:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ac967 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.694959:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739059.694961:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739059.694963:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739059.694986:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665633792:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739059.695204:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739059.695245:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665633792 00000100:00100000:1.0:1787739059.695249:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665633792 00010000:00010000:1.0:1787739059.695267:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac967 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ac960 expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739059.695290:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1468 00000100:00100000:1.0:1787739059.695297:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+10:3653:x1874579665633792:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739059.695308:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787739059.695462:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561cf00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665633664:0@lo:104: 00010000:00010000:1.0:1787739059.695483:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac99f lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.702079:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ac8d4 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac8db expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739059.702100:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ac8d4 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac8db expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.702141:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ac8d4 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac8db expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.702149:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739059.702187:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ac8d4 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac8db expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787739059.702207:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968354 is committed 00000001:00080000:1.0:1787739059.702224:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968353 is committed 00000020:00080000:1.0:1787739059.702232:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 354 batchid updated 00000020:00080000:1.0:1787739059.702240:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa0743c413cc0 refcount 2 committed 0 result 0 batchid 355 00000020:00080000:1.0:1787739059.702243:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730415f700 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa0743c413c00 00000020:00080000:1.0:1787739059.702246:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].348 00000020:00080000:1.0:1787739059.702248:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730415ff00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa0730541f000 00000020:00080000:1.0:1787739059.702251:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].348 00000020:00080000:1.0:1787739059.702265:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 353 batchid updated 00000100:00100000:1.0:1787739059.702283:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787739059.702302:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c1e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665633920:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787739059.702362:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787739059.702441:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac864 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x16e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.702450:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac864 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x16e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.702456:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac864 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x16e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.702462:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633de00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739059.702466:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac864 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x16e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.702472:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac864 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x16e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.702487:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16e:0x0].0x0 (ffffa07303c78a00) 00010000:00010000:1.0:1787739059.702489:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739059.702494:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac864 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x16e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.702500:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ac864 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x16e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.702517:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ac86b lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.702523:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ac86b lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.702529:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ac86b lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.702534:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739059.702536:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ac86b lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.702542:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ac86b lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.702549:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xbe:0x0].0x0 (ffffa07303c78000) 00010000:00010000:1.0:1787739059.702551:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739059.702554:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ac86b lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.702560:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ac86b lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xbe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:1.0:1787739059.702583:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 355 committed batchid 354 00000004:00080000:1.0:1787739059.702678:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730610ec00 with version 663 00000020:00080000:1.0:1787739059.702700:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 351 cancel update log [0x200000401:0x3:0x0].344: rc = 0 00000020:00080000:1.0:1787739059.702740:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 351 cancel update log [0x240000400:0x3:0x0].343: rc = 0 00000004:00080000:1.0:1787739059.702762:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730610e480 with version 664 00000020:00080000:1.0:1787739059.702765:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 352 cancel update log [0x200000401:0x3:0x0].345: rc = 0 00000020:00080000:1.0:1787739059.702783:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 352 cancel update log [0x240000400:0x3:0x0].344: rc = 0 00000004:00080000:1.0:1787739059.702802:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730610e0c0 with version 665 00000020:00080000:1.0:1787739059.702805:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 353 cancel update log [0x200000401:0x3:0x0].346: rc = 0 00000020:00080000:1.0:1787739059.702823:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 353 cancel update log [0x240000400:0x3:0x0].346: rc = 0 00000004:00080000:1.0:1787739059.702848:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730610ecc0 with version 666 00000020:00080000:1.0:1787739059.702851:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 354 cancel update log [0x200000401:0x3:0x0].347: rc = 0 00000020:00080000:1.0:1787739059.702867:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 354 cancel update log [0x240000400:0x3:0x0].347: rc = 0 00000020:00080000:1.0:1787739059.702873:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 354 00010000:00010000:1.0:1787739059.702915:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ac967 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ac960 expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.702926:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc1:0x0].0x0 (ffffa07303c78300) 00010000:00010000:1.0:1787739059.702930:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac99f lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.702939:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac99f lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.702947:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac99f lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.702968:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739059.703009:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+5:3653:x1874579665633792:12345-0@lo:103:ldlm_bl.0 Request processed in 7712us (7977us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.703019:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1468 00000100:00100000:1.0:1787739059.703051:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665633920 00000100:00100000:1.0:1787739059.703057:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665633920 00000100:00100000:1.0:1787739059.703094:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1278 00000100:00100000:1.0:1787739059.703103:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c2580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+7:3651:x1874579665633920:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:1.0:1787739059.703114:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739059.703127:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ac8db lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xee50ff8a bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac8d4 expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.703138:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xee50ff8a (ffffa07305eae100) 00010000:00010000:1.0:1787739059.703145:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739059.703173:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c2580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+6:3651:x1874579665633920:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 71us (810us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.703179:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1278 00000004:00080000:1.0:1787739059.703195:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 663 rpc_version 663 00000020:00100000:1.0:1787739059.703214:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739059.703222:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073043c0f00 x1874579665634176/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739059.703239:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c0f00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665634176:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739059.703268:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739059.703287:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610eb40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739059.703317:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665633792:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739059.703345:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665634176 00000100:00100000:1.0:1787739059.703350:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665634176 00000100:00100000:1.0:1787739059.703374:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2962 00000100:00100000:1.0:1787739059.703380:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c30c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:7467:x1874579665634176:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739059.703509:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c30c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665634176:12345-0@lo:1000:osp_up0-1.0 Request processed in 128us (239us total) trans 4294983362 rc 0/0 00000100:00100000:1.0:1787739059.703516:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2962 00000100:00100000:1.0:1787739059.703551:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c0f00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665634176:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787739059.703559:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 664 rpc_version 664 00000020:00100000:1.0:1787739059.703569:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739059.703574:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073043c1a40 x1874579665634432/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739059.703586:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c1a40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665634432:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739059.703610:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739059.703627:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610eb40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739059.703638:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665634432 00000100:00100000:1.0:1787739059.703642:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665634432 00000100:00100000:1.0:1787739059.703667:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2963 00000100:00100000:1.0:1787739059.703673:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c2940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665634432:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739059.703746:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c2940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665634432:12345-0@lo:1000:osp_up0-1.0 Request processed in 72us (134us total) trans 4294983363 rc 0/0 00000100:00100000:1.0:1787739059.703753:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2963 00000100:00100000:1.0:1787739059.703779:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c1e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665633920:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1787739059.703804:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c1a40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665634432:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787739059.703810:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 665 rpc_version 665 00000020:00100000:1.0:1787739059.703819:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739059.703824:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073043c1e00 x1874579665634688/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739059.703837:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c1e00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665634688:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739059.703860:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739059.703875:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610eb40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739059.703886:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665634688 00000100:00100000:1.0:1787739059.703889:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665634688 00000100:00100000:1.0:1787739059.703910:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2964 00000100:00100000:1.0:1787739059.703916:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665634688:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739059.703984:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665634688:12345-0@lo:1000:osp_up0-1.0 Request processed in 67us (122us total) trans 4294983364 rc 0/0 00000100:00100000:1.0:1787739059.703991:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2964 00000100:00100000:1.0:1787739059.704019:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c1e00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665634688:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787739059.704026:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 666 rpc_version 666 00000020:00100000:1.0:1787739059.704035:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739059.704051:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073043c12c0 x1874579665634944/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739059.704063:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c12c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665634944:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739059.704087:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739059.704098:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610eb40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739059.704107:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665634944 00000100:00100000:1.0:1787739059.704110:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665634944 00000100:00100000:1.0:1787739059.704127:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2965 00000100:00100000:1.0:1787739059.704132:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07436d8f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665634944:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739059.704193:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07436d8f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665634944:12345-0@lo:1000:osp_up0-1.0 Request processed in 60us (104us total) trans 4294983365 rc 0/0 00000100:00100000:1.0:1787739059.704202:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2965 00010000:00010000:1.0:1787739059.704323:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac99f lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.704332:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac99f lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739059.704378:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xc1:0x0]: returning size 4096 00000004:00200000:1.0:1787739059.704383:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739059.704440:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ac998 lrc: 0/0,0 mode: --/CR res: [0x240000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e3c expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.704457:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac99f lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e3c expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.704465:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc1:0x0].0x0 (ffffa07303c78300) 00010000:00010000:1.0:1787739059.704468:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739059.704471:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a200, rc 0) 00000100:00100000:1.0:1787739059.704512:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17545:x1874579652457600:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 11282us (11709us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.704519:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23361 00000100:00100000:1.0:1787739059.704643:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c12c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665634944:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739059.717636:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739059.718831:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652457856 00000100:00100000:0.0:1787739059.718842:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652457856 00000100:00100000:0.0:1787739059.718936:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23362 00000100:00100000:0.0:1787739059.718947:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17545:x1874579652457856:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:0.0:1787739059.719016:0:6523:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739059.719159:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ac983 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x150e:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e2e expref: 17 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.719177:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x150e:0x0].0x0 (ffffa074039f2f00) 00010000:00010000:0.0:1787739059.719182:0:6523:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739059.719207:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac9a6 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.719219:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac9a6 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.719233:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac9a6 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.719243:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac9ad lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xb994c874 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.719250:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac9ad lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.719257:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac9ad lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.719332:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac9b4 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x150e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.719340:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac9b4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x150e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.719347:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac9b4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x150e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.719365:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9bb lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.719375:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac991 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e35 expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.719395:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9bb lrc: 4/0,1 mode: --/EX res: [0x200000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.719442:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac991 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e35 expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.719456:0:6523:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac991 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e35 expref: 16 pid: 6524 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739059.719469:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561f840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665635072:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739059.719789:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739059.726592:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561f840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665635072:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739059.726620:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9bb lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739059.731826:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739059.731936:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652457984 00000100:00100000:1.0:1787739059.731952:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652457984 00010000:00010000:1.0:1787739059.732013:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac991 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e35 expref: 17 pid: 6524 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739059.732059:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1469 00000100:00100000:1.0:1787739059.732070:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2408:x1874579652457984:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739059.732086:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739059.732093:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac991 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e35 expref: 17 pid: 6524 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.732105:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac991 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e35 expref: 17 pid: 6524 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.732127:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ac991 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e35 expref: 17 pid: 6524 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.732143:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc1:0x0].0x0 (ffffa074039f2000) 00010000:00010000:1.0:1787739059.732148:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9bb lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.732158:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9bb lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.732166:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9bb lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.732199:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739059.732253:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:2408:x1874579652457984:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 185us (436us total) trans 0 rc 0/0 00010000:00010000:0.0:1787739059.732350:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9bb lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.732359:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9bb lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.732446:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac9c2 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.732455:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac9c2 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.732464:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac9c2 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739059.732483:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561f840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665635200:0@lo:101:mdt00_000.0 00000100:00100000:0.0:1787739059.732558:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739059.732591:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.732611:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665635200 00000100:00100000:3.0:1787739059.732617:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665635200 00000100:00100000:3.0:1787739059.732666:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2966 00000100:00100000:3.0:1787739059.732690:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+5:6523:x1874579665635200:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739059.732706:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739059.732723:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac9c9 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ac9c2 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.732744:0:17834:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac99f lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e3c expref: 7 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.732754:0:17834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac9c9 lrc: 4/0,0 mode: --/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac9c2 expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.732777:0:17834:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac99f lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e3c expref: 7 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.732791:0:17834:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac99f lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e3c expref: 7 pid: 7944 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.732809:0:17834:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f76580 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:17834:1874579665635328:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739059.732823:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1469 00000100:00100000:3.0:1787739059.732878:0:17834:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba300 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.737242:0:17834:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f76580 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:17834:1874579665635328:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739059.737278:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac9c9 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac9c2 expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.737292:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc1:0x0].0x0 (ffffa07303c78300) 00010000:00010000:3.0:1787739059.737295:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739059.737300:0:17834:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac9c9 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac9c2 expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.737310:0:17834:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac9c9 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac9c2 expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.737317:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab000, rc 0) 00000100:00100000:3.0:1787739059.737378:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6523:x1874579665635200:12345-0@lo:101:mdt00_000.0 Request processed in 4696us (4819us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.737387:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2966 00000100:00100000:0.0:1787739059.737460:0:6523:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968354 gen 2 00000004:00080000:0.0:1787739059.737467:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968349 committed transno 4294968354 00000020:00080000:0.0:1787739059.737472:0:6523:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07410404180 refcount 2 committed 0 result 0 batchid 364 00000020:00080000:0.0:1787739059.737477:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa074356c7f00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07410404000 00000020:00080000:0.0:1787739059.737480:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].361 00000020:00080000:0.0:1787739059.737482:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa074356c7700 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304010c00 00000020:00080000:0.0:1787739059.737485:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].361 00000004:00080000:0.0:1787739059.737518:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968351 committed transno 4294968354 00000004:00080000:0.0:1787739059.737529:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968352 committed transno 4294968354 00000004:00080000:0.0:1787739059.737536:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968354 committed transno 4294968354 00000100:00100000:0.0:1787739059.737548:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561f840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665635200:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739059.737564:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac9c2 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac9c9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.737576:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac9c2 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac9c9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739059.737579:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 364 committed batchid 363 00000020:00080000:2.0:1787739059.737660:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 363 00000100:00100000:2.0:1787739059.751254:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739059.751326:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652458112 00000100:00100000:1.0:1787739059.751337:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652458112 00010000:00010000:1.0:1787739059.751495:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac99f lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e3c expref: 8 pid: 7944 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739059.751545:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1470 00000100:00100000:1.0:1787739059.751556:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2409:x1874579652458112:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739059.751570:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739059.751577:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac99f lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e3c expref: 8 pid: 7944 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.751591:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac99f lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e3c expref: 8 pid: 7944 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.751615:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ac99f lrc: 0/0,0 mode: --/PR res: [0x240000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e3c expref: 8 pid: 7944 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.751797:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc1:0x0].0x0 (ffffa07303c78300) 00010000:00010000:1.0:1787739059.751806:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac9c9 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac9c2 expref: 5 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.751817:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac9c9 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac9c2 expref: 5 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.751826:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac9c9 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ac9c2 expref: 5 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.752093:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac9c9 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ac9c2 expref: 5 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739059.752111:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c30c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665635456:0@lo:105: 00000100:00100000:1.0:1787739059.752167:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739059.752213:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610eb40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739059.752234:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665635456 00000100:00100000:2.0:1787739059.752240:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665635456 00000100:00100000:2.0:1787739059.752279:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1124 00000100:00100000:2.0:1787739059.752287:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c2940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665635456:12345-0@lo:105: 00010000:00010000:2.0:1787739059.752299:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac9c2 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac9c9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.752310:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac9c2 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac9c9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.752320:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac9c2 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ac9c9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.752332:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac9c2 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac9c9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.752349:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac9c2 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ac9c9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.752357:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac9c2 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ac9c9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.752382:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609be00) 00000100:00100000:2.0:1787739059.752449:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c2940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665635456:12345-0@lo:105: Request processed in 162us (281us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.752456:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1124 00010000:00010000:0.0:1787739059.752532:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac9c2 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac9c9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.752541:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac9c2 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac9c9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739059.752559:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c30c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665635456:0@lo:105: 00010000:00010000:1.0:1787739059.752572:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739059.752619:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2409:x1874579652458112:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 1064us (1374us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.752626:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1470 00000100:00100000:0.0:1787739059.753257:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561f840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665635584:0@lo:403:mdt00_000.0 00000100:00100000:0.0:1787739059.753300:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739059.753326:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404240 going to sleep for 11 seconds 00000100:00100000:1.0:1787739059.754325:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665635584 00000100:00100000:1.0:1787739059.754334:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665635584 00000100:00100000:1.0:1787739059.754382:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2967 00000100:00100000:1.0:1787739059.754390:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6523:x1874579665635584:12345-0@lo:403:mdt00_000.0 00000100:00100000:1.0:1787739059.758180:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:6523:x1874579665635584:12345-0@lo:403:mdt00_000.0 Request processed in 3780us (4869us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.758207:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2967 00000100:00100000:0.0:1787739059.758458:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561f840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665635584:0@lo:403:mdt00_000.0 00000040:00080000:0.0:1787739059.759444:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].363, 664 off556016 00000040:00080000:0.0:1787739059.759627:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].363, 744 off602560 00000004:00080000:0.0:1787739059.759633:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07410404240 with version 692 00000004:00080000:3.0:1787739059.759788:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 692 rpc_version 692 00000020:00100000:3.0:1787739059.759818:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739059.759827:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303f76580 x1874579665635840/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1448/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739059.759849:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f76580 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665635840:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739059.759910:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.759952:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739059.761552:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665635840 00000100:00100000:0.0:1787739059.761563:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665635840 00000100:00100000:0.0:1787739059.761628:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2968 00000100:00100000:0.0:1787739059.761639:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665635840:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739059.762026:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665635840:12345-0@lo:1000:osp_up1-0.0 Request processed in 388us (2114us total) trans 4294968355 rc 0/0 00000100:00100000:0.0:1787739059.762034:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2968 00000100:00100000:1.0:1787739059.763406:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f76580 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665635840:0@lo:1000:osp_up1-0.0 00000004:00200000:0.0:1787739059.764624:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x150e:0x0]: returning size 4096 00010000:00010000:0.0:1787739059.764653:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9bb lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764664:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9bb lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764736:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9bb lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764743:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739059.764747:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9bb lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764756:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9bb lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764774:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc1:0x0].0x0 (ffffa074039f2000) 00010000:00010000:0.0:1787739059.764777:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739059.764781:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9bb lrc: 2/0,0 mode: --/EX res: [0x200000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764790:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9bb lrc: 0/0,0 mode: --/EX res: [0x200000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764807:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac9c2 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ac9c9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764815:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac9c2 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac9c9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764821:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac9c2 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ac9c9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764841:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739059.764851:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xc1:0x0] 00010000:00010000:0.0:1787739059.764858:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac9b4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x150e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764864:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac9b4 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x150e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764869:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac9b4 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x150e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764874:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739059.764877:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac9b4 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x150e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764882:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac9b4 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x150e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764892:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x150e:0x0].0x0 (ffffa074039f2f00) 00010000:00010000:0.0:1787739059.764894:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739059.764897:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac9b4 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x150e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764901:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac9b4 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x150e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764916:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac9a6 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764922:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac9a6 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764927:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac9a6 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764931:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac9a6 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764935:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739059.764937:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac9a6 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764942:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac9a6 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764948:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07307064800) 00010000:00010000:0.0:1787739059.764950:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739059.764953:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac9a6 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764962:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac9a6 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739059.764976:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d3abdc93 reply state 000000006b6c099b transno 4294983366 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac9ad lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764987:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac9ad lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.764994:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xb994c874 (ffffa074039f2e00) 00010000:00010000:0.0:1787739059.764996:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739059.764998:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787739059.765099:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17545:x1874579652457856:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 46151us (47470us total) trans 4294983366 rc 0/0 00000100:00100000:0.0:1787739059.765108:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23362 00010000:00010000:1.0:1787739059.765808:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739059.765817:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac9c2 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ac9c9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.765826:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609be00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739059.765831:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac9c2 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ac9c9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739059.765839:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac9c2 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ac9c9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.765873:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac9c2 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac9c9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.765881:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739059.765911:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac9c2 lrc: 1/0,0 mode: --/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac9c9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.765919:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609be00/0x7ee4453e4a1ac9c2 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ac9c9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.765936:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739059.765938:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739059.765940:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739059.765980:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787739059.765998:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665635968:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739059.766063:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739059.766150:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665635968 00000100:00100000:1.0:1787739059.766157:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665635968 00000100:00100000:1.0:1787739059.766209:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1279 00000100:00100000:1.0:1787739059.766218:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074184c2850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+7:3650:x1874579665635968:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739059.766231:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739059.766248:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ac9c9 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ac9c2 expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.766260:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc1:0x0].0x0 (ffffa07303c78300) 00010000:00010000:1.0:1787739059.766269:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739059.766305:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074184c2850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+6:3650:x1874579665635968:12345-0@lo:103:ldlm_bl.0 Request processed in 87us (242us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.766312:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1279 00000100:00100000:1.0:1787739059.766365:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665635968:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739059.788933:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739059.789035:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652458240 00000100:00100000:0.0:1787739059.789052:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652458240 00000100:00100000:0.0:1787739059.789110:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23363 00000100:00100000:0.0:1787739059.789120:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17546:x1874579652458240:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739059.789144:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739059.789166:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac9d0 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094e43 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.789242:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac9d7 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.789255:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac9d7 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.789271:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac9d7 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.789284:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9de lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.789291:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9de lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.789298:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9de lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.789339:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac9d7 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.789347:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac9d7 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.789353:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac9d7 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.789360:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac9d7 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.789366:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739059.789370:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac9d7 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.789377:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac9d7 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.789388:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07307064800) 00010000:00010000:0.0:1787739059.789392:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739059.789396:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac9d7 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.789402:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ac9d7 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.789448:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9de lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.789456:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9de lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.789462:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9de lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.789468:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739059.789471:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9de lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.789476:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9de lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.789483:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x872f39a (ffffa07307064600) 00010000:00010000:0.0:1787739059.789486:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739059.789488:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9de lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.789494:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ac9de lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.789526:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac9d0 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e43 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.789541:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07307064800) 00010000:00010000:0.0:1787739059.789543:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739059.789547:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ac9d0 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e43 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.789557:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b600, rc 0) 00000100:00100000:0.0:1787739059.789609:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17546:x1874579652458240:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 491us (684us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739059.789617:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23363 00000100:00100000:0.0:1787739059.800063:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739059.800283:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652458368 00000100:00100000:1.0:1787739059.800292:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652458368 00000100:00100000:1.0:1787739059.800357:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23364 00000100:00100000:1.0:1787739059.800368:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17546:x1874579652458368:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739059.800389:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739059.800451:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac9e5 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094e4a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.800515:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac9ec lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.800529:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac9ec lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.800541:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac9ec lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.800554:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac9f3 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.800561:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac9f3 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.800568:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac9f3 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.800611:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac9ec lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.800619:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac9ec lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.800626:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac9ec lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.800635:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac9ec lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.800641:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739059.800645:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac9ec lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.800653:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac9ec lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.800664:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235200) 00010000:00010000:1.0:1787739059.800677:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739059.800681:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac9ec lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.800688:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ac9ec lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.800702:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac9f3 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.800712:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac9f3 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.800718:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac9f3 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.800723:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739059.800726:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac9f3 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.800735:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac9f3 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.800742:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x872f39a (ffffa07433235900) 00010000:00010000:1.0:1787739059.800744:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739059.800747:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac9f3 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.800751:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ac9f3 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.800781:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac9e5 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e4a expref: 18 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.800790:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235200) 00010000:00010000:1.0:1787739059.800792:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739059.800795:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac9e5 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e4a expref: 18 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.800806:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1ca00, rc 0) 00000100:00100000:1.0:1787739059.800866:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17546:x1874579652458368:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 499us (811us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739059.800873:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23364 00000100:00100000:1.0:1787739059.812391:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739059.813627:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652458496 00000100:00100000:0.0:1787739059.813639:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652458496 00000100:00100000:1.0:1787739059.813775:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23365 00000100:00100000:1.0:1787739059.813787:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17546:x1874579652458496:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739059.813815:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739059.813840:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac9fa lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094e51 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.813904:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aca01 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.813918:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aca01 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.813930:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aca01 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739059.813976:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:1.0:1787739059.813986:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739059.814020:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ac9fa lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e51 expref: 19 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.814037:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aca01 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e51 expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.814154:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235200) 00010000:00010000:1.0:1787739059.814157:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739059.814161:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c800, rc 0) 00000100:00100000:1.0:1787739059.814230:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17546:x1874579652458496:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 445us (1848us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.814237:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23365 00000100:00100000:1.0:1787739059.822966:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739059.823098:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652458752 00000100:00100000:3.0:1787739059.823108:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652458752 00000100:00100000:3.0:1787739059.823201:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23366 00000100:00100000:3.0:1787739059.823212:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17546:x1874579652458752:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:3.0:1787739059.823322:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f76d00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665636096:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739059.823489:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.823543:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739059.823932:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665636096 00000100:00100000:2.0:1787739059.823938:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665636096 00000100:00100000:2.0:1787739059.823986:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2969 00000100:00100000:2.0:1787739059.823994:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f752c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6525:x1874579665636096:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739059.824090:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f752c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6525:x1874579665636096:12345-0@lo:1000:mdt00_002.0 Request processed in 96us (600us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.824096:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2969 00000100:00100000:0.0:1787739059.824969:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f76d00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665636096:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739059.825035:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561e940 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665636224:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739059.825088:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739059.825116:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074104046c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739059.825956:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665636224 00000100:00100000:1.0:1787739059.825964:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665636224 00000100:00100000:1.0:1787739059.826014:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2970 00000100:00100000:1.0:1787739059.826023:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6525:x1874579665636224:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739059.826153:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6525:x1874579665636224:12345-0@lo:1000:mdt00_002.0 Request processed in 131us (1063us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739059.826160:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2970 00000100:00100000:0.0:1787739059.826287:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561e940 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665636224:0@lo:1000:mdt00_002.0 00010000:00010000:0.0:1787739059.826352:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1aca08 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.826362:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1aca08 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.826373:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1aca08 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739059.826389:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561e940 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665636352:0@lo:101:mdt00_002.0 00000100:00100000:0.0:1787739059.826455:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739059.826489:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739059.826765:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665636352 00000100:00100000:2.0:1787739059.826771:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665636352 00000100:00100000:2.0:1787739059.826805:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2971 00000100:00100000:2.0:1787739059.826813:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6525:x1874579665636352:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787739059.826824:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739059.826839:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aca0f lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aca08 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.826864:0:18535:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aca01 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e51 expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.826875:0:18535:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aca0f lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aca08 expref: 11 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.826898:0:18535:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aca01 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e51 expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.826911:0:18535:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aca01 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e51 expref: 17 pid: 6523 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739059.826922:0:18535:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529a40 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:18535:1874579665636480:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739059.827115:0:18535:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887600 going to sleep for 11 seconds 00000100:00100000:0.0:1787739059.838217:0:18535:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529a40 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:18535:1874579665636480:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739059.838257:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aca0f lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aca08 expref: 11 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.838271:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235200) 00010000:00010000:0.0:1787739059.838274:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739059.838277:0:18535:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aca0f lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aca08 expref: 11 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.838288:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c400, rc 0) 00000100:00100000:0.0:1787739059.838358:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:6525:x1874579665636352:12345-0@lo:101:mdt00_002.0 Request processed in 11544us (11901us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739059.838368:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2971 00000100:00100000:3.0:1787739059.838449:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561e940 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665636352:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739059.838469:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1aca08 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aca0f expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.838480:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1aca08 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aca0f expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739059.842149:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739059.842800:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652458880 00000100:00100000:2.0:1787739059.842811:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652458880 00010000:00010000:2.0:1787739059.842852:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aca01 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e51 expref: 18 pid: 6523 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739059.842893:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1471 00000100:00100000:2.0:1787739059.842902:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c2580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2408:x1874579652458880:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739059.842919:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739059.842925:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aca01 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e51 expref: 18 pid: 6523 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.842945:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aca01 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e51 expref: 18 pid: 6523 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.842967:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1aca01 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e51 expref: 18 pid: 6523 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.842984:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235200) 00010000:00010000:2.0:1787739059.842991:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aca0f lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aca08 expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.843002:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aca0f lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1aca08 expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.843012:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aca0f lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1aca08 expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.843057:0:6508:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aca0f lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aca08 expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739059.843070:0:6508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529a40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665636608:0@lo:105: 00000100:00100000:2.0:1787739059.843122:0:6508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739059.843168:0:6508:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068873c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.843234:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665636608 00000100:00100000:3.0:1787739059.843238:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665636608 00000100:00100000:3.0:1787739059.843277:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1125 00000100:00100000:3.0:1787739059.843284:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665636608:12345-0@lo:105: 00010000:00010000:3.0:1787739059.843295:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1aca08 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aca0f expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.843305:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1aca08 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aca0f expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.843314:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1aca08 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aca0f expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.843322:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1aca08 lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1aca0f expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.843329:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1aca08 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1aca0f expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.843338:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1aca08 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1aca0f expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.843358:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609b600) 00000100:00100000:3.0:1787739059.843402:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665636608:12345-0@lo:105: Request processed in 119us (280us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.843409:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1125 00010000:00010000:3.0:1787739059.843458:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1aca08 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aca0f expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.843469:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1aca08 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aca0f expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739059.843494:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739059.843525:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aca16 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.843533:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aca16 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.843543:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aca16 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.843556:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665636736:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1787739059.843583:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.843602:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba000 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.843614:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665636736 00000100:00100000:3.0:1787739059.843616:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665636736 00000100:00100000:3.0:1787739059.843642:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2972 00000100:00100000:3.0:1787739059.843648:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f77c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:6525:x1874579665636736:12345-0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739059.843663:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739059.843688:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aca1d lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aca16 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.843707:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aca1d lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aca16 expref: 12 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.843720:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aca1d lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aca16 expref: 12 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.843727:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x872f39a (ffffa074039f2400) 00010000:00010000:3.0:1787739059.843730:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739059.843732:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab600, rc 0) 00000100:00100000:3.0:1787739059.843764:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f77c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:6525:x1874579665636736:12345-0@lo:101:mdt00_002.0 Request processed in 116us (181us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.843771:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2972 00000100:00100000:3.0:1787739059.843806:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665636736:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739059.843819:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aca16 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aca1d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.843828:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aca16 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aca1d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739059.843839:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:3.0:1787739059.843866:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665636864:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739059.843891:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.843905:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.843918:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665636864 00000100:00100000:3.0:1787739059.843922:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665636864 00000100:00100000:3.0:1787739059.843943:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2973 00000100:00100000:3.0:1787739059.843949:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f76d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:6525:x1874579665636864:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739059.844004:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f76d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:6525:x1874579665636864:12345-0@lo:1000:mdt00_002.0 Request processed in 55us (112us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.844011:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2973 00000100:00100000:3.0:1787739059.844033:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665636864:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739059.844079:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665636992:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739059.844102:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.844116:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.844127:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665636992 00000100:00100000:3.0:1787739059.844131:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665636992 00000100:00100000:3.0:1787739059.844150:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2974 00000100:00100000:3.0:1787739059.844155:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f743c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:6525:x1874579665636992:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739059.844214:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f743c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:6525:x1874579665636992:12345-0@lo:1000:mdt00_002.0 Request processed in 58us (111us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739059.844222:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2974 00000100:00100000:3.0:1787739059.844245:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665636992:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739059.844304:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665637120:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739059.844326:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.844340:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.844351:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665637120 00000100:00100000:3.0:1787739059.844353:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665637120 00000100:00100000:3.0:1787739059.844375:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2975 00000100:00100000:3.0:1787739059.844380:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f76940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:6525:x1874579665637120:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739059.844449:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f76940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:6525:x1874579665637120:12345-0@lo:1000:mdt00_002.0 Request processed in 68us (122us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739059.844455:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2975 00000100:00100000:3.0:1787739059.844480:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665637120:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739059.844519:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665637248:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739059.844542:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.844558:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.844569:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665637248 00000100:00100000:3.0:1787739059.844572:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665637248 00000100:00100000:3.0:1787739059.844594:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2976 00000100:00100000:3.0:1787739059.844600:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f75e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:6525:x1874579665637248:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739059.844651:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f75e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:6525:x1874579665637248:12345-0@lo:1000:mdt00_002.0 Request processed in 51us (107us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.844657:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2976 00000100:00100000:3.0:1787739059.844680:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665637248:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739059.844719:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665637376:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739059.844742:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.844758:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.844769:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665637376 00000100:00100000:3.0:1787739059.844772:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665637376 00000100:00100000:3.0:1787739059.844791:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2977 00000100:00100000:3.0:1787739059.844797:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f77840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:6525:x1874579665637376:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739059.844846:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f77840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:6525:x1874579665637376:12345-0@lo:1000:mdt00_002.0 Request processed in 49us (103us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.844853:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2977 00000100:00100000:3.0:1787739059.844878:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665637376:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739059.844911:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665637504:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739059.844934:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.844950:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.844961:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665637504 00000100:00100000:3.0:1787739059.844963:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665637504 00000100:00100000:3.0:1787739059.844984:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2978 00000100:00100000:3.0:1787739059.844988:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f75a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:6525:x1874579665637504:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739059.845033:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f75a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:6525:x1874579665637504:12345-0@lo:1000:mdt00_002.0 Request processed in 44us (98us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.845052:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2978 00000100:00100000:3.0:1787739059.845074:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665637504:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739059.845102:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665637632:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739059.845123:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.845137:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.845148:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665637632 00000100:00100000:3.0:1787739059.845151:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665637632 00000100:00100000:3.0:1787739059.845170:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2979 00000100:00100000:3.0:1787739059.845176:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:6525:x1874579665637632:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739059.845217:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:6525:x1874579665637632:12345-0@lo:1000:mdt00_002.0 Request processed in 41us (92us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.845222:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2979 00000100:00100000:3.0:1787739059.845246:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665637632:0@lo:1000:mdt00_002.0 00000040:00080000:3.0:1787739059.846036:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].350, 2168 off532408 00000100:00100000:2.0:1787739059.846473:0:6508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529a40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665636608:0@lo:105: 00010000:00010000:2.0:1787739059.846491:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739059.846543:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c2580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2408:x1874579652458880:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 3641us (4401us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.846556:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1471 00000040:00080000:3.0:1787739059.850259:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].350, 2368 off581784 00000004:00080000:3.0:1787739059.850267:0:6525:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa073068ba000 with version 667 00000004:00080000:1.0:1787739059.850360:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 667 rpc_version 667 00000020:00100000:1.0:1787739059.850403:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739059.850455:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073043c3c00 x1874579665637888/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739059.850479:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c3c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665637888:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739059.850553:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739059.850595:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.850983:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665637888 00000100:00100000:3.0:1787739059.850992:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665637888 00000100:00100000:3.0:1787739059.851054:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2980 00000100:00100000:3.0:1787739059.851064:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+12:7467:x1874579665637888:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739059.852232:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+13:7467:x1874579665637888:12345-0@lo:1000:osp_up0-1.0 Request processed in 1167us (1676us total) trans 4294983367 rc 0/0 00000100:00100000:3.0:1787739059.852244:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2980 00000100:00100000:1.0:1787739059.852409:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c3c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665637888:0@lo:1000:osp_up0-1.0 00010000:00010000:3.0:1787739059.852657:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aca24 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x16f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.852737:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aca24 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x16f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.852750:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aca24 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x16f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.852771:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1aca2b lrc: 3/0,1 mode: --/PW res: [0x240000400:0xbf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.852779:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1aca2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.852786:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1aca2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.852881:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca32 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.852889:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca32 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.852899:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca32 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.852917:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665638016:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1787739059.852954:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.853032:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46c3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739059.855127:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665638016 00000100:00100000:0.0:1787739059.855141:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665638016 00000100:00100000:0.0:1787739059.855264:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2981 00000100:00100000:0.0:1787739059.855274:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c381680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+13:6525:x1874579665638016:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787739059.855296:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739059.855317:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1aca39 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aca32 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.855344:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1aca39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aca32 expref: 14 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.855358:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1aca39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aca32 expref: 14 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.855368:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbf:0x0].0x0 (ffffa07307064600) 00010000:00010000:0.0:1787739059.855371:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739059.855374:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609ac00, rc 0) 00000100:00100000:0.0:1787739059.855468:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c381680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+14:6525:x1874579665638016:12345-0@lo:101:mdt00_002.0 Request processed in 196us (2513us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739059.855475:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2981 00000100:00100000:3.0:1787739059.856085:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665638016:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739059.856104:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca32 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aca39 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.856115:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca32 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aca39 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739059.856140:0:6525:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000cccf7c24 reply state 00000000c285cfff transno 4294968356 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1aca2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.856153:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1aca2b lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.856160:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xbf:0x0].0x0 (ffffa074039f2700) 00010000:00010000:3.0:1787739059.856161:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739059.856165:0:6525:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787739059.856170:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca32 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aca39 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.856176:0:6525:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca32 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aca39 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739059.856186:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xbf:0x0] 00000004:00010000:3.0:1787739059.856192:0:6525:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000cccf7c24 reply state 00000000c285cfff transno 4294968356 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aca24 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x16f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.856199:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aca24 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x16f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.856204:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16f:0x0].0x0 (ffffa074039f2a00) 00010000:00010000:3.0:1787739059.856206:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739059.856208:0:6525:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:3.0:1787739059.856213:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x16f:0x0]: returning size 4096 00000004:00080000:3.0:1787739059.856221:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739059.856229:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1aca08 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aca0f expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.856236:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1aca08 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aca0f expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.856243:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1aca08 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aca0f expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.856277:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739059.856281:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739059.856285:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aca16 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aca1d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.856293:0:6525:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aca16 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aca1d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.856370:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17546:x1874579652458752:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 33159us (33412us total) trans 4294968356 rc 0/0 00000100:00100000:3.0:1787739059.856378:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23366 00010000:00010000:1.0:1787739059.856538:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739059.856545:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1aca08 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aca0f expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.856553:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739059.856559:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1aca08 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aca0f expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739059.856567:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1aca08 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aca0f expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.856607:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1aca08 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aca0f expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.856620:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739059.856648:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1aca08 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aca0f expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.856656:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1aca08 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aca0f expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.856681:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739059.856683:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739059.856684:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739059.856693:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787739059.856705:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f712c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665638144:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739059.856748:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739059.856785:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665638144 00000100:00100000:1.0:1787739059.856790:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665638144 00000100:00100000:1.0:1787739059.856837:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1280 00000100:00100000:1.0:1787739059.856844:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415701e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+14:3653:x1874579665638144:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739059.856856:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739059.856870:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1aca0f lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aca08 expref: 14 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.856882:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235200) 00010000:00010000:1.0:1787739059.856888:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739059.856918:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415701e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+13:3653:x1874579665638144:12345-0@lo:103:ldlm_bl.0 Request processed in 75us (169us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.856924:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1280 00000100:00100000:1.0:1787739059.856953:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f712c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665638144:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739059.881222:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739059.881370:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652459136 00000100:00100000:2.0:1787739059.881380:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652459136 00000100:00100000:2.0:1787739059.881459:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23367 00000100:00100000:2.0:1787739059.881470:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415701680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17547:x1874579652459136:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:2.0:1787739059.881628:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415701680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17547:x1874579652459136:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 159us (415us total) trans 0 rc -17/-17 00000100:00100000:2.0:1787739059.881635:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23367 00000100:00100000:2.0:1787739059.895601:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739059.895709:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652459264 00000100:00100000:0.0:1787739059.895718:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652459264 00000100:00100000:0.0:1787739059.895778:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23368 00000100:00100000:0.0:1787739059.895788:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17548:x1874579652459264:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739059.895810:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739059.895832:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1aca40 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094e58 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.895907:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1aca47 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.895921:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1aca47 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.895933:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1aca47 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.895945:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aca4e lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.895955:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aca1d lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aca16 expref: 12 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.895969:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aca4e lrc: 4/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.895999:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aca1d lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1aca16 expref: 12 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739059.896013:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561cb40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665638272:0@lo:104: 00000100:00100000:0.0:1787739059.896076:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739059.896117:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074104046c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739059.896153:0:6506:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:2.0:1787739059.896168:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665638272 00000100:00100000:2.0:1787739059.896172:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665638272 00000100:00100000:2.0:1787739059.896211:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1126 00000100:00100000:2.0:1787739059.896219:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f805450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:13793:x1874579665638272:12345-0@lo:104: 00010000:00010000:2.0:1787739059.896231:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aca16 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aca1d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.896288:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739059.896294:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f805450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:13793:x1874579665638272:12345-0@lo:104: Request processed in 76us (218us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.896301:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1126 00010000:00010000:1.0:1787739059.896488:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739059.896495:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aca16 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aca1d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.896504:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739059.896509:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aca16 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1aca1d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739059.896518:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aca16 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1aca1d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.896541:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aca16 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aca1d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.896546:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1787739059.896584:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561cb40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665638272:0@lo:104: 00000100:00100000:2.0:1787739059.896591:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415700b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665638400:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739059.896602:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aca4e lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739059.896617:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.896658:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665638400 00000100:00100000:3.0:1787739059.896663:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665638400 00010000:00010000:1.0:1787739059.896664:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aca16 lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aca1d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.896672:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1aca16 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aca1d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.896681:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aca1d lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1aca16 expref: 13 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.896688:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739059.896690:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739059.896691:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739059.896730:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1472 00000100:00100000:3.0:1787739059.896738:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+13:3653:x1874579665638400:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739059.896748:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:3.0:1787739059.903697:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983367 is committed 00000001:00080000:3.0:1787739059.903725:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983366 is committed 00000020:00080000:3.0:1787739059.903734:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 364 batchid updated 00000020:00080000:3.0:1787739059.903756:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa073068ba3c0 refcount 2 committed 0 result 0 batchid 365 00010000:00010000:1.0:1787739059.903761:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ac94b lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1787739059.903762:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa074356c7400 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa073068ba9c0 00000020:00080000:3.0:1787739059.903765:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].362 00000020:00080000:3.0:1787739059.903768:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa074356c7100 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306aac600 00000020:00080000:3.0:1787739059.903771:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].362 00010000:00010000:1.0:1787739059.903773:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ac94b lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.903781:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ac94b lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.903788:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739059.903792:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ac94b lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.903800:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ac94b lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.903816:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xb994c874 (ffffa074039f2e00) 00010000:00010000:1.0:1787739059.903819:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739059.903825:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ac94b lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.903831:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ac94b lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.903845:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ac952 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x150e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.903852:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ac952 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x150e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.903865:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ac952 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x150e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.903870:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739059.903874:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ac952 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x150e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.903884:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ac952 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x150e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.903893:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x150e:0x0].0x0 (ffffa074039f2f00) 00010000:00010000:1.0:1787739059.903896:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739059.903899:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ac952 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x150e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.903904:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ac952 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x150e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.903913:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aca1d lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1aca16 expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.903922:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac959 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.903929:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x872f39a (ffffa074039f2400) 00010000:00010000:1.0:1787739059.903931:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac959 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.903937:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aca4e lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.903940:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac959 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.903953:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739059.903955:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aca4e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.903957:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac959 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.903973:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aca4e lrc: 4/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.903974:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac959 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.903995:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc1:0x0].0x0 (ffffa074039f2000) 00010000:00010000:1.0:1787739059.903997:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739059.904000:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac959 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.904010:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ac959 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.904018:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739059.904028:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac9ad lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.904045:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac9ad lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.904052:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac9ad lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.904058:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739059.904060:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac9ad lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.904066:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac9ad lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.904074:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xb994c874 (ffffa074039f2e00) 00010000:00010000:1.0:1787739059.904087:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739059.904091:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac9ad lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.904096:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ac9ad lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xb994c874 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739059.904106:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+6:3653:x1874579665638400:12345-0@lo:103:ldlm_bl.0 Request processed in 7366us (7485us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739059.904116:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1472 00000020:00080000:2.0:1787739059.904159:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 365 committed batchid 364 00000004:00080000:2.0:1787739059.904266:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa073068876c0 with version 693 00000020:00080000:2.0:1787739059.904282:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 364 cancel update log [0x240000401:0x3:0x0].361: rc = 0 00000020:00080000:2.0:1787739059.904335:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 364 cancel update log [0x200000400:0x3:0x0].361: rc = 0 00000020:00080000:2.0:1787739059.904344:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 364 00000100:00100000:2.0:1787739059.904383:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415700b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665638400:0@lo:103:ldlm_bl.0 00000004:00080000:1.0:1787739059.904678:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 693 rpc_version 693 00000020:00100000:1.0:1787739059.904710:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739059.904719:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07415701a40 x1874579665638656/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739059.904739:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665638656:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739059.904784:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739059.904815:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739059.904972:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665638656 00000100:00100000:0.0:1787739059.904987:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665638656 00000100:00100000:0.0:1787739059.905035:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2982 00000100:00100000:0.0:1787739059.905053:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415700b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665638656:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739059.905187:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415700b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665638656:12345-0@lo:1000:osp_up1-0.0 Request processed in 134us (401us total) trans 4294968357 rc 0/0 00000100:00100000:0.0:1787739059.905194:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2982 00000100:00100000:2.0:1787739059.905400:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665638656:0@lo:1000:osp_up1-0.0 00010000:00010000:3.0:1787739059.905497:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aca4e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.905507:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aca4e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.905594:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c380f00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665638784:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739059.905626:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.905658:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068ba600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739059.905683:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665638784 00000100:00100000:2.0:1787739059.905688:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665638784 00000100:00100000:2.0:1787739059.905725:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2983 00000100:00100000:2.0:1787739059.905732:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665638784:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739059.905799:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665638784:12345-0@lo:1000:mdt00_004.0 Request processed in 68us (172us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.905806:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2983 00000100:00100000:3.0:1787739059.905837:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c380f00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665638784:0@lo:1000:mdt00_004.0 00010000:00010000:3.0:1787739059.905898:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aca55 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x16f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.905921:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aca55 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.905932:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aca55 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739059.905953:0:13793:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aca55 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.905965:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1aca47 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.905971:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1aca47 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.905977:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1aca47 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.905988:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1aca47 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.905993:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739059.905996:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1aca47 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.906002:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1aca47 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.906012:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07307064500) 00010000:00010000:3.0:1787739059.906015:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739059.906018:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1aca47 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.906025:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1aca47 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.906037:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aca4e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.906054:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aca4e lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.906059:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aca4e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.906064:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ba00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739059.906067:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aca4e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.906076:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aca4e lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.906083:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x872f39a (ffffa074039f2400) 00010000:00010000:3.0:1787739059.906085:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739059.906087:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aca4e lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.906092:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1aca4e lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.906127:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1aca40 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e58 expref: 15 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.906145:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aca55 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x16f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e58 expref: 14 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.906152:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16f:0x0].0x0 (ffffa074039f2b00) 00010000:00010000:3.0:1787739059.906155:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739059.906157:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abe00, rc 0) 00000100:00100000:3.0:1787739059.906206:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17548:x1874579652459264:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 10419us (10613us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.906213:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23368 00000100:00100000:0.0:1787739059.911692:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739059.911745:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652459392 00000100:00100000:3.0:1787739059.911756:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652459392 00000100:00100000:3.0:1787739059.911822:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23369 00000100:00100000:3.0:1787739059.911832:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17548:x1874579652459392:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739059.911866:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739059.911896:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aca5c lrc: 2/0,0 mode: --/CR res: [0x240000402:0x16f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094e5f expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.911971:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1aca63 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x16f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.911984:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1aca63 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.911993:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1aca63 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x16f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739059.912098:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x16f:0x0]: returning size 4096 00000004:00200000:3.0:1787739059.912103:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739059.912139:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aca5c lrc: 0/0,0 mode: --/CR res: [0x240000402:0x16f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e5f expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.912158:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1aca63 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x16f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e5f expref: 10 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.912167:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16f:0x0].0x0 (ffffa074039f2a00) 00010000:00010000:3.0:1787739059.912169:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739059.912173:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa800, rc 0) 00000100:00100000:3.0:1787739059.912399:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17548:x1874579652459392:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 568us (715us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.912406:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23369 00000100:00100000:0.0:1787739059.923371:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739059.923764:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652459520 00000100:00100000:1.0:1787739059.923777:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652459520 00000100:00100000:1.0:1787739059.923842:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23370 00000100:00100000:1.0:1787739059.923854:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17548:x1874579652459520:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739059.923876:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739059.923900:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aca6a lrc: 2/0,0 mode: --/CR res: [0x240000400:0xbf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094e66 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.923970:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aca71 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xbf:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.923985:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aca71 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.923999:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aca71 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739059.924069:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xbf:0x0]: returning size 4096 00000004:00200000:1.0:1787739059.924075:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739059.924113:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1aca6a lrc: 0/0,0 mode: --/CR res: [0x240000400:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e66 expref: 12 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.924130:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aca71 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e66 expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.924144:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xbf:0x0].0x0 (ffffa074039f2700) 00010000:00010000:1.0:1787739059.924147:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739059.924151:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c000, rc 0) 00000100:00100000:1.0:1787739059.924214:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17548:x1874579652459520:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 363us (853us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.924222:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23370 00000100:00100000:0.0:1787739059.929632:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739059.929715:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652459648 00000100:00100000:3.0:1787739059.929725:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652459648 00000100:00100000:3.0:1787739059.929775:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23371 00000100:00100000:3.0:1787739059.929786:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17548:x1874579652459648:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739059.929805:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739059.929823:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aca78 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xbf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094e6d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.929878:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aca7f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.929892:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1aca39 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aca32 expref: 5 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.929901:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aca7f lrc: 4/1,0 mode: --/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.929931:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1aca39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1aca32 expref: 5 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.929946:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c380f00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665638912:0@lo:104: 00000100:00100000:3.0:1787739059.929991:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739059.930019:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665638912 00000100:00100000:2.0:1787739059.930025:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665638912 00000100:00100000:3.0:1787739059.930035:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739059.930074:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1127 00000100:00100000:2.0:1787739059.930082:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:13793:x1874579665638912:12345-0@lo:104: 00010000:00010000:2.0:1787739059.930095:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca32 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aca39 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.930158:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739059.930164:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:13793:x1874579665638912:12345-0@lo:104: Request processed in 84us (173us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.930172:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1127 00000100:00100000:3.0:1787739059.930209:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c380f00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665638912:0@lo:104: 00010000:00010000:3.0:1787739059.930229:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aca7f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.931680:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739059.931691:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca32 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aca39 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.931703:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739059.931708:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca32 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1aca39 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739059.931719:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca32 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1aca39 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.931758:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca32 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aca39 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.931766:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739059.931807:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca32 lrc: 1/0,0 mode: --/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aca39 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.931814:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca32 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aca39 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.931838:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739059.931840:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739059.931842:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739059.932085:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787739059.932104:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f73c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665639040:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739059.932151:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739059.932333:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665639040 00000100:00100000:1.0:1787739059.932340:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665639040 00010000:00010000:1.0:1787739059.932368:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1aca39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1aca32 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739059.932406:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1473 00000100:00100000:1.0:1787739059.932448:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+6:3651:x1874579665639040:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739059.932461:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00080000:0.0:1787739059.934012:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 365 batchid updated 00000020:00080000:2.0:1787739059.934090:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 365 00000004:00080000:2.0:1787739059.934146:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306887600 with version 694 00000020:00080000:2.0:1787739059.934156:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 365 cancel update log [0x240000401:0x3:0x0].362: rc = 0 00010000:00010000:1.0:1787739059.934167:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1aca39 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1aca32 expref: 6 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.934186:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbf:0x0].0x0 (ffffa07307064600) 00010000:00010000:1.0:1787739059.934192:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aca7f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.934203:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aca7f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.934210:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aca7f lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739059.934212:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 365 cancel update log [0x200000400:0x3:0x0].362: rc = 0 00000020:00080000:2.0:1787739059.934226:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 365 00000004:00080000:2.0:1787739059.934236:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 694 rpc_version 694 00010000:00010000:1.0:1787739059.934245:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000020:00100000:2.0:1787739059.934263:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739059.934272:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0730252a940 x1874579665639296/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739059.934290:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252a940 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665639296:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739059.934347:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739059.934366:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665639040:12345-0@lo:103:ldlm_bl.0 Request processed in 1950us (2211us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.934376:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739059.934377:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1473 00000100:00100000:2.0:1787739059.934396:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665639296 00000100:00100000:2.0:1787739059.934403:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665639296 00000100:00100000:2.0:1787739059.934456:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2984 00000100:00100000:2.0:1787739059.934463:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c2940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665639296:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739059.934558:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c2940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665639296:12345-0@lo:1000:osp_up1-0.0 Request processed in 94us (210us total) trans 4294968358 rc 0/0 00000100:00100000:2.0:1787739059.934564:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2984 00000100:00100000:2.0:1787739059.934597:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252a940 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665639296:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739059.935035:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f73c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665639040:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739059.935082:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aca7f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.935090:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aca7f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739059.935139:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xbf:0x0]: returning size 4096 00000004:00200000:1.0:1787739059.935145:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739059.935188:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aca78 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e6d expref: 16 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.935205:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aca7f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e6d expref: 15 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.935213:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbf:0x0].0x0 (ffffa07307064600) 00010000:00010000:1.0:1787739059.935216:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739059.935220:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa200, rc 0) 00000100:00100000:1.0:1787739059.935269:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17548:x1874579652459648:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 5485us (5644us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.935278:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23371 00000100:00100000:0.0:1787739059.950271:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739059.950967:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652460032 00000100:00100000:3.0:1787739059.950978:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652460032 00000100:00100000:3.0:1787739059.951051:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23372 00000100:00100000:3.0:1787739059.951062:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17548:x1874579652460032:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:3.0:1787739059.951112:0:6525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739059.951134:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1aca63 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x16f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e5f expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.951147:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16f:0x0].0x0 (ffffa074039f2a00) 00010000:00010000:3.0:1787739059.951150:0:6525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739059.951188:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1aca86 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.951193:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1aca86 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.951199:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1aca86 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.951213:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c380f00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665639424:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1787739059.951252:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.951290:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739059.951464:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665639424 00000100:00100000:2.0:1787739059.951468:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665639424 00000100:00100000:2.0:1787739059.951506:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2985 00000100:00100000:2.0:1787739059.951512:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356afc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+5:6525:x1874579665639424:12345-0@lo:101:mdt00_002.0 00000100:00100000:0.0:1787739059.951518:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00010000:00010000:2.0:1787739059.951521:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739059.951536:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1aca8d lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aca86 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.951558:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1aca8d lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aca86 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.951574:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1aca8d lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aca86 expref: 6 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.951582:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78f00) 00010000:00010000:2.0:1787739059.951585:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739059.951587:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d200, rc 0) 00000100:00100000:2.0:1787739059.951624:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356afc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:6525:x1874579665639424:12345-0@lo:101:mdt00_002.0 Request processed in 112us (370us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.951631:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2985 00000100:00100000:3.0:1787739059.951701:0:6525:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983367 gen 2 00000004:00080000:3.0:1787739059.951706:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983360 committed transno 4294983367 00000020:00080000:3.0:1787739059.951833:0:6525:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa0743c413c00 refcount 2 committed 0 result 0 batchid 356 00000020:00080000:3.0:1787739059.951847:0:6525:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730415fc00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa0743c413780 00000020:00080000:3.0:1787739059.951851:0:6525:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].349 00000020:00080000:3.0:1787739059.951854:0:6525:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c691e00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304277e00 00000020:00080000:3.0:1787739059.951857:0:6525:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].349 00000020:00080000:0.0:1787739059.951888:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 356 committed batchid 354 00000020:00080000:0.0:1787739059.951950:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 354 00000004:00080000:3.0:1787739059.951988:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983362 committed transno 4294983367 00000004:00080000:3.0:1787739059.951997:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983363 committed transno 4294983367 00000004:00080000:3.0:1787739059.952003:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983364 committed transno 4294983367 00000004:00080000:3.0:1787739059.952010:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983365 committed transno 4294983367 00000004:00080000:3.0:1787739059.952020:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983367 committed transno 4294983367 00000100:00100000:3.0:1787739059.952051:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c380f00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665639424:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739059.952068:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1aca86 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aca8d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.952078:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1aca86 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aca8d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739059.952092:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739059.952114:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca94 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.952122:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca94 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.952129:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca94 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.952138:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665639552:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1787739059.952157:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.952177:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739059.952189:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665639552 00000100:00100000:2.0:1787739059.952193:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665639552 00000100:00100000:2.0:1787739059.952228:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2986 00000100:00100000:2.0:1787739059.952237:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356acb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:6525:x1874579665639552:12345-0@lo:101:mdt00_002.0 00000100:00100000:1.0:1787739059.952246:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652459776 00010000:00010000:2.0:1787739059.952247:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1787739059.952253:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652459776 00010000:00010000:2.0:1787739059.952260:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1aca9b lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aca94 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.952278:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1aca9b lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aca94 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.952288:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1aca9b lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aca94 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739059.952295:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x872f39a (ffffa07303c78900) 00010000:00010000:2.0:1787739059.952298:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739059.952301:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d600, rc 0) 00000100:00100000:3.0:1787739059.952342:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665639552:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739059.952353:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca94 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aca9b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.952384:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca94 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aca9b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739059.952401:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:3.0:1787739059.952448:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665639680:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739059.952469:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356acb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6525:x1874579665639552:12345-0@lo:101:mdt00_002.0 Request processed in 233us (309us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.952474:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739059.952476:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2986 00000100:00100000:3.0:1787739059.952497:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739059.952526:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665639680 00000100:00100000:2.0:1787739059.952530:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665639680 00000100:00100000:2.0:1787739059.952550:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2987 00000100:00100000:2.0:1787739059.952557:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ada40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6525:x1874579665639680:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739059.952612:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ada40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6525:x1874579665639680:12345-0@lo:1000:mdt00_002.0 Request processed in 55us (136us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.952618:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2987 00000100:00100000:1.0:1787739059.953110:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1281 00000100:00100000:1.0:1787739059.953137:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561e940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2407:x1874579652459776:12345-192.168.203.15@tcp:103:rmdir.0 00000100:00100000:3.0:1787739059.953148:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665639680:0@lo:1000:mdt00_002.0 00010000:00010000:1.0:1787739059.953150:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739059.953170:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aca55 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x16f:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e58 expref: 15 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.953189:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16f:0x0].0x0 (ffffa074039f2b00) 00000100:00100000:3.0:1787739059.953193:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665639808:0@lo:1000:mdt00_002.0 00010000:00010000:1.0:1787739059.953200:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739059.953231:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.953262:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e300 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.953277:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665639808 00000100:00100000:3.0:1787739059.953281:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665639808 00000100:00100000:3.0:1787739059.953494:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2988 00000100:00100000:3.0:1787739059.953498:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356af0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6525:x1874579665639808:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739059.953566:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356af0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6525:x1874579665639808:12345-0@lo:1000:mdt00_002.0 Request processed in 66us (335us total) trans 0 rc 1/1 00000100:00100000:3.0:1787739059.953573:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2988 00000100:00100000:3.0:1787739059.953597:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665639808:0@lo:1000:mdt00_002.0 00010000:00010000:3.0:1787739059.953640:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acaa2 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.953647:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acaa2 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.953655:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acaa2 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.953666:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665639936:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1787739059.953700:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739059.953909:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561e940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2407:x1874579652459776:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 786us (2388us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.953917:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1281 00000100:00100000:3.0:1787739059.954162:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.954176:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665639936 00000100:00100000:3.0:1787739059.954180:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665639936 00000100:00100000:3.0:1787739059.954204:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2989 00000100:00100000:3.0:1787739059.954209:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ae940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6525:x1874579665639936:12345-0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739059.954217:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739059.954230:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1acaa9 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1acaa2 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.954248:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1acaa9 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acaa2 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.954260:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1acaa9 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acaa2 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.954267:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16f:0x0].0x0 (ffffa074039f2000) 00010000:00010000:3.0:1787739059.954270:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739059.954273:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abc00, rc 0) 00000100:00100000:3.0:1787739059.954307:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ae940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6525:x1874579665639936:12345-0@lo:101:mdt00_002.0 Request processed in 97us (605us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.954313:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2989 00000100:00100000:3.0:1787739059.954354:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665639936:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739059.954367:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acaa2 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acaa9 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.954376:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acaa2 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acaa9 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.954401:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acab0 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.954409:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acab0 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.954433:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acab0 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.954449:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1acab7 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xbf:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.954458:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aca71 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e66 expref: 10 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.954465:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1acab7 lrc: 4/0,1 mode: --/EX res: [0x240000400:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.954485:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aca71 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e66 expref: 10 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.954499:0:6525:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aca71 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e66 expref: 10 pid: 6523 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.954510:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665640064:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739059.955139:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.960228:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665640064:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739059.960257:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1acab7 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.960925:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739059.961123:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652460160 00000100:00100000:0.0:1787739059.961130:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652460160 00010000:00010000:0.0:1787739059.961166:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aca71 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e66 expref: 11 pid: 6523 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739059.961204:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1474 00000100:00100000:0.0:1787739059.961214:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ae580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2409:x1874579652460160:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739059.961232:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739059.961239:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aca71 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e66 expref: 11 pid: 6523 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.961251:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aca71 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e66 expref: 11 pid: 6523 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.961275:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1aca71 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e66 expref: 11 pid: 6523 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.961289:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xbf:0x0].0x0 (ffffa074039f2700) 00010000:00010000:0.0:1787739059.961294:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1acab7 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.961304:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1acab7 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.961311:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1acab7 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.961339:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739059.961390:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ae580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2409:x1874579652460160:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 178us (467us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739059.961398:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1474 00010000:00010000:3.0:1787739059.961990:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1acab7 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.962000:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1acab7 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.962204:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1acabe lrc: 3/0,1 mode: --/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.962212:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1acabe lrc: 3/0,1 mode: --/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.962222:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1acabe lrc: 3/0,1 mode: --/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.962241:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ac000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665640192:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1787739059.962283:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.962321:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739059.962365:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665640192 00000100:00100000:0.0:1787739059.962370:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665640192 00000100:00100000:0.0:1787739059.962408:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2990 00000100:00100000:0.0:1787739059.962437:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6525:x1874579665640192:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787739059.962450:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739059.962465:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1acac5 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1acabe expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.962486:0:18535:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aca7f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e6d expref: 13 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.962496:0:18535:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1acac5 lrc: 4/0,0 mode: --/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acabe expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.962517:0:18535:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aca7f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e6d expref: 13 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739059.962529:0:18535:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aca7f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e6d expref: 13 pid: 13793 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739059.962541:0:18535:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304262940 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:18535:1874579665640320:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739059.962586:0:18535:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074104046c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739059.968727:0:18535:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304262940 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:18535:1874579665640320:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739059.968766:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1acac5 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acabe expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.968771:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00010000:00010000:1.0:1787739059.968779:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbf:0x0].0x0 (ffffa07307064600) 00010000:00010000:1.0:1787739059.968782:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739059.968785:0:18535:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1acac5 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acabe expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.968795:0:18535:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1acac5 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acabe expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.968802:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652460288 00010000:00010000:1.0:1787739059.968802:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a000, rc 0) 00000100:00100000:3.0:1787739059.968808:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652460288 00000100:00100000:1.0:1787739059.968856:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6525:x1874579665640192:12345-0@lo:101:mdt00_002.0 Request processed in 6421us (6572us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.968864:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2990 00010000:00010000:3.0:1787739059.969206:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aca7f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e6d expref: 14 pid: 13793 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.969241:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1475 00000100:00100000:3.0:1787739059.969250:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074184c0050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652460288:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739059.969264:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739059.969271:0:6510:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aca7f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e6d expref: 14 pid: 13793 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.969282:0:6510:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aca7f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e6d expref: 14 pid: 13793 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.969301:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aca7f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xbf:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e6d expref: 14 pid: 13793 timeout: 746 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.969312:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbf:0x0].0x0 (ffffa07307064600) 00010000:00010000:3.0:1787739059.969316:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1acac5 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acabe expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.969325:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1acac5 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1acabe expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.969334:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1acac5 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1acabe expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.969362:0:6510:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1acac5 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acabe expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.969373:0:6510:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ad680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6510:1874579665640448:0@lo:105: 00000100:00100000:3.0:1787739059.969410:0:6510:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.969472:0:6510:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e300 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.969487:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665640448 00000100:00100000:3.0:1787739059.969491:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665640448 00000100:00100000:3.0:1787739059.969519:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1128 00000100:00100000:3.0:1787739059.969526:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6510:x1874579665640448:12345-0@lo:105: 00010000:00010000:3.0:1787739059.969536:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1acabe lrc: 4/0,1 mode: --/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acac5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.969545:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1acabe lrc: 4/0,1 mode: --/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acac5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.969554:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1acabe lrc: 4/0,1 mode: --/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acac5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.969562:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1acabe lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1acac5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.969568:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1acabe lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1acac5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.969576:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1acabe lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1acac5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.969587:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069aa800) 00000100:00100000:3.0:1787739059.969617:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6510:x1874579665640448:12345-0@lo:105: Request processed in 92us (206us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.969624:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1128 00000100:00100000:3.0:1787739059.969655:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ac000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665640192:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739059.969668:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1acabe lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acac5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739059.971197:0:6510:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ad680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6510:1874579665640448:0@lo:105: 00010000:00010000:2.0:1787739059.971214:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739059.971265:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074184c0050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:2409:x1874579652460288:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 2014us (2492us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739059.971273:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1475 00000100:00100000:3.0:1787739059.971343:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ac000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665640576:0@lo:403:mdt00_002.0 00000100:00100000:3.0:1787739059.971394:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.971435:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.971451:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665640576 00000100:00100000:3.0:1787739059.971456:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665640576 00000100:00100000:3.0:1787739059.971492:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2991 00000100:00100000:3.0:1787739059.971499:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356af840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6525:x1874579665640576:12345-0@lo:403:mdt00_002.0 00000100:00100000:3.0:1787739059.977623:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356af840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6525:x1874579665640576:12345-0@lo:403:mdt00_002.0 Request processed in 6117us (6220us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.977636:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2991 00000100:00100000:0.0:1787739059.978115:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ac000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665640576:0@lo:403:mdt00_002.0 00000040:00080000:0.0:1787739059.978966:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].351, 664 off533072 00000040:00080000:0.0:1787739059.979158:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].351, 744 off582528 00000004:00080000:0.0:1787739059.979164:0:6525:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07410404300 with version 668 00000004:00080000:2.0:1787739059.979556:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 668 rpc_version 668 00000020:00100000:2.0:1787739059.979588:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739059.979598:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073043c2580 x1874579665640832/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1808/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739059.979620:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c2580 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665640832:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739059.979695:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739059.979763:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739059.980187:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665640832 00000100:00100000:3.0:1787739059.980194:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665640832 00000100:00100000:3.0:1787739059.980249:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2992 00000100:00100000:3.0:1787739059.980257:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665640832:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739059.983335:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c3c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665640832:12345-0@lo:1000:osp_up0-1.0 Request processed in 3073us (3645us total) trans 4294983368 rc 0/0 00000100:00100000:3.0:1787739059.983350:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2992 00000100:00100000:3.0:1787739059.983520:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c2580 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665640832:0@lo:1000:osp_up0-1.0 00000004:00200000:3.0:1787739059.983551:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x16f:0x0]: returning size 4096 00010000:00010000:3.0:1787739059.983571:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1acab7 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.983580:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1acab7 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.983587:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1acab7 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.983594:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739059.983599:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1acab7 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.983606:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1acab7 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.983620:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xbf:0x0].0x0 (ffffa074039f2700) 00010000:00010000:3.0:1787739059.983623:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739059.983628:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1acab7 lrc: 2/0,0 mode: --/EX res: [0x240000400:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.983634:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1acab7 lrc: 0/0,0 mode: --/EX res: [0x240000400:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.983646:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1acabe lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acac5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.983653:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1acabe lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acac5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.983660:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1acabe lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acac5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.983681:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1787739059.983687:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739059.983694:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1acabe lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1acac5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739059.983695:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xbf:0x0] 00010000:00010000:3.0:1787739059.983701:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acab0 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.983701:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739059.983705:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1acabe lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1acac5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.983707:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acab0 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739059.983713:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1acabe lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1acac5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.983742:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1acabe lrc: 2/0,0 mode: --/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acac5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.983748:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739059.983766:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1acabe lrc: 1/0,0 mode: --/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acac5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.983773:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa800/0x7ee4453e4a1acabe lrc: 0/0,0 mode: --/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acac5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.983788:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739059.983789:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739059.983791:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739059.983803:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f73c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665640960:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739059.983845:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787739059.983883:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acab0 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739059.983883:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665640960 00000100:00100000:1.0:1787739059.983888:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665640960 00010000:00010000:3.0:1787739059.983889:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739059.983892:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acab0 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.983897:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acab0 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.983905:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16f:0x0].0x0 (ffffa074039f2a00) 00010000:00010000:3.0:1787739059.983907:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739059.983910:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acab0 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.983916:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acab0 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739059.983922:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1282 00000100:00100000:1.0:1787739059.983929:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f712c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+10:3651:x1874579665640960:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739059.983941:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739059.983955:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1acac5 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xbf:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1acabe expref: 10 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739059.983971:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xbf:0x0].0x0 (ffffa07307064600) 00010000:00010000:1.0:1787739059.983979:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739059.983990:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acaa2 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acaa9 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.983998:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acaa2 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acaa9 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739059.984007:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f712c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3651:x1874579665640960:12345-0@lo:103:ldlm_bl.0 Request processed in 79us (163us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739059.984013:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1282 00000100:00100000:1.0:1787739059.984050:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f73c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665640960:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739059.984077:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acaa2 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acaa9 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.984095:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739059.984105:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739059.984109:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1aca86 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aca8d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.984120:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1aca86 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aca8d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.984127:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1aca86 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aca8d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.984144:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:3.0:1787739059.984147:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739059.984152:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca94 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aca9b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.984158:0:6525:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca94 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aca9b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739059.984220:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17548:x1874579652460032:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 33158us (33956us total) trans 4294968359 rc 0/0 00000100:00100000:3.0:1787739059.984227:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23372 00010000:00010000:3.0:1787739059.984250:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787739059.984255:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acaa2 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1acaa9 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.984262:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739059.984266:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acaa2 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1acaa9 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739059.984272:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acaa2 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1acaa9 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.984290:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acaa2 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acaa9 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.984296:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739059.984319:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acaa2 lrc: 1/0,0 mode: --/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acaa9 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.984337:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acaa2 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acaa9 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.984350:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739059.984353:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1aca86 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aca8d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.984359:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aaa00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739059.984363:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1aca86 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aca8d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739059.984370:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1aca86 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aca8d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.984996:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1aca86 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aca8d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.985185:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739059.985198:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1aca86 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aca8d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.985211:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aaa00/0x7ee4453e4a1aca86 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aca8d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.985226:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739059.985227:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739059.985229:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739059.985235:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739059.985246:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356acf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665641088:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739059.985292:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739059.985647:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787739059.985653:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665641216:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739059.985677:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739059.987909:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665641088 00000100:00100000:0.0:1787739059.987980:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665641088 00000100:00100000:0.0:1787739059.988143:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665641216 00000100:00100000:0.0:1787739059.988147:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665641216 00000100:00100000:3.0:1787739059.988669:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1283 00000100:00100000:3.0:1787739059.988695:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ade00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+10:3652:x1874579665641088:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739059.988709:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739059.989117:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1acaa9 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x16f:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1acaa2 expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.989134:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16f:0x0].0x0 (ffffa074039f2000) 00010000:00010000:3.0:1787739059.989144:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739059.989227:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ade00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+9:3652:x1874579665641088:12345-0@lo:103:ldlm_bl.0 Request processed in 549us (3935us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.989235:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1283 00000100:00100000:3.0:1787739059.989259:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1284 00000100:00100000:3.0:1787739059.989265:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+8:3652:x1874579665641216:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739059.989275:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739059.989286:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1aca8d lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aca86 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739059.989297:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78f00) 00010000:00010000:3.0:1787739059.989305:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739059.989327:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+7:3652:x1874579665641216:12345-0@lo:103:ldlm_bl.0 Request processed in 62us (3649us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739059.989333:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1284 00000100:00100000:3.0:1787739059.989390:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ac780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665641216:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739059.989402:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356acf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665641088:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739060.033554:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739060.033595:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652460416 00000100:00100000:1.0:1787739060.033605:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652460416 00000100:00100000:1.0:1787739060.033656:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23373 00000100:00100000:1.0:1787739060.033667:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f73c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17549:x1874579652460416:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739060.033688:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739060.033708:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1acacc lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094e74 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.033772:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1acad3 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.033785:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1acad3 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.033797:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1acad3 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.033811:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acada lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.033817:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acada lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.033824:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acada lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.033867:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1acad3 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.033874:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1acad3 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.033881:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1acad3 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.033888:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1acad3 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.033894:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739060.033898:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1acad3 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.033905:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1acad3 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.033916:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9800) 00010000:00010000:1.0:1787739060.033919:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739060.033935:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1acad3 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.033942:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1acad3 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.033953:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acada lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.033959:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acada lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.033965:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acada lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.033970:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739060.033973:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acada lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.033978:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acada lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.033984:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xcefb0e88 (ffffa07433235200) 00010000:00010000:1.0:1787739060.033987:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739060.033989:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acada lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.033995:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acada lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.034029:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1acacc lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e74 expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.034053:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9800) 00010000:00010000:1.0:1787739060.034056:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739060.034060:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1acacc lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e74 expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.034073:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d800, rc 0) 00000100:00100000:1.0:1787739060.034138:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f73c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17549:x1874579652460416:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 473us (600us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739060.034147:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23373 00000100:00100000:0.0:1787739060.042233:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739060.042314:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652460544 00000100:00100000:1.0:1787739060.042323:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652460544 00000100:00100000:1.0:1787739060.042561:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23374 00000100:00100000:1.0:1787739060.042573:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17549:x1874579652460544:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739060.042592:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739060.042619:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1acae1 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094e7b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.042687:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acae8 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.042701:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acae8 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.042712:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acae8 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739060.042764:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:1.0:1787739060.042769:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739060.042805:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1acae1 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e7b expref: 15 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.042820:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acae8 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e7b expref: 14 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.042828:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235900) 00010000:00010000:1.0:1787739060.042832:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739060.042835:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1de00, rc 0) 00000100:00100000:1.0:1787739060.042900:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17549:x1874579652460544:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 330us (678us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739060.042907:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23374 00000100:00100000:2.0:1787739060.050928:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739060.050968:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652460800 00000100:00100000:3.0:1787739060.050978:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652460800 00000100:00100000:3.0:1787739060.051228:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23375 00000100:00100000:3.0:1787739060.051240:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17549:x1874579652460800:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:3.0:1787739060.051352:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356acf00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665641344:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739060.051404:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739060.052012:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.053949:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665641344 00000100:00100000:2.0:1787739060.053963:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665641344 00000100:00100000:0.0:1787739060.054189:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2993 00000100:00100000:0.0:1787739060.054199:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ac780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6525:x1874579665641344:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739060.054318:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ac780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6525:x1874579665641344:12345-0@lo:1000:mdt00_002.0 Request processed in 121us (2913us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.054326:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2993 00000100:00100000:2.0:1787739060.054459:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356acf00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665641344:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739060.054526:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665641472:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739060.054566:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.054971:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.059210:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665641472 00000100:00100000:0.0:1787739060.059224:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665641472 00000100:00100000:0.0:1787739060.059285:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2994 00000100:00100000:0.0:1787739060.059296:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c1a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6525:x1874579665641472:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739060.059569:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c1a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6525:x1874579665641472:12345-0@lo:1000:mdt00_002.0 Request processed in 274us (5001us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739060.059577:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2994 00000100:00100000:3.0:1787739060.059668:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665641472:0@lo:1000:mdt00_002.0 00010000:00010000:3.0:1787739060.059741:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1acaef lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.059752:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1acaef lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.059763:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1acaef lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.059782:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356aed00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665641600:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1787739060.059820:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739060.059869:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6ed80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.060119:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665641600 00000100:00100000:2.0:1787739060.060124:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665641600 00000100:00100000:2.0:1787739060.060173:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2995 00000100:00100000:2.0:1787739060.060179:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356acf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:6525:x1874579665641600:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787739060.060191:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739060.060207:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1acaf6 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1acaef expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.060234:0:17834:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac681 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d9b expref: 9 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.060245:0:17834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1acaf6 lrc: 4/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acaef expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.060269:0:17834:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac681 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d9b expref: 9 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.060282:0:17834:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac681 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d9b expref: 9 pid: 6525 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.060293:0:17834:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c0f00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:17834:1874579665641728:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739060.060355:0:17834:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068879c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.067196:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739060.067233:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652460928 00000100:00100000:2.0:1787739060.067242:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652460928 00010000:00010000:2.0:1787739060.067282:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac681 lrc: 5/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d9b expref: 10 pid: 6525 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.067295:0:17834:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c0f00 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:17834:1874579665641728:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739060.067324:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1476 00010000:00010000:3.0:1787739060.067327:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1acaf6 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acaef expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.067337:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c1e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2409:x1874579652460928:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739060.067342:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9800) 00010000:00010000:3.0:1787739060.067347:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739060.067351:0:17834:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1acaf6 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acaef expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.067354:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739060.067363:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac681 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d9b expref: 10 pid: 6525 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.067364:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c200, rc 0) 00010000:00010000:2.0:1787739060.067378:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac681 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d9b expref: 10 pid: 6525 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.067400:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ac681 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094d9b expref: 10 pid: 6525 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.067437:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9800) 00010000:00010000:2.0:1787739060.067443:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1acaf6 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acaef expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.067455:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1acaf6 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1acaef expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.067462:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356acf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6525:x1874579665641600:12345-0@lo:101:mdt00_002.0 Request processed in 7271us (7630us total) trans 0 rc 0/0 00010000:00010000:2.0:1787739060.067466:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1acaf6 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1acaef expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.067475:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2995 00010000:00010000:2.0:1787739060.067500:0:6509:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1acaf6 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acaef expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.067515:0:6509:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c0f00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001_UUID:6509:1874579665641856:0@lo:105: 00000100:00100000:3.0:1787739060.067521:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356aed00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665641600:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739060.067538:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1acaef lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acaf6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.067549:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1acaef lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acaf6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.067554:0:6509:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.067587:0:6509:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887900 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.067608:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665641856 00000100:00100000:2.0:1787739060.067611:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665641856 00000100:00100000:2.0:1787739060.067642:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1129 00000100:00100000:2.0:1787739060.067649:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415701e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6509:x1874579665641856:12345-0@lo:105: 00010000:00010000:2.0:1787739060.067658:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1acaef lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acaf6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.067668:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1acaef lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acaf6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.067676:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1acaef lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acaf6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.067684:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1acaef lrc: 5/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1acaf6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.067691:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1acaef lrc: 6/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1acaf6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.067699:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1acaef lrc: 6/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1acaf6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.067720:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069abc00) 00000100:00100000:2.0:1787739060.067750:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415701e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6509:x1874579665641856:12345-0@lo:105: Request processed in 102us (198us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.067756:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1129 00000100:00100000:2.0:1787739060.067781:0:6509:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c0f00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001_UUID:6509:1874579665641856:0@lo:105: 00010000:00010000:2.0:1787739060.067791:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739060.067825:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c1e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:2409:x1874579652460928:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 491us (636us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.067832:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1476 00010000:00010000:3.0:1787739060.067920:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1acaef lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acaf6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.067934:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1acaef lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acaf6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739060.067954:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739060.067992:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1acafd lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.068000:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1acafd lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.068009:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1acafd lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.068024:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356aed00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665641984:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1787739060.068061:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739060.068088:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6ed80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.068270:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665641984 00000100:00100000:2.0:1787739060.068275:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665641984 00000100:00100000:2.0:1787739060.068313:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2996 00000100:00100000:2.0:1787739060.068321:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:6525:x1874579665641984:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787739060.068335:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739060.068349:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acb04 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1acafd expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.068366:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acb04 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acafd expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.068376:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acb04 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acafd expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.068391:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xcefb0e88 (ffffa07303c78e00) 00010000:00010000:2.0:1787739060.068395:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739060.068397:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d800, rc 0) 00000100:00100000:2.0:1787739060.068451:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665641984:12345-0@lo:101:mdt00_002.0 Request processed in 132us (389us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.068457:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2996 00000100:00100000:2.0:1787739060.068486:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356aed00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665641984:0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787739060.068496:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1acafd lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acb04 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.068505:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1acafd lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acb04 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739060.068516:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:2.0:1787739060.068560:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c0f00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665642112:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739060.068585:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.068601:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.068614:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665642112 00000100:00100000:2.0:1787739060.068617:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665642112 00000100:00100000:2.0:1787739060.068771:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2997 00000100:00100000:2.0:1787739060.068778:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665642112:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739060.068836:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665642112:12345-0@lo:1000:mdt00_002.0 Request processed in 58us (249us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.068841:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2997 00000100:00100000:2.0:1787739060.068865:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c0f00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665642112:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739060.068900:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c0f00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665642240:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739060.068921:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.068933:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.068943:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665642240 00000100:00100000:2.0:1787739060.068945:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665642240 00000100:00100000:2.0:1787739060.068964:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2998 00000100:00100000:2.0:1787739060.068971:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665642240:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739060.069034:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665642240:12345-0@lo:1000:mdt00_002.0 Request processed in 63us (112us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739060.069053:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2998 00000100:00100000:2.0:1787739060.069078:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c0f00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665642240:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739060.069141:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561e940 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665642368:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739060.069162:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.069186:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739060.069305:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665642368 00000100:00100000:3.0:1787739060.069309:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665642368 00000100:00100000:3.0:1787739060.069339:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2999 00000100:00100000:3.0:1787739060.069344:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665642368:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739060.069548:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665642368:12345-0@lo:1000:mdt00_002.0 Request processed in 204us (385us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739060.069553:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2999 00000100:00100000:2.0:1787739060.069756:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561e940 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665642368:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739060.069795:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561e940 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665642496:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739060.069894:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.069934:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739060.069957:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665642496 00000100:00100000:3.0:1787739060.069960:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665642496 00000100:00100000:3.0:1787739060.070064:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3000 00000100:00100000:3.0:1787739060.070072:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665642496:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739060.070154:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665642496:12345-0@lo:1000:mdt00_002.0 Request processed in 75us (250us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739060.070161:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3000 00000100:00100000:0.0:1787739060.070220:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561e940 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665642496:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739060.070266:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304261a40 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665642624:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739060.070296:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.071959:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665642624 00000100:00100000:2.0:1787739060.071971:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665642624 00000100:00100000:0.0:1787739060.071982:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.072029:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3001 00000100:00100000:2.0:1787739060.072046:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304263840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665642624:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739060.072253:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304263840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665642624:12345-0@lo:1000:mdt00_002.0 Request processed in 215us (1955us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.072260:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3001 00000100:00100000:0.0:1787739060.072530:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304261a40 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665642624:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739060.072983:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304261a40 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665642752:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739060.073024:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739060.073098:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074355edf00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.074765:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665642752 00000100:00100000:2.0:1787739060.074777:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665642752 00000100:00100000:2.0:1787739060.074841:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3002 00000100:00100000:2.0:1787739060.074851:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042621c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665642752:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739060.074987:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042621c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665642752:12345-0@lo:1000:mdt00_002.0 Request processed in 137us (1962us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.074995:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3002 00000100:00100000:3.0:1787739060.075084:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304261a40 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665642752:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739060.075226:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665642880:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739060.075274:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739060.075315:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.077227:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665642880 00000100:00100000:2.0:1787739060.077239:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665642880 00000100:00100000:2.0:1787739060.077294:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3003 00000100:00100000:2.0:1787739060.077303:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ad680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665642880:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739060.077440:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ad680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665642880:12345-0@lo:1000:mdt00_002.0 Request processed in 138us (2165us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.077448:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3003 00000100:00100000:2.0:1787739060.077502:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665642880:0@lo:1000:mdt00_002.0 00000040:00080000:2.0:1787739060.079786:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].365, 2168 off559224 00000040:00080000:2.0:1787739060.079902:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].364, 2368 off604928 00000004:00080000:2.0:1787739060.079909:0:6525:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0743c413780 with version 695 00000004:00080000:3.0:1787739060.081862:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 695 rpc_version 695 00000020:00100000:3.0:1787739060.081912:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739060.081922:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074356ae580 x1874579665643136/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739060.081945:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665643136:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739060.082017:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739060.082074:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6eb40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.082133:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665643136 00000100:00100000:0.0:1787739060.082140:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665643136 00000100:00100000:0.0:1787739060.082197:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3004 00000100:00100000:0.0:1787739060.082206:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356aed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665643136:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739060.083017:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356aed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:7740:x1874579665643136:12345-0@lo:1000:osp_up1-0.0 Request processed in 812us (998us total) trans 4294968360 rc 0/0 00000100:00100000:0.0:1787739060.083023:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3004 00000100:00100000:3.0:1787739060.083145:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665643136:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787739060.083555:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acb0b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1511:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.083570:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acb0b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1511:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.083581:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acb0b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1511:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.083600:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1acb12 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.083606:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1acb12 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.083612:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1acb12 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.083639:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1acb19 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.083645:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1acb19 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.083655:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1acb19 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739060.083670:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304260f00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665643264:0@lo:101:mdt00_002.0 00000100:00100000:0.0:1787739060.083702:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739060.083729:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740352a540 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.083778:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665643264 00000100:00100000:2.0:1787739060.083783:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665643264 00000100:00100000:2.0:1787739060.083815:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3005 00000100:00100000:2.0:1787739060.083822:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042612c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+11:6525:x1874579665643264:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787739060.083837:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739060.083851:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acb20 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1acb19 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.083870:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acb20 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acb19 expref: 12 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.083886:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acb20 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acb19 expref: 12 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.083894:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc2:0x0].0x0 (ffffa07303c78700) 00010000:00010000:2.0:1787739060.083897:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739060.083900:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633de00, rc 0) 00000100:00100000:2.0:1787739060.083940:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042612c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+12:6525:x1874579665643264:12345-0@lo:101:mdt00_002.0 Request processed in 120us (238us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.083946:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3005 00000100:00100000:3.0:1787739060.084510:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304260f00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665643264:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739060.084526:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1acb19 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acb20 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.084536:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1acb19 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acb20 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739060.084556:0:6525:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007adf58dc reply state 000000002a74f282 transno 4294983369 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1acb12 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.084567:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1acb12 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.084574:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc2:0x0].0x0 (ffffa073068a9c00) 00010000:00010000:3.0:1787739060.084576:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739060.084579:0:6525:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787739060.084584:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1acb19 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acb20 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.084591:0:6525:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1acb19 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acb20 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739060.084601:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xc2:0x0] 00000004:00010000:3.0:1787739060.084606:0:6525:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007adf58dc reply state 000000002a74f282 transno 4294983369 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acb0b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1511:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.084613:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acb0b lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1511:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.084619:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1511:0x0].0x0 (ffffa073068a9700) 00010000:00010000:3.0:1787739060.084621:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739060.084623:0:6525:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:3.0:1787739060.084628:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1511:0x0]: returning size 4096 00000004:00080000:3.0:1787739060.084705:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739060.084714:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1acaef lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acaf6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.084721:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1acaef lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acaf6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.084729:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1acaef lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acaf6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.084843:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739060.084850:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739060.084855:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1acafd lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acb04 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.084863:0:6525:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1acafd lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acb04 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.084946:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17549:x1874579652460800:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 33707us (34024us total) trans 4294983369 rc 0/0 00000100:00100000:3.0:1787739060.084954:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23375 00010000:00010000:1.0:1787739060.085869:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739060.085879:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1acaef lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1acaf6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.085889:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739060.085894:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1acaef lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1acaf6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739060.085903:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1acaef lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1acaf6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.085936:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1acaef lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acaf6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.085943:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1787739060.086032:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00010000:00010000:1.0:1787739060.086062:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1acaef lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acaf6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.086076:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abc00/0x7ee4453e4a1acaef lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acaf6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.086093:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739060.086095:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739060.086097:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739060.087478:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f71680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665643392:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739060.087551:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739060.091913:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665643392 00000100:00100000:0.0:1787739060.091928:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665643392 00000100:00100000:0.0:1787739060.092056:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1285 00000100:00100000:0.0:1787739060.092066:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305618f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+12:3651:x1874579665643392:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739060.092081:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739060.092107:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1acaf6 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1acaef expref: 12 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.092123:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9800) 00010000:00010000:0.0:1787739060.092133:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739060.092209:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305618f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+11:3651:x1874579665643392:12345-0@lo:103:ldlm_bl.0 Request processed in 144us (4657us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.092217:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1285 00000100:00100000:2.0:1787739060.092286:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f71680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665643392:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739060.110943:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739060.111028:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652461184 00000100:00100000:0.0:1787739060.111048:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652461184 00000100:00100000:0.0:1787739060.111106:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23376 00000100:00100000:0.0:1787739060.111116:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c380f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17550:x1874579652461184:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:0.0:1787739060.111271:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c380f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17550:x1874579652461184:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 156us (337us total) trans 0 rc -17/-17 00000100:00100000:0.0:1787739060.111279:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23376 00000100:00100000:0.0:1787739060.147561:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739060.147635:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652461312 00000100:00100000:1.0:1787739060.147644:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652461312 00000100:00100000:1.0:1787739060.147774:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23377 00000100:00100000:1.0:1787739060.147784:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304260b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17551:x1874579652461312:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739060.147978:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739060.148007:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1acb27 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094e82 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.148090:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1acb2e lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.148106:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1acb2e lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.148118:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1acb2e lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.148128:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acb35 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.148138:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acb04 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acafd expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.148147:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acb35 lrc: 4/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.148197:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acb04 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1acafd expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739060.148211:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f71680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665643520:0@lo:104: 00000100:00100000:1.0:1787739060.148594:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739060.148656:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4dd80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739060.148657:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665643520 00000100:00100000:3.0:1787739060.148664:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665643520 00000100:00100000:3.0:1787739060.149003:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1130 00000100:00100000:3.0:1787739060.149013:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:13793:x1874579665643520:12345-0@lo:104: 00010000:00010000:3.0:1787739060.149027:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1acafd lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1acb04 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.149128:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739060.149133:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:13793:x1874579665643520:12345-0@lo:104: Request processed in 123us (540us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739060.149140:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1130 00010000:00010000:2.0:1787739060.149194:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739060.149201:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1acafd lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1acb04 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.149210:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aaa00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739060.149215:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1acafd lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1acb04 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739060.149225:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1acafd lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1acb04 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.149255:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1acafd lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1acb04 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.149262:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739060.149284:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1acafd lrc: 1/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1acb04 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.149291:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1acafd lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1acb04 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.149309:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739060.149310:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739060.149312:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739060.149321:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787739060.149333:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561b840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665643648:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739060.149365:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.149397:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665643648 00000100:00100000:2.0:1787739060.149402:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665643648 00010000:00010000:2.0:1787739060.149450:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acb04 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1acafd expref: 11 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.149472:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1477 00000100:00100000:2.0:1787739060.149479:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+11:3650:x1874579665643648:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739060.149489:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787739060.150661:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f71680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665643520:0@lo:104: 00010000:00010000:1.0:1787739060.150699:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acb35 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.153809:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca94 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aca9b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739060.153829:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca94 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aca9b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.153872:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca94 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aca9b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.153879:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739060.153920:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1aca94 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aca9b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787739060.153939:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968360 is committed 00000001:00080000:2.0:1787739060.153960:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968359 is committed 00000020:00080000:2.0:1787739060.153968:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 356 batchid updated 00000020:00080000:2.0:1787739060.153977:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa073068ba480 refcount 2 committed 0 result 0 batchid 357 00000020:00080000:2.0:1787739060.153981:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743563fa00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa073068ba000 00000020:00080000:2.0:1787739060.153984:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].350 00000020:00080000:2.0:1787739060.153987:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743563f300 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306aad800 00000020:00080000:2.0:1787739060.153990:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].350 00000020:00080000:2.0:1787739060.154003:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 355 batchid updated 00010000:00010000:2.0:1787739060.154074:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acb04 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1acafd expref: 7 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.154089:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xcefb0e88 (ffffa07303c78e00) 00010000:00010000:2.0:1787739060.154094:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acb35 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.154103:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acb35 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.154110:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acb35 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.154135:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739060.154206:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+6:3650:x1874579665643648:12345-0@lo:103:ldlm_bl.0 Request processed in 4726us (4838us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.154216:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1477 00000100:00100000:2.0:1787739060.154270:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561b840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665643648:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739060.154297:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acb35 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.154305:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acb35 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.154402:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb5680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665643904:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739060.154463:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.154484:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413f00 going to sleep for 11 seconds 00010000:00010000:1.0:1787739060.154648:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aca24 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x16f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.154656:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aca24 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x16f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.154662:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aca24 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x16f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.154668:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739060.154672:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aca24 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x16f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.154679:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aca24 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x16f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.154688:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x16f:0x0].0x0 (ffffa074039f2a00) 00010000:00010000:1.0:1787739060.154691:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739060.154695:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aca24 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x16f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.154701:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aca24 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x16f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.154717:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1aca2b lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.154723:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1aca2b lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.154728:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1aca2b lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.154733:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739060.154736:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1aca2b lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.154742:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1aca2b lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.154755:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xbf:0x0].0x0 (ffffa074039f2700) 00010000:00010000:1.0:1787739060.154758:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739060.154761:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1aca2b lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.154765:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1aca2b lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xbf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739060.154946:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787739060.154957:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665643776:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:0.0:1787739060.154991:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000020:00080000:0.0:1787739060.155024:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 357 committed batchid 356 00000004:00080000:0.0:1787739060.155127:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0740352ae40 with version 669 00000020:00080000:0.0:1787739060.155145:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 355 cancel update log [0x200000401:0x3:0x0].348: rc = 0 00000020:00080000:0.0:1787739060.155186:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 355 cancel update log [0x240000400:0x3:0x0].348: rc = 0 00000004:00080000:0.0:1787739060.155209:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07410404f00 with version 670 00000020:00080000:0.0:1787739060.155213:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 356 cancel update log [0x200000401:0x3:0x0].349: rc = 0 00000020:00080000:0.0:1787739060.155233:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 356 cancel update log [0x240000400:0x3:0x0].349: rc = 0 00000020:00080000:0.0:1787739060.155242:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 356 00000100:00100000:3.0:1787739060.155386:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665643904 00000100:00100000:3.0:1787739060.155394:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665643904 00000100:00100000:3.0:1787739060.155449:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3006 00000100:00100000:3.0:1787739060.155459:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb61c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665643904:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739060.155540:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb61c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665643904:12345-0@lo:1000:mdt00_004.0 Request processed in 83us (1076us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739060.155548:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3006 00000100:00100000:3.0:1787739060.155581:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb5680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665643904:0@lo:1000:mdt00_004.0 00010000:00010000:3.0:1787739060.155622:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1acb3c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1511:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.155634:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1acb3c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.155642:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1acb3c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739060.155661:0:13793:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1acb3c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.155673:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1acb2e lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.155680:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1acb2e lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.155696:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1acb2e lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.155702:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1acb2e lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.155708:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ca00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739060.155711:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1acb2e lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.155720:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1acb2e lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.155729:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07433235200) 00010000:00010000:3.0:1787739060.155732:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739060.155736:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1acb2e lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.155742:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1acb2e lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.155758:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acb35 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.155764:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acb35 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.155771:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acb35 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.155776:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739060.155780:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acb35 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.155785:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acb35 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.155792:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xcefb0e88 (ffffa07303c78e00) 00010000:00010000:3.0:1787739060.155795:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739060.155798:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acb35 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.155807:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acb35 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.155842:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1acb27 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e82 expref: 8 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.155858:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1acb3c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e82 expref: 7 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.155866:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1511:0x0].0x0 (ffffa074039f2300) 00010000:00010000:3.0:1787739060.155868:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739060.155870:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab600, rc 0) 00000100:00100000:3.0:1787739060.155910:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304260b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17551:x1874579652461312:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 8128us (8360us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739060.155917:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23377 00000004:00080000:1.0:1787739060.157133:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 669 rpc_version 669 00000020:00100000:1.0:1787739060.157166:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739060.157176:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303f71680 x1874579665644160/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739060.157197:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f71680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665644160:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739060.157246:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739060.157274:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4dd80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739060.161203:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665644160 00000100:00100000:3.0:1787739060.161218:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665644160 00000100:00100000:3.0:1787739060.161301:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3007 00000100:00100000:3.0:1787739060.161312:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c4152c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665644160:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739060.161525:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c4152c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665644160:12345-0@lo:1000:osp_up0-1.0 Request processed in 214us (4277us total) trans 4294983370 rc 0/0 00000100:00100000:3.0:1787739060.161533:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3007 00000100:00100000:3.0:1787739060.161598:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f71680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665644160:0@lo:1000:osp_up0-1.0 00000004:00080000:3.0:1787739060.161612:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 670 rpc_version 670 00000020:00100000:3.0:1787739060.161635:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739060.161642:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0730383cb40 x1874579665644416/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739060.161663:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383cb40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665644416:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739060.161700:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739060.161730:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739060.161744:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665644416 00000100:00100000:3.0:1787739060.161747:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665644416 00000100:00100000:3.0:1787739060.161772:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3008 00000100:00100000:3.0:1787739060.161778:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665644416:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739060.161850:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665644416:12345-0@lo:1000:osp_up0-1.0 Request processed in 72us (150us total) trans 4294983371 rc 0/0 00000100:00100000:3.0:1787739060.161857:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3008 00000100:00100000:3.0:1787739060.161886:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383cb40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665644416:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739060.162529:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739060.162595:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652461440 00000100:00100000:1.0:1787739060.162601:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652461440 00000100:00100000:1.0:1787739060.162641:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23378 00000100:00100000:1.0:1787739060.162648:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17551:x1874579652461440:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739060.162662:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739060.162689:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1acb43 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1511:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094e89 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.162741:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1acb4a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1511:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.162765:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1acb4a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1511:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.162776:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1acb4a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1511:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739060.162869:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1511:0x0]: returning size 4096 00000004:00200000:1.0:1787739060.162874:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739060.162910:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1acb43 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1511:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e89 expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.162928:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1acb4a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1511:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e89 expref: 17 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.162936:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1511:0x0].0x0 (ffffa073068a9700) 00010000:00010000:1.0:1787739060.162939:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739060.162942:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d600, rc 0) 00000100:00100000:1.0:1787739060.162990:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17551:x1874579652461440:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 343us (463us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739060.163001:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23378 00000100:00100000:2.0:1787739060.163846:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665643776 00000100:00100000:2.0:1787739060.163854:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665643776 00000100:00100000:2.0:1787739060.163892:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1286 00000100:00100000:2.0:1787739060.163900:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074184c1c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+9:3653:x1874579665643776:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:2.0:1787739060.163911:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739060.163928:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1aca9b lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x872f39a bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aca94 expref: 9 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.163944:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x872f39a (ffffa07303c78900) 00010000:00010000:2.0:1787739060.163953:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739060.164003:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074184c1c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+8:3653:x1874579665643776:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 102us (9009us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.164010:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1286 00000100:00100000:0.0:1787739060.164124:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665643776:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:0.0:1787739060.179093:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739060.181725:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652461568 00000100:00100000:1.0:1787739060.181738:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652461568 00000100:00100000:1.0:1787739060.181813:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23379 00000100:00100000:1.0:1787739060.181823:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042630c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17551:x1874579652461568:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739060.181841:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739060.181862:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1acb51 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094e90 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.181925:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1acb58 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.181939:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1acb58 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.181950:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1acb58 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739060.181991:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xc2:0x0]: returning size 4096 00000004:00200000:1.0:1787739060.181995:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739060.182026:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1acb51 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e90 expref: 19 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.182052:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1acb58 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e90 expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.182061:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc2:0x0].0x0 (ffffa073068a9c00) 00010000:00010000:1.0:1787739060.182064:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739060.182075:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1ce00, rc 0) 00000100:00100000:1.0:1787739060.182145:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042630c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17551:x1874579652461568:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 323us (3058us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739060.182152:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23379 00000100:00100000:0.0:1787739060.189777:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739060.190100:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652461696 00000100:00100000:2.0:1787739060.190115:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652461696 00000100:00100000:2.0:1787739060.190178:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23380 00000100:00100000:2.0:1787739060.190189:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304260f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17551:x1874579652461696:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739060.190210:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739060.190232:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1acb5f lrc: 2/0,0 mode: --/CR res: [0x240000401:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094e97 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.190290:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acb66 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.190305:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acb20 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acb19 expref: 5 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.190314:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acb66 lrc: 4/1,0 mode: --/PR res: [0x240000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.190348:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acb20 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1acb19 expref: 5 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.190363:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665644544:0@lo:104: 00000100:00100000:2.0:1787739060.190439:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.190479:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c4133c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.190619:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665644544 00000100:00100000:0.0:1787739060.190624:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665644544 00000100:00100000:0.0:1787739060.190668:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1131 00000100:00100000:0.0:1787739060.190675:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:13793:x1874579665644544:12345-0@lo:104: 00010000:00010000:0.0:1787739060.190688:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1acb19 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1acb20 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.190747:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787739060.190754:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:13793:x1874579665644544:12345-0@lo:104: Request processed in 80us (315us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.190756:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665644544:0@lo:104: 00000100:00100000:0.0:1787739060.190768:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1131 00010000:00010000:2.0:1787739060.190782:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acb66 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.190794:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739060.190801:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1acb19 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1acb20 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.190810:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609bc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739060.190815:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1acb19 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xc2:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1acb20 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739060.190823:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1acb19 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xc2:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1acb20 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.191374:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1acb19 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xc2:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1acb20 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.191383:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739060.191976:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1acb19 lrc: 1/0,0 mode: --/PW res: [0x240000401:0xc2:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1acb20 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.191987:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609bc00/0x7ee4453e4a1acb19 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xc2:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1acb20 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.192005:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739060.192008:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739060.192010:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739060.192019:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787739060.192046:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304261a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665644672:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739060.192219:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739060.192263:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665644672 00000100:00100000:0.0:1787739060.192270:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665644672 00010000:00010000:0.0:1787739060.193147:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acb20 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1acb19 expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739060.193189:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1478 00000100:00100000:0.0:1787739060.193202:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+6:3652:x1874579665644672:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739060.193216:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00080000:3.0:1787739060.205982:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 357 batchid updated 00000020:00080000:0.0:1787739060.206213:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 357 00000004:00080000:0.0:1787739060.206274:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07306241240 with version 671 00000020:00080000:0.0:1787739060.206292:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 357 cancel update log [0x200000401:0x3:0x0].350: rc = 0 00000020:00080000:0.0:1787739060.206368:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 357 cancel update log [0x240000400:0x3:0x0].350: rc = 0 00000020:00080000:0.0:1787739060.206379:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 357 00000004:00080000:3.0:1787739060.206632:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 671 rpc_version 671 00000020:00100000:3.0:1787739060.206665:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739060.206676:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0730383d680 x1874579665644928/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739060.206700:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383d680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665644928:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739060.206771:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739060.206815:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.206944:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665644928 00000100:00100000:0.0:1787739060.206952:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665644928 00000100:00100000:0.0:1787739060.207011:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3009 00010000:00010000:2.0:1787739060.207020:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acb20 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1acb19 expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739060.207022:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383f840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665644928:12345-0@lo:1000:osp_up0-1.0 00010000:00010000:2.0:1787739060.207055:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc2:0x0].0x0 (ffffa07303c78700) 00010000:00010000:2.0:1787739060.207060:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acb66 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.207070:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acb66 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.207079:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acb66 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.207104:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739060.207153:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+5:3652:x1874579665644672:12345-0@lo:103:ldlm_bl.0 Request processed in 13955us (14935us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.207186:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383f840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665644928:12345-0@lo:1000:osp_up0-1.0 Request processed in 166us (413us total) trans 4294983372 rc 0/0 00000100:00100000:0.0:1787739060.207194:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3009 00000100:00100000:0.0:1787739060.207238:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304261a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665644672:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739060.207275:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383d680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665644928:0@lo:1000:osp_up0-1.0 00010000:00010000:3.0:1787739060.207728:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acb66 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.207738:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acb66 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739060.207795:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xc2:0x0]: returning size 4096 00000004:00200000:3.0:1787739060.207800:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739060.207845:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1acb5f lrc: 0/0,0 mode: --/CR res: [0x240000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e97 expref: 9 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.207863:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acb66 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e97 expref: 8 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.207871:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc2:0x0].0x0 (ffffa07303c78700) 00010000:00010000:3.0:1787739060.207874:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739060.207877:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d800, rc 0) 00000100:00100000:3.0:1787739060.207936:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304260f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17551:x1874579652461696:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 17748us (18691us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739060.207944:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23380 00000100:00100000:2.0:1787739060.207962:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1478 00000100:00100000:3.0:1787739060.225172:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739060.225226:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739060.225254:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652462080 00000100:00100000:3.0:1787739060.225264:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652462080 00000100:00100000:3.0:1787739060.225315:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23381 00000100:00100000:3.0:1787739060.225326:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17551:x1874579652462080:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:3.0:1787739060.225378:0:7944:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739060.225405:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1acb4a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1511:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e89 expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.225461:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1511:0x0].0x0 (ffffa073068a9700) 00010000:00010000:3.0:1787739060.225466:0:7944:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739060.225518:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1acb6d lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.225530:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1acb6d lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.225543:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1acb6d lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.225564:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665645056:0@lo:101:mdt00_003.0 00000100:00100000:3.0:1787739060.225607:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739060.225643:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e300 going to sleep for 11 seconds 00000100:00100000:3.0:1787739060.225661:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665645056 00000100:00100000:3.0:1787739060.225664:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665645056 00000100:00100000:3.0:1787739060.225706:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3010 00000100:00100000:3.0:1787739060.225713:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+5:7944:x1874579665645056:12345-0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739060.225722:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739060.225736:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1acb74 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1acb6d expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.225765:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1acb74 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acb6d expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.225778:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1acb74 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acb6d expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.225785:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074039f2500) 00010000:00010000:3.0:1787739060.225788:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739060.225790:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa400, rc 0) 00000100:00100000:3.0:1787739060.225828:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7944:x1874579665645056:12345-0@lo:101:mdt00_003.0 Request processed in 116us (220us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739060.225839:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3010 00000100:00100000:3.0:1787739060.225877:0:7944:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968360 gen 2 00000004:00080000:3.0:1787739060.225883:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968355 committed transno 4294968360 00000020:00080000:3.0:1787739060.225889:0:7944:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07410404000 refcount 2 committed 0 result 0 batchid 366 00000020:00080000:3.0:1787739060.225892:0:7944:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2800 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07410404240 00000020:00080000:3.0:1787739060.225895:0:7944:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].363 00000020:00080000:3.0:1787739060.225897:0:7944:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2f00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304cf5400 00000020:00080000:3.0:1787739060.225900:0:7944:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].363 00000004:00080000:3.0:1787739060.225924:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968357 committed transno 4294968360 00000004:00080000:3.0:1787739060.225935:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968358 committed transno 4294968360 00000004:00080000:3.0:1787739060.225944:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968360 committed transno 4294968360 00000100:00100000:3.0:1787739060.225961:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665645056:0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739060.225976:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1acb6d lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acb74 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.225987:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1acb6d lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acb74 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739060.226002:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739060.226048:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1acb7b lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.226055:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1acb7b lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.226063:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1acb7b lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.226080:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665645184:0@lo:101:mdt00_003.0 00000100:00100000:3.0:1787739060.226102:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739060.226127:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e300 going to sleep for 11 seconds 00000020:00080000:3.0:1787739060.226139:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 366 committed batchid 365 00000020:00080000:3.0:1787739060.226206:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 365 00000100:00100000:3.0:1787739060.226221:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665645184 00000100:00100000:3.0:1787739060.226225:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665645184 00000100:00100000:3.0:1787739060.226253:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3011 00000100:00100000:3.0:1787739060.226260:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7944:x1874579665645184:12345-0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739060.226269:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739060.226283:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1acb82 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1acb7b expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.226300:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1acb82 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acb7b expref: 7 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.226311:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1acb82 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acb7b expref: 7 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.226318:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xcefb0e88 (ffffa074039f2100) 00010000:00010000:3.0:1787739060.226321:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739060.226323:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa200, rc 0) 00000100:00100000:3.0:1787739060.226356:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665645184:12345-0@lo:101:mdt00_003.0 Request processed in 95us (252us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739060.226362:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3011 00000100:00100000:3.0:1787739060.226397:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665645184:0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739060.226408:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1acb7b lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acb82 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.226441:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1acb7b lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acb82 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739060.226453:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:3.0:1787739060.226487:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665645312:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739060.226514:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739060.226533:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e000 going to sleep for 11 seconds 00000100:00100000:3.0:1787739060.226545:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665645312 00000100:00100000:3.0:1787739060.226548:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665645312 00000100:00100000:3.0:1787739060.226573:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3012 00000100:00100000:3.0:1787739060.226580:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665645312:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739060.226638:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665645312:12345-0@lo:1000:mdt00_003.0 Request processed in 59us (123us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739060.226645:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3012 00000100:00100000:3.0:1787739060.226672:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665645312:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739060.226704:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665645440:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739060.226725:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739060.226740:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e000 going to sleep for 11 seconds 00000100:00100000:3.0:1787739060.226751:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665645440 00000100:00100000:3.0:1787739060.226754:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665645440 00000100:00100000:3.0:1787739060.226776:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3013 00000100:00100000:3.0:1787739060.226782:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665645440:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739060.226851:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665645440:12345-0@lo:1000:mdt00_003.0 Request processed in 69us (124us total) trans 0 rc 1/1 00000100:00100000:3.0:1787739060.226859:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3013 00000100:00100000:3.0:1787739060.226889:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665645440:0@lo:1000:mdt00_003.0 00010000:00010000:3.0:1787739060.226928:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1acb89 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.226934:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1acb89 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.226942:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1acb89 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.226953:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665645568:0@lo:101:mdt00_003.0 00000100:00100000:3.0:1787739060.226977:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739060.226993:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e300 going to sleep for 11 seconds 00000100:00100000:3.0:1787739060.227005:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665645568 00000100:00100000:3.0:1787739060.227008:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665645568 00000100:00100000:3.0:1787739060.227028:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3014 00000100:00100000:3.0:1787739060.227034:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665645568:12345-0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739060.227056:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739060.227066:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1acb90 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1acb89 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.227084:0:8447:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1acb3c lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e82 expref: 7 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.227093:0:8447:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1acb90 lrc: 4/0,0 mode: --/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acb89 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.227119:0:8447:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1acb3c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e82 expref: 7 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.227132:0:8447:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1acb3c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e82 expref: 7 pid: 13793 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.227143:0:8447:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383c780 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8447:1874579665645696:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739060.227205:0:8447:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e000 going to sleep for 11 seconds 00000100:00100000:1.0:1787739060.227535:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652461824 00000100:00100000:1.0:1787739060.227540:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652461824 00010000:00010000:1.0:1787739060.227561:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1acb3c lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e82 expref: 8 pid: 13793 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739060.227589:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1479 00000100:00100000:1.0:1787739060.227596:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383cf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2409:x1874579652461824:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:1.0:1787739060.227607:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739060.227612:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1acb3c lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e82 expref: 8 pid: 13793 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.227624:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1acb3c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e82 expref: 8 pid: 13793 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.227641:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1511:0x0].0x0 (ffffa074039f2300) 00010000:00010000:1.0:1787739060.227647:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1acb90 lrc: 3/0,0 mode: --/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acb89 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.227658:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1acb90 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1acb89 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.227666:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1acb90 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1acb89 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.227700:0:6508:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1acb90 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acb89 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739060.227709:0:6508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001_UUID:6508:1874579665645824:0@lo:105: 00000100:00100000:1.0:1787739060.227737:0:6508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739060.227762:0:6508:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89900 going to sleep for 11 seconds 00000100:00100000:1.0:1787739060.227775:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665645824 00000100:00100000:1.0:1787739060.227779:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665645824 00000100:00100000:1.0:1787739060.227814:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1132 00000100:00100000:1.0:1787739060.227822:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060061c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6508:x1874579665645824:12345-0@lo:105: 00010000:00010000:1.0:1787739060.227831:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1acb89 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acb90 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.227840:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1acb89 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acb90 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.227848:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1acb89 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acb90 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.227857:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1acb89 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1acb90 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.227864:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1acb89 lrc: 5/0,1 mode: EX/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1acb90 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.227871:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1acb89 lrc: 5/0,1 mode: EX/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1acb90 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.227882:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069ab000) 00000100:00100000:1.0:1787739060.227915:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060061c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6508:x1874579665645824:12345-0@lo:105: Request processed in 94us (177us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739060.227922:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1132 00000100:00100000:1.0:1787739060.227950:0:6508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001_UUID:6508:1874579665645824:0@lo:105: 00010000:00010000:1.0:1787739060.227959:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739060.227985:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383cf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2409:x1874579652461824:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 390us (2822us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739060.227992:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1479 00010000:00010000:0.0:1787739060.234889:0:8447:0:(ldlm_lockd.c:741:ldlm_handle_ast_error()) ### client (nid 192.168.203.15@tcp) returned -22 from blocking AST (req@000000006a3e71cb x1874579665645696) - normal race ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1acb3c lrc: 1/0,0 mode: --/PR res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e82 expref: 7 pid: 13793 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.234916:0:8447:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1acb3c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e82 expref: 7 pid: 13793 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739060.234934:0:8447:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383c780 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8447:1874579665645696:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739060.234956:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1511:0x0].0x0 (ffffa074039f2300) 00010000:00010000:0.0:1787739060.234970:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1acb90 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acb89 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.234987:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1511:0x0].0x0 (ffffa074039f2300) 00010000:00010000:0.0:1787739060.234991:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739060.234994:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:0.0:1787739060.235079:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383ed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665645568:12345-0@lo:101:mdt00_003.0 Request processed in 8044us (8100us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.235088:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3014 00000100:00100000:1.0:1787739060.235192:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665645568:0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739060.235210:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1acb89 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acb90 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.235253:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1acb97 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.235265:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1acb97 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.235281:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1acb97 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.235300:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acb9e lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.235310:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1acb58 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e90 expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.235324:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acb9e lrc: 4/0,1 mode: --/EX res: [0x200000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.235356:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1acb58 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e90 expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.235371:0:7944:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1acb58 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e90 expref: 17 pid: 6523 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739060.235384:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665645952:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739060.235546:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89900 going to sleep for 11 seconds 00000100:00100000:1.0:1787739060.248993:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739060.249069:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652462208 00000100:00100000:2.0:1787739060.249079:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652462208 00010000:00010000:2.0:1787739060.249116:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1acb58 lrc: 5/0,0 mode: PR/PR res: [0x200000400:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e90 expref: 18 pid: 6523 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.249157:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1480 00000100:00100000:2.0:1787739060.249167:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306006580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2408:x1874579652462208:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739060.249182:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739060.249189:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1acb58 lrc: 5/0,0 mode: PR/PR res: [0x200000400:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e90 expref: 18 pid: 6523 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.249202:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1acb58 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e90 expref: 18 pid: 6523 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.249226:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc2:0x0].0x0 (ffffa073068a9c00) 00010000:00010000:2.0:1787739060.249232:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acb9e lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.249241:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acb9e lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.249249:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acb9e lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.249266:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739060.249313:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306006580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2408:x1874579652462208:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 147us (328us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.249321:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1480 00010000:00010000:3.0:1787739060.249803:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1acb58 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e90 expref: 17 pid: 6523 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.249820:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665645952:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739060.249839:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acb9e lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.249849:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acb9e lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.249855:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acb9e lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.249895:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1acba5 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.249902:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1acba5 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.249910:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1acba5 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.249928:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665646080:0@lo:101:mdt00_003.0 00000100:00100000:3.0:1787739060.249967:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739060.250119:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6eb40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739060.250386:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665646080 00000100:00100000:1.0:1787739060.250391:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665646080 00000100:00100000:2.0:1787739060.250884:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3015 00000100:00100000:2.0:1787739060.250894:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665646080:12345-0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739060.250910:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739060.250928:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbac lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1acba5 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.250954:0:17834:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acb66 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e97 expref: 6 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.250964:0:17834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbac lrc: 4/0,0 mode: --/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acba5 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.250988:0:17834:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acb66 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e97 expref: 6 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.251001:0:17834:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acb66 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e97 expref: 6 pid: 13793 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.251012:0:17834:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb6580 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:17834:1874579665646208:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739060.251089:0:17834:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739060.262263:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739060.262338:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652462336 00000100:00100000:0.0:1787739060.262347:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652462336 00010000:00010000:0.0:1787739060.262387:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acb66 lrc: 5/0,0 mode: PR/PR res: [0x240000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e97 expref: 7 pid: 13793 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739060.262447:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1481 00000100:00100000:0.0:1787739060.262457:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652462336:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739060.262472:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739060.262479:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acb66 lrc: 5/0,0 mode: PR/PR res: [0x240000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e97 expref: 7 pid: 13793 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.262492:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acb66 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e97 expref: 7 pid: 13793 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.262515:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc2:0x0].0x0 (ffffa07303c78700) 00010000:00010000:0.0:1787739060.262521:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbac lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acba5 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.262533:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbac lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1acba5 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.262541:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbac lrc: 4/0,0 mode: EX/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1acba5 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.262580:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbac lrc: 4/0,0 mode: EX/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acba5 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739060.262594:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304261a40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665646336:0@lo:105: 00000100:00100000:0.0:1787739060.262640:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739060.262672:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062413c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.262704:0:6506:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787739060.262716:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665646336 00000100:00100000:0.0:1787739060.262719:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665646336 00000100:00100000:0.0:1787739060.262748:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1133 00000100:00100000:0.0:1787739060.262755:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f913c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665646336:12345-0@lo:105: 00010000:00010000:0.0:1787739060.262765:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1acba5 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acbac expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.262775:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1acba5 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acbac expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.262784:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1acba5 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acbac expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.262791:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1acba5 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1acbac expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.262798:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1acba5 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1acbac expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.262806:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1acba5 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1acbac expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.262820:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069aa600) 00000100:00100000:0.0:1787739060.262855:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f913c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665646336:12345-0@lo:105: Request processed in 100us (214us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.262862:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1133 00000100:00100000:0.0:1787739060.262930:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304261a40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665646336:0@lo:105: 00010000:00010000:0.0:1787739060.262944:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739060.262981:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652462336:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 526us (729us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.262988:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1481 00010000:00010000:3.0:1787739060.265785:0:17834:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acb66 lrc: 0/0,0 mode: --/PR res: [0x240000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e97 expref: 6 pid: 13793 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.265809:0:17834:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb6580 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:17834:1874579665646208:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739060.265839:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbac lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acba5 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.265849:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc2:0x0].0x0 (ffffa07303c78700) 00010000:00010000:3.0:1787739060.265852:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739060.265855:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633dc00, rc 0) 00000100:00100000:0.0:1787739060.266050:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665646080:12345-0@lo:101:mdt00_003.0 Request processed in 15157us (16081us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.266059:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3015 00000100:00100000:3.0:1787739060.266146:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665646080:0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739060.266163:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1acba5 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acbac expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.266818:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665646464:0@lo:403:mdt00_003.0 00000100:00100000:3.0:1787739060.266867:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739060.266899:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6eb40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.266968:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665646464 00000100:00100000:0.0:1787739060.266974:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665646464 00000100:00100000:0.0:1787739060.267019:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3016 00000100:00100000:0.0:1787739060.267028:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665646464:12345-0@lo:403:mdt00_003.0 00000100:00100000:0.0:1787739060.268633:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665646464:12345-0@lo:403:mdt00_003.0 Request processed in 1600us (1760us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.268645:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3016 00000100:00100000:0.0:1787739060.268708:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665646464:0@lo:403:mdt00_003.0 00000040:00080000:0.0:1787739060.269608:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].366, 664 off559888 00000040:00080000:0.0:1787739060.269782:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].365, 744 off605672 00000004:00080000:0.0:1787739060.269788:0:7944:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa073062413c0 with version 696 00000004:00080000:0.0:1787739060.269813:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 696 rpc_version 696 00000020:00100000:0.0:1787739060.269840:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739060.269851:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304261a40 x1874579665646720/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1808/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739060.269872:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304261a40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665646720:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739060.269921:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739060.269947:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.269967:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665646720 00000100:00100000:0.0:1787739060.269973:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665646720 00000100:00100000:0.0:1787739060.270011:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3017 00000100:00100000:0.0:1787739060.270019:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665646720:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739060.276977:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665646720:12345-0@lo:1000:osp_up1-0.0 Request processed in 6953us (7049us total) trans 4294968361 rc 0/0 00000100:00100000:0.0:1787739060.276990:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3017 00000100:00100000:3.0:1787739060.277126:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304261a40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665646720:0@lo:1000:osp_up1-0.0 00000004:00200000:0.0:1787739060.277185:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1511:0x0]: returning size 4096 00010000:00010000:0.0:1787739060.277209:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acb9e lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.277219:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acb9e lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.277227:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acb9e lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.277234:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739060.277238:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acb9e lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.277246:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acb9e lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.277261:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc2:0x0].0x0 (ffffa073068a9c00) 00010000:00010000:0.0:1787739060.277264:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739060.277269:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acb9e lrc: 2/0,0 mode: --/EX res: [0x200000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.277275:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acb9e lrc: 0/0,0 mode: --/EX res: [0x200000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.277288:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1acba5 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acbac expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.277296:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1acba5 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acbac expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.277303:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1acba5 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acbac expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.277326:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739060.277335:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xc2:0x0] 00010000:00010000:0.0:1787739060.277342:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1acb97 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.277349:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1acb97 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.277354:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1acb97 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.277360:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739060.277363:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1acb97 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.277369:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1acb97 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.277378:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1511:0x0].0x0 (ffffa073068a9700) 00010000:00010000:0.0:1787739060.277381:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739060.277384:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1acb97 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.277389:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1acb97 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.277399:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1acb89 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acb90 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.277406:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1acb89 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acb90 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.277440:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1acb89 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acb90 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.277456:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:0.0:1787739060.277464:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739060.277468:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1acb6d lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acb74 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.277475:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1acb6d lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acb74 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.277482:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1acb6d lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acb74 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.277499:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:0.0:1787739060.277504:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739060.277508:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1acb7b lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acb82 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.277515:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1acb7b lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acb82 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739060.277576:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17551:x1874579652462080:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 52251us (52347us total) trans 4294983373 rc 0/0 00000100:00100000:0.0:1787739060.277584:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23381 00010000:00010000:3.0:1787739060.277766:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1787739060.277772:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1acba5 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1acbac expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.277785:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739060.277789:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1acba5 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1acbac expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739060.277797:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1acba5 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1acbac expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.277887:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1acba5 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acbac expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.277895:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739060.277917:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1acba5 lrc: 1/0,0 mode: --/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acbac expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.277923:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1acba5 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acbac expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.277939:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787739060.277943:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1acb89 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1acb90 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.277949:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739060.277953:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1acb89 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1acb90 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739060.277960:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1acb89 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1acb90 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.277981:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1acb89 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acb90 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.277987:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739060.277992:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1acb89 lrc: 1/0,0 mode: --/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acb90 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.277999:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1acb89 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acb90 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.278010:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739060.278013:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1acb6d lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1acb74 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.278019:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aac00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739060.278022:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1acb6d lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1acb74 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739060.278029:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1acb6d lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1acb74 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.278055:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1acb6d lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acb74 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.278061:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739060.278076:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1acb6d lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acb74 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.278088:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1acb6d lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acb74 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.278099:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739060.278101:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739060.278102:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739060.278116:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415701e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665646848:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739060.278161:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739060.278203:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787739060.278209:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665646976:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739060.278232:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739060.278260:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787739060.278268:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665647104:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739060.278293:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739060.278322:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665646848 00000100:00100000:3.0:1787739060.278328:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665646848 00000100:00100000:3.0:1787739060.278362:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665646976 00000100:00100000:3.0:1787739060.278365:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665646976 00000100:00100000:3.0:1787739060.278384:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665647104 00000100:00100000:3.0:1787739060.278386:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665647104 00000100:00100000:3.0:1787739060.278403:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1287 00000100:00100000:3.0:1787739060.278433:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f76580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+12:3650:x1874579665646848:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739060.278446:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739060.278463:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbac lrc: 0/0,0 mode: --/EX res: [0x240000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1acba5 expref: 12 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.278476:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc2:0x0].0x0 (ffffa07303c78700) 00010000:00010000:3.0:1787739060.278484:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739060.278516:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f76580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+11:3650:x1874579665646848:12345-0@lo:103:ldlm_bl.0 Request processed in 106us (355us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739060.278522:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1287 00000100:00100000:3.0:1787739060.278538:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1288 00000100:00100000:3.0:1787739060.278544:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+10:3650:x1874579665646976:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739060.278551:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739060.278561:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1acb90 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1511:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1acb89 expref: 10 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.278571:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1511:0x0].0x0 (ffffa074039f2300) 00010000:00010000:3.0:1787739060.278578:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739060.278601:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+9:3650:x1874579665646976:12345-0@lo:103:ldlm_bl.0 Request processed in 57us (368us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739060.278606:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1288 00000100:00100000:3.0:1787739060.278618:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1289 00000100:00100000:3.0:1787739060.278624:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074184c2450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+8:3653:x1874579665647104:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739060.278631:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739060.278641:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1acb74 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1acb6d expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.278650:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074039f2500) 00010000:00010000:3.0:1787739060.278655:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739060.278681:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074184c2450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+7:3653:x1874579665647104:12345-0@lo:103:ldlm_bl.0 Request processed in 57us (387us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739060.278687:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1289 00000100:00100000:3.0:1787739060.278750:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665647104:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739060.278776:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665646976:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739060.278784:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415701e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665646848:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739060.279980:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739060.282142:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739060.311237:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739060.312904:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652462464 00000100:00100000:2.0:1787739060.312917:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652462464 00000100:00100000:2.0:1787739060.313017:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23382 00000100:00100000:2.0:1787739060.313029:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306007c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17552:x1874579652462464:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739060.313058:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739060.313081:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1acbb3 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094e9e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.313144:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1acbba lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.313158:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1acbba lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.313170:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1acbba lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.313184:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbc1 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.313192:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbc1 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.313199:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbc1 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.313235:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1acbba lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.313243:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1acbba lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.313248:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1acbba lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.313254:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1acbba lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.313260:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739060.313263:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1acbba lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.313271:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1acbba lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.313283:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074419d5200) 00010000:00010000:2.0:1787739060.313286:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739060.313290:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1acbba lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.313296:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1acbba lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.313310:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbc1 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.313316:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbc1 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.313322:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbc1 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.313341:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739060.313344:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbc1 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.313351:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbc1 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.313358:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x3caee148 (ffffa074419d5800) 00010000:00010000:2.0:1787739060.313361:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739060.313364:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbc1 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.313369:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbc1 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.313405:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1acbb3 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e9e expref: 7 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.313436:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074419d5200) 00010000:00010000:2.0:1787739060.313439:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739060.313443:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1acbb3 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e9e expref: 7 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.313456:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633ce00, rc 0) 00000100:00100000:2.0:1787739060.313513:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306007c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17552:x1874579652462464:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 486us (2283us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739060.313520:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23382 00000100:00100000:1.0:1787739060.321866:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739060.321931:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652462592 00000100:00100000:3.0:1787739060.321940:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652462592 00000100:00100000:3.0:1787739060.321995:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23383 00000100:00100000:3.0:1787739060.322005:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17552:x1874579652462592:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739060.322026:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739060.322056:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1acbc8 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094ea5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.322120:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1acbcf lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.322133:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1acbcf lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.322144:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1acbcf lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.322157:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acbd6 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.322164:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acbd6 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.322171:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acbd6 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.322215:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1acbcf lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.322223:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1acbcf lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.322230:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1acbcf lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.322236:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1acbcf lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.322242:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aba00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739060.322246:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1acbcf lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.322253:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1acbcf lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.322264:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074039f2500) 00010000:00010000:3.0:1787739060.322267:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739060.322271:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1acbcf lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.322277:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1acbcf lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.322289:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acbd6 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.322295:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acbd6 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.322301:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acbd6 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.322306:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739060.322309:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acbd6 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.322315:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acbd6 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.322322:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x3caee148 (ffffa074039f2300) 00010000:00010000:3.0:1787739060.322336:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739060.322339:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acbd6 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.322344:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acbd6 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.322374:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1acbc8 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ea5 expref: 7 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.322384:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074039f2500) 00010000:00010000:3.0:1787739060.322387:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739060.322390:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1acbc8 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ea5 expref: 7 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.322402:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa400, rc 0) 00000100:00100000:3.0:1787739060.322619:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17552:x1874579652462592:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 614us (763us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739060.322631:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23383 00000100:00100000:0.0:1787739060.327552:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739060.327661:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652462720 00000100:00100000:1.0:1787739060.327671:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652462720 00000100:00100000:1.0:1787739060.327808:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23384 00000100:00100000:1.0:1787739060.327879:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17552:x1874579652462720:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739060.327899:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739060.327920:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acbdd lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094eac expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.327980:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1acbe4 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.327993:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1acbe4 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.328004:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1acbe4 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739060.328057:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x5:0x0]: returning size 4096 00000004:00200000:1.0:1787739060.328062:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739060.328097:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acbdd lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094eac expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.328114:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1acbe4 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094eac expref: 7 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.328122:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07433235f00) 00010000:00010000:1.0:1787739060.328124:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739060.328128:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d800, rc 0) 00000100:00100000:1.0:1787739060.328230:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17552:x1874579652462720:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 354us (755us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739060.328238:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23384 00000100:00100000:1.0:1787739060.337665:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739060.337868:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652462976 00000100:00100000:2.0:1787739060.337879:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652462976 00000100:00100000:2.0:1787739060.337941:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23385 00000100:00100000:2.0:1787739060.337952:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306006580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17552:x1874579652462976:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:2.0:1787739060.338012:0:6524:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739060.338049:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1acbe4 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094eac expref: 7 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.338067:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07433235f00) 00010000:00010000:2.0:1787739060.338076:0:6524:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739060.338132:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbeb lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.338145:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbeb lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.338156:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbeb lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.338169:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1acbf2 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x3caee148 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.338176:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1acbf2 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.338183:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1acbf2 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1787739060.342003:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].352, 2168 off535240 00000040:00080000:0.0:1787739060.342110:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].352, 2368 off584896 00000004:00080000:0.0:1787739060.342117:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0743c4133c0 with version 672 00000004:00080000:1.0:1787739060.342403:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 672 rpc_version 672 00000020:00100000:1.0:1787739060.342547:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739060.342558:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306004780 x1874579665647360/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 4232/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739060.342582:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004780 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665647360:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739060.342650:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739060.342789:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89000 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.342987:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665647360 00000100:00100000:0.0:1787739060.342995:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665647360 00000100:00100000:0.0:1787739060.343082:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3018 00000100:00100000:0.0:1787739060.343092:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c42f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665647360:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739060.343966:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004780 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665647360:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739060.344003:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c42f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665647360:12345-0@lo:1000:osp_up0-1.0 Request processed in 592us (1032us total) trans 4294983374 rc 0/0 00000100:00100000:0.0:1787739060.344012:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3018 00010000:00010000:0.0:1787739060.344270:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1acbf9 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x171:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.344286:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1acbf9 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x171:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.344298:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1acbf9 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x171:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.344321:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1acc00 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.344646:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1acc00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.344654:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1acc00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.344750:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1acc07 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.344758:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1acc07 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.344768:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1acc07 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739060.344788:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383de00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665647488:0@lo:101:mdt00_001.0 00000100:00100000:0.0:1787739060.344902:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739060.344977:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062416c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739060.345032:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665647488 00000100:00100000:1.0:1787739060.345045:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665647488 00000100:00100000:1.0:1787739060.345112:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3019 00000100:00100000:1.0:1787739060.345120:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383f840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6524:x1874579665647488:12345-0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739060.345136:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739060.345150:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acc0e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1acc07 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.345169:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acc0e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acc07 expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.345180:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acc0e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acc07 expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.345188:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc0:0x0].0x0 (ffffa07433235000) 00010000:00010000:1.0:1787739060.345190:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739060.345193:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c600, rc 0) 00000100:00100000:1.0:1787739060.345237:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383f840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:6524:x1874579665647488:12345-0@lo:101:mdt00_001.0 Request processed in 118us (336us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739060.345244:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3019 00000100:00100000:2.0:1787739060.345300:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383de00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665647488:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739060.345314:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1acc07 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acc0e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.345339:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1acc07 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acc0e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739060.345360:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000006aea465 reply state 0000000045cd435e transno 4294968362 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1acc00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.345373:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1acc00 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.345379:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc0:0x0].0x0 (ffffa073068a9800) 00010000:00010000:2.0:1787739060.345382:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739060.345385:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787739060.345391:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1acc07 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acc0e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.345404:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1acc07 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acc0e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739060.345436:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xc0:0x0] 00000004:00010000:2.0:1787739060.345443:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000006aea465 reply state 0000000045cd435e transno 4294968362 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1acbf9 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x171:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.345451:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1acbf9 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x171:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.345457:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x171:0x0].0x0 (ffffa073068a9a00) 00010000:00010000:2.0:1787739060.345459:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739060.345461:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:2.0:1787739060.345466:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x171:0x0]: returning size 4096 00010000:00010000:2.0:1787739060.345476:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbeb lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.345483:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbeb lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.345490:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbeb lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.345497:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbeb lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.345503:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739060.345507:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbeb lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.345514:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbeb lrc: 5/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.345526:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074419d5200) 00010000:00010000:2.0:1787739060.345528:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739060.345532:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbeb lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.345538:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acbeb lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739060.345556:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000006aea465 reply state 0000000045cd435e transno 4294968362 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1acbf2 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.345564:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1acbf2 lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.345570:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x3caee148 (ffffa074419d5800) 00010000:00010000:2.0:1787739060.345572:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739060.345574:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 2 00000100:00100000:2.0:1787739060.345646:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306006580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17552:x1874579652462976:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 7695us (7988us total) trans 4294968362 rc 0/0 00000100:00100000:2.0:1787739060.345654:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23385 00000100:00100000:2.0:1787739060.375868:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739060.375985:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652463232 00000100:00100000:3.0:1787739060.375994:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652463232 00000100:00100000:3.0:1787739060.376057:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23386 00000100:00100000:3.0:1787739060.376067:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17553:x1874579652463232:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:3.0:1787739060.376251:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17553:x1874579652463232:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 185us (391us total) trans 0 rc -17/-17 00000100:00100000:3.0:1787739060.376258:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23386 00000100:00100000:3.0:1787739060.410048:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739060.410129:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652463360 00000100:00100000:0.0:1787739060.410139:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652463360 00000100:00100000:0.0:1787739060.410202:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23387 00000100:00100000:0.0:1787739060.410211:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17554:x1874579652463360:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739060.410231:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739060.410254:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acc15 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094eb3 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410334:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acc1c lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410348:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acc1c lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410359:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acc1c lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410372:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acc23 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410379:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acc23 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410385:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acc23 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410481:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1acc2a lrc: 3/1,0 mode: --/PR res: [0x240000402:0x171:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410490:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1acc2a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x171:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410496:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1acc2a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x171:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739060.410568:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x171:0x0]: returning size 4096 00000004:00200000:0.0:1787739060.410574:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:0.0:1787739060.410592:0:7944:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1acc2a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x171:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410607:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acc1c lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410613:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acc1c lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410619:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acc1c lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410625:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acc1c lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410631:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739060.410635:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acc1c lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410642:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acc1c lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410654:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9900) 00010000:00010000:0.0:1787739060.410657:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739060.410661:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acc1c lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410667:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acc1c lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410679:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acc23 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410685:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acc23 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410692:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acc23 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410697:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739060.410700:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acc23 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410706:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acc23 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410712:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x3caee148 (ffffa074419d5800) 00010000:00010000:0.0:1787739060.410715:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739060.410718:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acc23 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410723:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acc23 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410747:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acc15 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094eb3 expref: 10 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410764:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1acc2a lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x171:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094eb3 expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.410772:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x171:0x0].0x0 (ffffa073068a9a00) 00010000:00010000:0.0:1787739060.410774:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787739060.410776:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609ac00, rc 0) 00000100:00100000:0.0:1787739060.410848:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17554:x1874579652463360:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 637us (819us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.410858:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23387 00000100:00100000:3.0:1787739060.414251:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739060.414396:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652463488 00000100:00100000:0.0:1787739060.414406:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652463488 00000100:00100000:3.0:1787739060.414483:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23388 00000100:00100000:3.0:1787739060.414495:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17554:x1874579652463488:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739060.414518:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739060.414537:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1acc31 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094eba expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.414592:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acc38 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.414604:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acc38 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.414614:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acc38 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739060.414663:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xc0:0x0]: returning size 4096 00000004:00200000:3.0:1787739060.414668:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739060.414719:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1acc31 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094eba expref: 11 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.414733:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acc38 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094eba expref: 10 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.414740:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc0:0x0].0x0 (ffffa073068a9800) 00010000:00010000:3.0:1787739060.414743:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739060.414745:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aac00, rc 0) 00000100:00100000:3.0:1787739060.414819:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17554:x1874579652463488:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 327us (575us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739060.414827:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23388 00000100:00100000:3.0:1787739060.418675:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739060.418822:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652463616 00000100:00100000:1.0:1787739060.418833:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652463616 00000100:00100000:1.0:1787739060.418890:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23389 00000100:00100000:1.0:1787739060.418901:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17554:x1874579652463616:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739060.418923:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739060.418946:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1acc3f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094ec1 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.419006:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acc46 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.419021:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acc0e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acc07 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.419030:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acc46 lrc: 4/1,0 mode: --/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.419074:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acc0e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1acc07 expref: 10 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739060.419088:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060061c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6525:1874579665647616:0@lo:104: 00000100:00100000:1.0:1787739060.419129:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739060.419171:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.419542:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665647616 00000100:00100000:2.0:1787739060.419547:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665647616 00000100:00100000:2.0:1787739060.419589:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1134 00000100:00100000:2.0:1787739060.419597:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6525:x1874579665647616:12345-0@lo:104: 00010000:00010000:2.0:1787739060.419609:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1acc07 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1acc0e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.419669:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739060.419675:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6525:x1874579665647616:12345-0@lo:104: Request processed in 81us (546us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.419693:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1134 00000100:00100000:3.0:1787739060.419798:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060061c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6525:1874579665647616:0@lo:104: 00010000:00010000:3.0:1787739060.419827:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acc46 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.419919:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739060.419927:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1acc07 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1acc0e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.419935:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ba00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739060.419940:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1acc07 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1acc0e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739060.419949:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1acc07 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1acc0e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.419976:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1acc07 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1acc0e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.419983:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739060.420016:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1acc07 lrc: 1/0,0 mode: --/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1acc0e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.420024:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1acc07 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1acc0e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.420051:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739060.420054:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739060.420055:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739060.420095:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665647744:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739060.420127:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739060.420307:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665647744 00000100:00100000:0.0:1787739060.420311:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665647744 00010000:00010000:0.0:1787739060.420342:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acc0e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1acc07 expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739060.420370:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1482 00000100:00100000:0.0:1787739060.420378:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3653:x1874579665647744:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739060.420388:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739060.439507:0:6502:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1acb7b lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1acb82 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739060.439529:0:6502:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1acb7b lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1acb82 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.439571:0:6502:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1acb7b lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acb82 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.439577:0:6502:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1787739060.439628:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665647872:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787739060.439698:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787739060.439879:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1acb7b lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acb82 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787739060.439897:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983374 is committed 00000001:00080000:0.0:1787739060.439910:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983373 is committed 00000020:00080000:0.0:1787739060.439916:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 366 batchid updated 00000020:00080000:0.0:1787739060.439929:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa0743c4136c0 refcount 2 committed 0 result 0 batchid 367 00000020:00080000:0.0:1787739060.439933:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730424d500 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa0743c413780 00000020:00080000:0.0:1787739060.439936:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].364 00000020:00080000:0.0:1787739060.439938:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0730424df00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306941e00 00000020:00080000:0.0:1787739060.439941:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].365 00000020:00080000:0.0:1787739060.439962:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 367 committed batchid 366 00080000:02000000:0.0:1787739060.439979:0:6555:0:(osd_handler.c:2218:osd_trans_start()) *** cfs_fail_loc=19a, val=0*** 00000004:00080000:0.0:1787739060.440059:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306241f00 with version 697 00000020:00080000:0.0:1787739060.440070:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 366 cancel update log [0x240000401:0x3:0x0].363: rc = 0 00000020:00080000:0.0:1787739060.440134:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 366 cancel update log [0x200000400:0x3:0x0].363: rc = 0 00000020:00080000:0.0:1787739060.440142:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 367 committed batchid 366 00000004:00080000:0.0:1787739060.440165:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 697 rpc_version 697 00000020:00100000:0.0:1787739060.440183:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739060.440189:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0730383da40 x1874579665648128/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739060.440205:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383da40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665648128:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739060.440238:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739060.440257:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062410c0 going to sleep for 11 seconds 00010000:00010000:0.0:1787739060.440309:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acc0e lrc: 0/0,0 mode: --/PW res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1acc07 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.440322:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc0:0x0].0x0 (ffffa07433235000) 00010000:00010000:0.0:1787739060.440327:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acc46 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.440337:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acc46 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.440344:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acc46 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.440367:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739060.440404:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+5:3653:x1874579665647744:12345-0@lo:103:ldlm_bl.0 Request processed in 20026us (20275us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.440435:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1482 00000100:00100000:0.0:1787739060.440459:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665647872 00000100:00100000:0.0:1787739060.440465:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665647872 00000100:00100000:0.0:1787739060.440500:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1290 00000100:00100000:0.0:1787739060.440507:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+7:3650:x1874579665647872:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:0.0:1787739060.440518:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739060.440530:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1acb82 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xcefb0e88 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1acb7b expref: 7 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.440541:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xcefb0e88 (ffffa074039f2100) 00010000:00010000:0.0:1787739060.440547:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739060.440729:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acb0b lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1511:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.440739:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acb0b lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1511:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.440745:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acb0b lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1511:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.440751:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609aa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739060.440755:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acb0b lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1511:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.440765:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acb0b lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1511:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.440775:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1511:0x0].0x0 (ffffa073068a9700) 00010000:00010000:1.0:1787739060.440777:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739060.440780:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acb0b lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1511:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.440785:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acb0b lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1511:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.440797:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1acb12 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.440814:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1acb12 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.440819:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1acb12 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.440824:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739060.440826:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1acb12 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.440832:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1acb12 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.440838:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc2:0x0].0x0 (ffffa073068a9c00) 00010000:00010000:1.0:1787739060.440840:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739060.440842:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1acb12 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.440852:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1acb12 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739060.441161:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+6:3650:x1874579665647872:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 652us (1462us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.441169:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1290 00010000:00010000:0.0:1787739060.441192:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acc46 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.441200:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acc46 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739060.441247:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xc0:0x0]: returning size 4096 00000004:00200000:0.0:1787739060.441252:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739060.441297:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1acc3f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ec1 expref: 16 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.441311:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acc46 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ec1 expref: 15 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.441318:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc0:0x0].0x0 (ffffa07433235000) 00010000:00010000:0.0:1787739060.441320:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739060.441322:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1da00, rc 0) 00000100:00100000:0.0:1787739060.441388:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17554:x1874579652463616:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 22489us (22722us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.441396:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23389 00000100:00100000:0.0:1787739060.441470:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665647744:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739060.444472:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665647872:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787739060.444525:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665648128 00000100:00100000:2.0:1787739060.444531:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665648128 00000100:00100000:2.0:1787739060.444581:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3020 00000100:00100000:2.0:1787739060.444590:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665648128:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739060.445628:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665648128:12345-0@lo:1000:osp_up1-0.0 Request processed in 1034us (5384us total) trans 4294968363 rc 0/0 00000100:00100000:2.0:1787739060.445640:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3020 00000100:00100000:1.0:1787739060.446988:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383da40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665648128:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739060.447214:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739060.447290:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652463872 00000100:00100000:1.0:1787739060.447297:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652463872 00000100:00100000:2.0:1787739060.447636:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23390 00000100:00100000:2.0:1787739060.447647:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17554:x1874579652463872:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:2.0:1787739060.447772:0:13793:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739060.447799:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1acc2a lrc: 0/0,0 mode: --/PR res: [0x240000402:0x171:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094eb3 expref: 10 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.447888:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x171:0x0].0x0 (ffffa073068a9a00) 00010000:00010000:2.0:1787739060.447893:0:13793:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739060.447920:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acc4d lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.447933:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acc4d lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.447946:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acc4d lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.447957:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acc54 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x3caee148 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.447965:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acc54 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.447972:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acc54 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.448060:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acc5b lrc: 3/0,1 mode: --/EX res: [0x240000402:0x171:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.448069:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acc5b lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x171:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.448076:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acc5b lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x171:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.448093:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1acc62 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.448104:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acc38 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094eba expref: 9 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.448112:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1acc62 lrc: 4/0,1 mode: --/EX res: [0x240000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.448141:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acc38 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094eba expref: 9 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.448155:0:13793:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acc38 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094eba expref: 9 pid: 13793 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.448169:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb70c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665648256:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739060.448313:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.453673:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739060.453716:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652464000 00000100:00100000:0.0:1787739060.453725:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652464000 00010000:00010000:0.0:1787739060.453764:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acc38 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094eba expref: 10 pid: 13793 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739060.453799:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1483 00000100:00100000:0.0:1787739060.453920:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383ed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2406:x1874579652464000:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739060.453935:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739060.453943:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acc38 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094eba expref: 10 pid: 13793 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.453956:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acc38 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094eba expref: 10 pid: 13793 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.453978:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc0:0x0].0x0 (ffffa073068a9800) 00010000:00010000:0.0:1787739060.453984:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1acc62 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.453994:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1acc62 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.454006:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1acc62 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.454024:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739060.454082:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383ed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2406:x1874579652464000:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 275us (418us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.454089:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1483 00010000:00010000:2.0:1787739060.454593:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1acc38 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094eba expref: 9 pid: 13793 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.454610:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb70c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665648256:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739060.454631:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1acc62 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.454640:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1acc62 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.454646:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1acc62 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.454699:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acc69 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.454708:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acc69 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.454717:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acc69 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.454736:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb70c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665648384:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1787739060.454779:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.455237:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.457926:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665648384 00000100:00100000:0.0:1787739060.457941:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665648384 00000100:00100000:0.0:1787739060.458009:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3021 00000100:00100000:0.0:1787739060.458019:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb7840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+5:13793:x1874579665648384:12345-0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739060.458046:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739060.458103:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1acc70 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1acc69 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.458164:0:17834:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acc46 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ec1 expref: 14 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.458175:0:17834:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1acc70 lrc: 4/0,0 mode: --/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acc69 expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.458207:0:17834:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acc46 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ec1 expref: 14 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.458221:0:17834:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acc46 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ec1 expref: 14 pid: 6525 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739060.458241:0:17834:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383c780 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:17834:1874579665648512:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739060.458327:0:17834:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.461656:0:17834:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383c780 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:17834:1874579665648512:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739060.461701:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1acc70 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acc69 expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.461714:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc0:0x0].0x0 (ffffa07433235000) 00010000:00010000:0.0:1787739060.461717:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739060.461721:0:17834:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1acc70 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acc69 expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.461745:0:17834:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1acc70 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acc69 expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.461753:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a800, rc 0) 00000100:00100000:0.0:1787739060.462385:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb7840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:13793:x1874579665648384:12345-0@lo:101:mdt00_004.0 Request processed in 4365us (7601us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.462395:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3021 00000100:00100000:2.0:1787739060.464452:0:13793:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983374 gen 2 00000004:00080000:2.0:1787739060.464464:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983368 committed transno 4294983374 00000020:00080000:2.0:1787739060.464471:0:13793:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa074104046c0 refcount 2 committed 0 result 0 batchid 358 00000020:00080000:2.0:1787739060.464476:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2500 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07410404300 00000020:00080000:2.0:1787739060.464480:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].351 00000020:00080000:2.0:1787739060.464483:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2400 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304cf4400 00000020:00080000:2.0:1787739060.464485:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].351 00000004:00080000:2.0:1787739060.464516:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983370 committed transno 4294983374 00000004:00080000:2.0:1787739060.464528:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983371 committed transno 4294983374 00000004:00080000:2.0:1787739060.464536:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983372 committed transno 4294983374 00000004:00080000:2.0:1787739060.464543:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983374 committed transno 4294983374 00000100:00100000:2.0:1787739060.464559:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb70c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665648384:0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739060.464580:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acc69 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acc70 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.464591:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acc69 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acc70 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739060.464612:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 358 committed batchid 357 00000020:00080000:2.0:1787739060.464671:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 357 00000100:00100000:0.0:1787739060.465401:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739060.466862:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652464128 00000100:00100000:2.0:1787739060.466875:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652464128 00010000:00010000:2.0:1787739060.466912:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acc46 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ec1 expref: 15 pid: 6525 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.466951:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1484 00000100:00100000:2.0:1787739060.466962:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2409:x1874579652464128:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739060.466978:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739060.466984:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acc46 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ec1 expref: 15 pid: 6525 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.466997:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acc46 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ec1 expref: 15 pid: 6525 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.467017:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acc46 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ec1 expref: 15 pid: 6525 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.467031:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc0:0x0].0x0 (ffffa07433235000) 00010000:00010000:2.0:1787739060.467051:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1acc70 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acc69 expref: 5 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.467061:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1acc70 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1acc69 expref: 5 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.467070:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1acc70 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1acc69 expref: 5 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.467102:0:6508:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1acc70 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acc69 expref: 5 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.467115:0:6508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb7c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665648640:0@lo:105: 00000100:00100000:2.0:1787739060.467167:0:6508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.467212:0:6508:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739060.467510:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665648640 00000100:00100000:1.0:1787739060.467516:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665648640 00000100:00100000:1.0:1787739060.467546:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1135 00000100:00100000:1.0:1787739060.467553:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb61c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665648640:12345-0@lo:105: 00010000:00010000:1.0:1787739060.467565:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acc69 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acc70 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.467574:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acc69 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acc70 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.467582:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acc69 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acc70 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.467589:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acc69 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1acc70 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.467597:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acc69 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1acc70 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.467605:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acc69 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1acc70 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.467629:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633c600) 00000100:00100000:1.0:1787739060.467672:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb61c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665648640:12345-0@lo:105: Request processed in 118us (503us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739060.467679:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1135 00010000:00010000:1.0:1787739060.467703:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acc69 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acc70 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.467712:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acc69 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acc70 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739060.468507:0:6508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb7c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665648640:0@lo:105: 00010000:00010000:0.0:1787739060.468524:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739060.468573:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652464128:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 1611us (3174us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.468581:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1484 00000100:00100000:1.0:1787739060.469790:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665648768:0@lo:403:mdt00_004.0 00000100:00100000:1.0:1787739060.469864:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739060.469892:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89600 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.470019:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665648768 00000100:00100000:0.0:1787739060.470026:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665648768 00000100:00100000:0.0:1787739060.470079:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3022 00000100:00100000:0.0:1787739060.470087:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060061c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:13793:x1874579665648768:12345-0@lo:403:mdt00_004.0 00000100:00100000:0.0:1787739060.471639:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060061c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:13793:x1874579665648768:12345-0@lo:403:mdt00_004.0 Request processed in 1546us (1768us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.471651:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3022 00000100:00100000:1.0:1787739060.471833:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665648768:0@lo:403:mdt00_004.0 00000040:00080000:1.0:1787739060.472719:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].353, 664 off535904 00000040:00080000:1.0:1787739060.472883:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].353, 744 off585640 00000004:00080000:1.0:1787739060.472889:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07302e89600 with version 673 00000004:00080000:0.0:1787739060.473292:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 673 rpc_version 673 00000020:00100000:0.0:1787739060.473323:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739060.473371:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0730383c000 x1874579665649024/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1448/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739060.473395:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665649024:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739060.475151:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739060.475225:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.475268:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665649024 00000100:00100000:2.0:1787739060.475276:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665649024 00000100:00100000:2.0:1787739060.476130:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3023 00000100:00100000:2.0:1787739060.476143:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:7467:x1874579665649024:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739060.476433:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665649024:12345-0@lo:1000:osp_up0-1.0 Request processed in 293us (1287us total) trans 4294983375 rc 0/0 00000100:00100000:2.0:1787739060.476441:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3023 00000100:00100000:1.0:1787739060.476538:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665649024:0@lo:1000:osp_up0-1.0 00000004:00200000:1.0:1787739060.476567:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x171:0x0]: returning size 4096 00010000:00010000:1.0:1787739060.476589:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1acc62 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.476599:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1acc62 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.476612:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1acc62 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.476619:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739060.476623:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1acc62 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.476631:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1acc62 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.476645:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc0:0x0].0x0 (ffffa073068a9800) 00010000:00010000:1.0:1787739060.476648:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739060.476652:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1acc62 lrc: 2/0,0 mode: --/EX res: [0x240000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.476658:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1acc62 lrc: 0/0,0 mode: --/EX res: [0x240000400:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.476671:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acc69 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acc70 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.476679:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acc69 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acc70 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.476686:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acc69 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acc70 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.476707:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739060.476715:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xc0:0x0] 00010000:00010000:3.0:1787739060.476717:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739060.476722:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acc5b lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x171:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.476724:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acc69 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1acc70 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.476728:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acc5b lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x171:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.476732:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739060.476734:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acc5b lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x171:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.476736:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acc69 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1acc70 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.476738:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633de00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739060.476741:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acc5b lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x171:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739060.476744:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acc69 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1acc70 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.476747:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acc5b lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x171:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.476759:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x171:0x0].0x0 (ffffa073068a9a00) 00010000:00010000:1.0:1787739060.476761:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739060.476764:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acc5b lrc: 2/0,0 mode: --/EX res: [0x240000402:0x171:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.476769:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acc5b lrc: 0/0,0 mode: --/EX res: [0x240000402:0x171:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.476772:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acc69 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acc70 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.476778:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739060.476783:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acc4d lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.476789:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acc4d lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.476794:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acc4d lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.476796:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acc69 lrc: 1/0,0 mode: --/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acc70 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.476800:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acc4d lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.476804:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acc69 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acc70 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.476815:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739060.476818:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acc4d lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.476820:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739060.476822:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739060.476823:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acc4d lrc: 5/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.476824:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739060.476831:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074419d5200) 00010000:00010000:1.0:1787739060.476833:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739060.476836:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acc4d lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.476841:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acc4d lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739060.476854:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b3008f80 reply state 0000000017d8aa02 transno 4294968364 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acc54 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.476866:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acc54 lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.476873:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x3caee148 (ffffa074419d5800) 00010000:00010000:1.0:1787739060.476875:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739060.476878:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787739060.476933:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17554:x1874579652463872:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 29287us (29718us total) trans 4294968364 rc 0/0 00000100:00100000:1.0:1787739060.476941:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23390 00000100:00100000:1.0:1787739060.478545:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665649152:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739060.478606:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.484266:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665649152 00000100:00100000:2.0:1787739060.484279:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665649152 00000100:00100000:2.0:1787739060.484350:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1291 00000100:00100000:2.0:1787739060.484360:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+7:3651:x1874579665649152:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739060.484378:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739060.484404:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1acc70 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1acc69 expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.485521:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc0:0x0].0x0 (ffffa07433235000) 00010000:00010000:2.0:1787739060.485532:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739060.485608:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+6:3651:x1874579665649152:12345-0@lo:103:ldlm_bl.0 Request processed in 1249us (7001us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.485616:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1291 00000100:00100000:3.0:1787739060.487886:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665649152:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739060.510940:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739060.511410:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652464256 00000100:00100000:3.0:1787739060.511438:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652464256 00000100:00100000:3.0:1787739060.511497:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23391 00000100:00100000:3.0:1787739060.511509:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17557:x1874579652464256:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739060.511529:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739060.511551:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1acc77 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094ec8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.511627:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1acc7e lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.511640:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1acc7e lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.511656:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1acc7e lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.511670:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acc85 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.511677:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acc85 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.511697:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acc85 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.511738:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1acc7e lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.511746:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1acc7e lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.511751:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1acc7e lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.513125:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1acc7e lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.513139:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aba00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739060.513144:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1acc7e lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.513152:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1acc7e lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.513168:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235900) 00010000:00010000:3.0:1787739060.513172:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739060.513177:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1acc7e lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.513184:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1acc7e lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.513201:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acc85 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.513207:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acc85 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.513214:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acc85 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.513219:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739060.513221:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acc85 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.513226:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acc85 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.513233:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xeb6b201b (ffffa07305d56000) 00010000:00010000:3.0:1787739060.513236:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739060.513238:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acc85 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.513244:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acc85 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.513284:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1acc77 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ec8 expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.513300:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235900) 00010000:00010000:3.0:1787739060.513302:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739060.513305:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1acc77 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ec8 expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.513314:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa400, rc 0) 00000100:00100000:3.0:1787739060.513397:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17557:x1874579652464256:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 1889us (2465us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739060.513406:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23391 00000100:00100000:1.0:1787739060.521300:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739060.521794:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652464384 00000100:00100000:0.0:1787739060.521870:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652464384 00000100:00100000:0.0:1787739060.521938:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23392 00000100:00100000:0.0:1787739060.521949:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061ff840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17557:x1874579652464384:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739060.521971:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739060.521994:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acc8c lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094ecf expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.522060:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1acc93 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.522073:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1acc93 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.522084:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1acc93 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739060.522131:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x5:0x0]: returning size 4096 00000004:00200000:0.0:1787739060.522136:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739060.522170:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acc8c lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ecf expref: 12 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.522189:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1acc93 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ecf expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.522197:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9300) 00010000:00010000:0.0:1787739060.522200:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739060.522203:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a400, rc 0) 00000100:00100000:0.0:1787739060.522270:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061ff840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17557:x1874579652464384:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 322us (979us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.522277:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23392 00000100:00100000:0.0:1787739060.529553:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739060.529645:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652464640 00000100:00100000:3.0:1787739060.529655:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652464640 00000100:00100000:3.0:1787739060.529724:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23393 00000100:00100000:3.0:1787739060.529733:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17557:x1874579652464640:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:3.0:1787739060.529967:0:13793:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739060.529996:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acae8 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094e7b expref: 14 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.533867:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235900) 00010000:00010000:3.0:1787739060.533885:0:13793:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739060.533954:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1acc9a lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.533970:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1acc9a lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.533982:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1acc9a lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.533996:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1acca1 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.534003:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1acca1 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.534010:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1acca1 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787739060.535350:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].367, 2168 off562056 00000040:00080000:3.0:1787739060.535566:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].366, 2368 off608040 00000004:00080000:3.0:1787739060.535572:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07303c6ed80 with version 698 00000004:00080000:0.0:1787739060.536170:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 698 rpc_version 698 00000020:00100000:0.0:1787739060.536207:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739060.536218:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0730383d2c0 x1874579665649408/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 3976/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739060.536241:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383d2c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665649408:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739060.536307:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739060.536357:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241840 going to sleep for 11 seconds 00000100:00100000:3.0:1787739060.536523:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665649408 00000100:00100000:3.0:1787739060.536531:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665649408 00000100:00100000:3.0:1787739060.536613:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3024 00000100:00100000:3.0:1787739060.536622:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665649408:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739060.537287:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665649408:12345-0@lo:1000:osp_up1-0.0 Request processed in 664us (978us total) trans 4294968365 rc 0/0 00000100:00100000:3.0:1787739060.537296:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3024 00000100:00100000:0.0:1787739060.539956:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383d2c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665649408:0@lo:1000:osp_up1-0.0 00010000:00010000:3.0:1787739060.541053:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1acca8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1512:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.541072:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1acca8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1512:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.541084:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1acca8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1512:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.541103:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1accaf lrc: 3/0,1 mode: --/PW res: [0x200000400:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.541111:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1accaf lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.541117:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1accaf lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.541151:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1accb6 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.541158:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1accb6 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.541168:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1accb6 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.541188:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41d2c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665649536:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787739060.541253:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739060.541308:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.541620:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665649536 00000100:00100000:0.0:1787739060.541627:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665649536 00000100:00100000:0.0:1787739060.541675:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3025 00000100:00100000:0.0:1787739060.541873:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665649536:12345-0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739060.541892:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739060.541909:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1accbd lrc: 2/0,0 mode: --/PW res: [0x240000401:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1accb6 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.541930:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1accbd lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1accb6 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.541941:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1accbd lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1accb6 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.541948:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc3:0x0].0x0 (ffffa073068a9900) 00010000:00010000:0.0:1787739060.541951:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739060.541954:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609be00, rc 0) 00000100:00100000:0.0:1787739060.542003:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:13793:x1874579665649536:12345-0@lo:101:mdt00_004.0 Request processed in 202us (750us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.542010:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3025 00000100:00100000:3.0:1787739060.542110:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41d2c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665649536:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739060.542126:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1accb6 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1accbd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.542134:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1accb6 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1accbd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739060.542155:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002d0c0900 reply state 00000000963d1332 transno 4294983376 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1accaf lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.542166:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1accaf lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.542173:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc3:0x0].0x0 (ffffa07305d56d00) 00010000:00010000:3.0:1787739060.542175:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739060.542178:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787739060.542183:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1accb6 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1accbd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.542190:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1accb6 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1accbd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739060.542201:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xc3:0x0] 00000004:00010000:3.0:1787739060.542206:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002d0c0900 reply state 00000000963d1332 transno 4294983376 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1acca8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1512:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.542213:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1acca8 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1512:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.542219:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1512:0x0].0x0 (ffffa07305d56300) 00010000:00010000:3.0:1787739060.542221:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739060.542222:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:3.0:1787739060.542227:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1512:0x0]: returning size 4096 00010000:00010000:3.0:1787739060.542237:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1acc9a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.542243:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1acc9a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.542248:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1acc9a lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.542254:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1acc9a lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.542263:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739060.542267:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1acc9a lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.542274:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1acc9a lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.542283:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305d56f00) 00010000:00010000:3.0:1787739060.542285:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739060.542288:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1acc9a lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.542295:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1acc9a lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739060.542309:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002d0c0900 reply state 00000000963d1332 transno 4294983376 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1acca1 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.542317:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1acca1 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.542334:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xeb6b201b (ffffa07305d56a00) 00010000:00010000:3.0:1787739060.542336:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739060.542338:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 2 00000100:00100000:3.0:1787739060.542432:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17557:x1874579652464640:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 12698us (12886us total) trans 4294983376 rc 0/0 00000100:00100000:3.0:1787739060.542440:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23393 00010000:00010000:3.0:1787739060.547608:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739060.573064:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739060.573131:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652464896 00000100:00100000:1.0:1787739060.573142:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652464896 00000100:00100000:1.0:1787739060.573204:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23394 00000100:00100000:1.0:1787739060.573215:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17565:x1874579652464896:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:1.0:1787739060.573549:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17565:x1874579652464896:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 336us (493us total) trans 0 rc -17/-17 00000100:00100000:1.0:1787739060.573557:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23394 00000100:00100000:2.0:1787739060.599222:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739060.599271:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652465024 00000100:00100000:2.0:1787739060.599281:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652465024 00000100:00100000:2.0:1787739060.601462:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23395 00000100:00100000:2.0:1787739060.601540:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17566:x1874579652465024:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739060.601562:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739060.601586:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1accc4 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094ed6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.601648:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1acccb lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.601767:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1acccb lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.601781:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1acccb lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.601795:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1accd2 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.601802:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1accd2 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.601810:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1accd2 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.601879:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1accd9 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1512:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.601888:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1accd9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1512:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.601896:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1accd9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1512:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739060.602813:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1512:0x0]: returning size 4096 00000004:00200000:2.0:1787739060.602819:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:2.0:1787739060.602842:0:7944:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1accd9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1512:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.602862:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1acccb lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.602870:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1acccb lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.602876:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1acccb lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.602883:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1acccb lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.602889:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739060.602893:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1acccb lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.602901:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1acccb lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.603611:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5a00) 00010000:00010000:2.0:1787739060.603616:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739060.603622:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1acccb lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.603629:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1acccb lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.603644:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1accd2 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.603650:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1accd2 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.603666:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1accd2 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.603672:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739060.603676:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1accd2 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.603683:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1accd2 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.603694:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xeb6b201b (ffffa07305d56a00) 00010000:00010000:2.0:1787739060.603697:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739060.603700:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1accd2 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.606103:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1accd2 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.606168:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1accc4 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ed6 expref: 17 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.606190:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1accd9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1512:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ed6 expref: 16 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.606199:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1512:0x0].0x0 (ffffa07305d56300) 00010000:00010000:2.0:1787739060.606202:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787739060.606206:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633cc00, rc 0) 00000100:00100000:2.0:1787739060.606280:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17566:x1874579652465024:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 4749us (7065us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.606303:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23395 00000100:00100000:0.0:1787739060.610489:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739060.610853:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652465152 00000100:00100000:3.0:1787739060.610862:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652465152 00000100:00100000:3.0:1787739060.610928:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23396 00000100:00100000:3.0:1787739060.610938:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c381680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17566:x1874579652465152:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739060.610958:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739060.610986:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1acce0 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094edd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.611053:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1acce7 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.611065:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1acce7 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.611075:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1acce7 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739060.611125:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xc3:0x0]: returning size 4096 00000004:00200000:3.0:1787739060.611129:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739060.611163:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1acce0 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094edd expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.611178:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1acce7 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094edd expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.611186:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc3:0x0].0x0 (ffffa07305d56d00) 00010000:00010000:3.0:1787739060.611189:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739060.611192:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa400, rc 0) 00000100:00100000:3.0:1787739060.611248:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c381680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17566:x1874579652465152:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 311us (768us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739060.611258:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23396 00000100:00100000:3.0:1787739060.614020:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739060.614794:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652465280 00000100:00100000:0.0:1787739060.614813:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652465280 00000100:00100000:0.0:1787739060.614871:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23397 00000100:00100000:0.0:1787739060.614881:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17566:x1874579652465280:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739060.614902:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739060.614923:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1accee lrc: 2/0,0 mode: --/CR res: [0x240000401:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094ee4 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.614982:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1accf5 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.614997:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1accbd lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1accb6 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.615006:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1accf5 lrc: 4/1,0 mode: --/PR res: [0x240000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.615047:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1accbd lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1accb6 expref: 8 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739060.615061:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c382940 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665649664:0@lo:104: 00000100:00100000:0.0:1787739060.615116:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739060.615168:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241840 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.615400:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665649664 00000100:00100000:2.0:1787739060.615406:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665649664 00000100:00100000:2.0:1787739060.615462:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1136 00000100:00100000:2.0:1787739060.615470:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c4152c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6525:x1874579665649664:12345-0@lo:104: 00010000:00010000:2.0:1787739060.615483:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1accb6 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1accbd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.615563:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787739060.615563:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c382940 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665649664:0@lo:104: 00000100:00100000:2.0:1787739060.615574:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c4152c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6525:x1874579665649664:12345-0@lo:104: Request processed in 104us (456us total) trans 0 rc 0/0 00010000:00010000:3.0:1787739060.615577:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:2.0:1787739060.615584:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1136 00010000:00010000:3.0:1787739060.615586:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1accb6 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1accbd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.615586:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1accf5 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.615596:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739060.615601:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1accb6 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xc3:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1accbd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739060.615611:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1accb6 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xc3:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1accbd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.615639:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1accb6 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xc3:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1accbd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.615647:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739060.615673:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1accb6 lrc: 1/0,0 mode: --/PW res: [0x240000401:0xc3:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1accbd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.615690:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1accb6 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xc3:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1accbd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.615707:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739060.615709:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739060.615711:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739060.615759:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787739060.615771:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665649792:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739060.615854:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739060.615905:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665649792 00000100:00100000:3.0:1787739060.615910:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665649792 00010000:00010000:3.0:1787739060.615929:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1accbd lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1accb6 expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.615957:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1485 00000100:00100000:3.0:1787739060.615964:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+9:3653:x1874579665649792:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739060.615973:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:3.0:1787739060.625673:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968365 is committed 00000001:00080000:3.0:1787739060.625710:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968364 is committed 00000020:00080000:3.0:1787739060.625717:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 358 batchid updated 00000020:00080000:3.0:1787739060.625736:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa0743c4139c0 refcount 2 committed 0 result 0 batchid 359 00000020:00080000:3.0:1787739060.625741:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88100 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa0743c4133c0 00000020:00080000:3.0:1787739060.625745:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].352 00000020:00080000:3.0:1787739060.625747:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88c00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306940600 00000020:00080000:2.0:1787739060.625780:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 358 00010000:00010000:0.0:1787739060.625788:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1acbf2 lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.625800:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1acbf2 lrc: 2/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739060.625831:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0743c413c00 with version 674 00000020:00080000:2.0:1787739060.625844:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 358 cancel update log [0x200000401:0x3:0x0].351: rc = 0 00000020:00080000:3.0:1787739060.625852:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].352 00010000:00010000:0.0:1787739060.625874:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1acbf2 lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.625881:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739060.625886:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1acbf2 lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.625893:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1acbf2 lrc: 4/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.625908:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x3caee148 (ffffa074419d5800) 00010000:00010000:0.0:1787739060.625911:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739060.625915:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1acbf2 lrc: 2/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.625918:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1accbd lrc: 0/0,0 mode: --/PW res: [0x240000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1accb6 expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.625922:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1acbf2 lrc: 0/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.625931:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc3:0x0].0x0 (ffffa073068a9900) 00010000:00010000:0.0:1787739060.625932:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1acbf9 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x171:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739060.625934:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 358 cancel update log [0x240000400:0x3:0x0].351: rc = 0 00010000:00010000:3.0:1787739060.625936:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1accf5 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.625943:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1acbf9 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x171:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739060.625948:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 359 committed batchid 358 00010000:00010000:3.0:1787739060.625950:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1accf5 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.625950:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1acbf9 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x171:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.625957:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739060.625959:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1accf5 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.625959:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1acbf9 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x171:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.625966:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1acbf9 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x171:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.625974:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x171:0x0].0x0 (ffffa073068a9a00) 00000020:00080000:2.0:1787739060.625976:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 358 00010000:00010000:0.0:1787739060.625977:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739060.625982:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1acbf9 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x171:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.625983:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:2.0:1787739060.625989:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 674 rpc_version 674 00010000:00010000:0.0:1787739060.625989:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1acbf9 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x171:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.626005:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1acc00 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.626010:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1acc00 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.626015:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1acc00 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787739060.626019:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00010000:00010000:0.0:1787739060.626021:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739060.626026:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1acc00 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.626031:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07409fb4f00 x1874579665650048/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787739060.626044:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1acc00 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.626059:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc0:0x0].0x0 (ffffa073068a9800) 00010000:00010000:0.0:1787739060.626063:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739060.626065:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1acc00 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.626070:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb4f00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665650048:0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1787739060.626072:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1acc00 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xc0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.626074:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+5:3653:x1874579665649792:12345-0@lo:103:ldlm_bl.0 Request processed in 10109us (10271us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739060.626085:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1485 00010000:00010000:0.0:1787739060.626091:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acc54 lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.626097:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acc54 lrc: 2/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.626103:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acc54 lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.626108:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739060.626112:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acc54 lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.626118:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acc54 lrc: 4/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.626119:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787739060.626130:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x3caee148 (ffffa074419d5800) 00010000:00010000:0.0:1787739060.626132:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739060.626136:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665649792:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739060.626136:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acc54 lrc: 2/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.626141:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1acc54 lrc: 0/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0x3caee148 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.626151:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413480 going to sleep for 11 seconds 00010000:00010000:3.0:1787739060.626166:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1accf5 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.626173:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665650048 00010000:00010000:3.0:1787739060.626175:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1accf5 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.626181:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665650048 00000004:00200000:3.0:1787739060.626224:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xc3:0x0]: returning size 4096 00000004:00200000:3.0:1787739060.626230:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739060.626269:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1accee lrc: 0/0,0 mode: --/CR res: [0x240000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ee4 expref: 9 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.626284:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1accf5 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ee4 expref: 8 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.626291:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3026 00010000:00010000:3.0:1787739060.626296:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc3:0x0].0x0 (ffffa073068a9900) 00010000:00010000:3.0:1787739060.626300:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00000100:00100000:2.0:1787739060.626303:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:7467:x1874579665650048:12345-0@lo:1000:osp_up0-1.0 00010000:00010000:3.0:1787739060.626304:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a000, rc 0) 00000100:00100000:3.0:1787739060.626348:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17566:x1874579652465280:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 11468us (12335us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739060.626355:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23397 00000100:00100000:2.0:1787739060.626452:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665650048:12345-0@lo:1000:osp_up0-1.0 Request processed in 152us (334us total) trans 4294983377 rc 0/0 00000100:00100000:2.0:1787739060.626460:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3026 00000100:00100000:3.0:1787739060.628223:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb4f00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665650048:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739060.634625:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739060.634664:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652465536 00000100:00100000:1.0:1787739060.634674:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652465536 00000100:00100000:1.0:1787739060.634837:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23398 00000100:00100000:1.0:1787739060.634846:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17566:x1874579652465536:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:1.0:1787739060.634899:0:7944:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739060.634924:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1accd9 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1512:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ed6 expref: 17 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.634942:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1512:0x0].0x0 (ffffa07305d56300) 00010000:00010000:1.0:1787739060.634947:0:7944:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739060.634968:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1accfc lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.634981:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1accfc lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.634990:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1accfc lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.635002:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1acd03 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.635014:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1acd03 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.635020:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1acd03 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.635104:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1acd0a lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1512:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.635113:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1acd0a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1512:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.635120:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1acd0a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1512:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.635138:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acd11 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.635147:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1acce7 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094edd expref: 16 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.635155:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acd11 lrc: 4/0,1 mode: --/EX res: [0x200000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.635186:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1acce7 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094edd expref: 16 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.635199:0:7944:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1acce7 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094edd expref: 16 pid: 13793 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739060.635219:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fed00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665650176:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739060.635299:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.641258:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fed00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665650176:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739060.641287:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acd11 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.643537:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739060.643823:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652465664 00000100:00100000:0.0:1787739060.643834:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652465664 00010000:00010000:0.0:1787739060.643874:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1acce7 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094edd expref: 17 pid: 13793 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739060.643922:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1486 00000100:00100000:0.0:1787739060.643935:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074184c3c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2407:x1874579652465664:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739060.643949:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739060.643955:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1acce7 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094edd expref: 17 pid: 13793 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.643972:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1acce7 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094edd expref: 17 pid: 13793 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.643993:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1acce7 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094edd expref: 17 pid: 13793 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.644007:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc3:0x0].0x0 (ffffa07305d56d00) 00010000:00010000:0.0:1787739060.644012:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acd11 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.644021:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acd11 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.644028:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acd11 lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.644069:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739060.644128:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074184c3c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:2407:x1874579652465664:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 197us (600us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.644136:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1486 00010000:00010000:2.0:1787739060.644358:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acd11 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.644368:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acd11 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.644437:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d200/0x7ee4453e4a1acd18 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.644446:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d200/0x7ee4453e4a1acd18 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.644455:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d200/0x7ee4453e4a1acd18 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.644475:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb5680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665650304:0@lo:101:mdt00_003.0 00000100:00100000:2.0:1787739060.644515:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.646365:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.646471:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665650304 00000100:00100000:0.0:1787739060.646481:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665650304 00000100:00100000:0.0:1787739060.646539:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3027 00000100:00100000:0.0:1787739060.646548:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+5:7944:x1874579665650304:12345-0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739060.646567:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739060.646586:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1acd1f lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1acd18 expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.646620:0:8448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1accf5 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ee4 expref: 7 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.646630:0:8448:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1acd1f lrc: 4/0,0 mode: --/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acd18 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.646659:0:8448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1accf5 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ee4 expref: 7 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.646672:0:8448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1accf5 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ee4 expref: 7 pid: 6525 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739060.646692:0:8448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c382940 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8448:1874579665650432:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739060.646776:0:8448:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.654899:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739060.654964:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652465792 00000100:00100000:1.0:1787739060.654974:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652465792 00010000:00010000:1.0:1787739060.655013:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1accf5 lrc: 5/0,0 mode: PR/PR res: [0x240000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ee4 expref: 8 pid: 6525 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739060.655060:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1487 00000100:00100000:1.0:1787739060.655070:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383ed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2408:x1874579652465792:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739060.655085:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739060.655092:0:6510:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1accf5 lrc: 5/0,0 mode: PR/PR res: [0x240000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ee4 expref: 8 pid: 6525 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.655106:0:6510:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1accf5 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ee4 expref: 8 pid: 6525 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.655128:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc3:0x0].0x0 (ffffa073068a9900) 00010000:00010000:1.0:1787739060.655135:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1acd1f lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acd18 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.655146:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1acd1f lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1acd18 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.655154:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1acd1f lrc: 4/0,0 mode: EX/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1acd18 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.655191:0:6510:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1acd1f lrc: 4/0,0 mode: EX/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acd18 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739060.655210:0:6510:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fe940 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6510:1874579665650560:0@lo:105: 00000100:00100000:1.0:1787739060.655261:0:6510:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739060.655300:0:6510:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739060.655321:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665650560 00000100:00100000:1.0:1787739060.655325:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665650560 00000100:00100000:1.0:1787739060.655358:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1137 00000100:00100000:1.0:1787739060.655364:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6510:x1874579665650560:12345-0@lo:105: 00010000:00010000:1.0:1787739060.655378:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d200/0x7ee4453e4a1acd18 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acd1f expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.655388:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d200/0x7ee4453e4a1acd18 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acd1f expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.655396:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d200/0x7ee4453e4a1acd18 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acd1f expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.655403:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d200/0x7ee4453e4a1acd18 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1acd1f expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.655438:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d200/0x7ee4453e4a1acd18 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1acd1f expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.655448:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d200/0x7ee4453e4a1acd18 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1acd1f expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.655472:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633d200) 00000100:00100000:1.0:1787739060.655514:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fc3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6510:x1874579665650560:12345-0@lo:105: Request processed in 150us (252us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739060.655526:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1137 00000100:00100000:1.0:1787739060.655576:0:6510:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fe940 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6510:1874579665650560:0@lo:105: 00010000:00010000:1.0:1787739060.655592:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739060.655633:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383ed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2408:x1874579652465792:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 565us (741us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739060.655641:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1487 00010000:00010000:2.0:1787739060.656112:0:8448:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1accf5 lrc: 0/0,0 mode: --/PR res: [0x240000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ee4 expref: 7 pid: 6525 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.656131:0:8448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c382940 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001_UUID:8448:1874579665650432:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739060.656150:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1acd1f lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acd18 expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.656158:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc3:0x0].0x0 (ffffa073068a9900) 00010000:00010000:2.0:1787739060.656161:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739060.656164:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b000, rc 0) 00000100:00100000:2.0:1787739060.656215:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7944:x1874579665650304:12345-0@lo:101:mdt00_003.0 Request processed in 9667us (11698us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.656222:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3027 00000100:00100000:1.0:1787739060.656282:0:7944:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968365 gen 2 00000004:00080000:1.0:1787739060.656289:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968361 committed transno 4294968365 00000020:00080000:1.0:1787739060.656296:0:7944:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07306241d80 refcount 2 committed 0 result 0 batchid 368 00000020:00080000:1.0:1787739060.656301:0:7944:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2200 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa073062413c0 00000020:00080000:1.0:1787739060.656303:0:7944:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].365 00000020:00080000:1.0:1787739060.656305:0:7944:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2900 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304cf4200 00000020:00080000:1.0:1787739060.656308:0:7944:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].366 00000004:00080000:1.0:1787739060.656331:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968363 committed transno 4294968365 00000004:00080000:1.0:1787739060.656341:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968365 committed transno 4294968365 00000100:00100000:1.0:1787739060.656352:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb5680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665650304:0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739060.656367:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d200/0x7ee4453e4a1acd18 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acd1f expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00080000:0.0:1787739060.656805:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 368 committed batchid 366 00000020:00080000:0.0:1787739060.656851:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 366 00000100:00100000:1.0:1787739060.657598:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fe940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665650688:0@lo:403:mdt00_003.0 00000100:00100000:1.0:1787739060.657661:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739060.657847:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.657887:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665650688 00000100:00100000:0.0:1787739060.657894:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665650688 00000100:00100000:0.0:1787739060.657973:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3028 00000100:00100000:0.0:1787739060.657984:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073061fed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7944:x1874579665650688:12345-0@lo:403:mdt00_003.0 00000100:00100000:0.0:1787739060.659701:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073061fed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7944:x1874579665650688:12345-0@lo:403:mdt00_003.0 Request processed in 1712us (2031us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.659713:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3028 00000100:00100000:1.0:1787739060.659825:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fe940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665650688:0@lo:403:mdt00_003.0 00000040:00080000:1.0:1787739060.660672:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].368, 664 off562720 00000040:00080000:1.0:1787739060.660856:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].367, 744 off608784 00000004:00080000:1.0:1787739060.660862:0:7944:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0743c0ef480 with version 699 00000004:00080000:1.0:1787739060.660889:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 699 rpc_version 699 00000020:00100000:1.0:1787739060.660917:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739060.660927:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073061fe940 x1874579665650944/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1448/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739060.660949:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073061fe940 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665650944:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739060.660999:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739060.661282:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.661327:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665650944 00000100:00100000:2.0:1787739060.661334:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665650944 00000100:00100000:2.0:1787739060.661376:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3029 00000100:00100000:2.0:1787739060.661386:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665650944:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739060.661602:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665650944:12345-0@lo:1000:osp_up1-0.0 Request processed in 217us (601us total) trans 4294968366 rc 0/0 00000100:00100000:2.0:1787739060.661609:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3029 00000100:00100000:1.0:1787739060.661941:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073061fe940 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665650944:0@lo:1000:osp_up1-0.0 00000004:00200000:0.0:1787739060.662003:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1512:0x0]: returning size 4096 00010000:00010000:0.0:1787739060.662025:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acd11 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662044:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acd11 lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662051:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acd11 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662058:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1de00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739060.662063:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acd11 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662071:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acd11 lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662086:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc3:0x0].0x0 (ffffa07305d56d00) 00010000:00010000:0.0:1787739060.662090:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739060.662094:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acd11 lrc: 2/0,0 mode: --/EX res: [0x200000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662100:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acd11 lrc: 0/0,0 mode: --/EX res: [0x200000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662114:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d200/0x7ee4453e4a1acd18 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acd1f expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662122:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d200/0x7ee4453e4a1acd18 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acd1f expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662129:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d200/0x7ee4453e4a1acd18 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acd1f expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662186:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739060.662195:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xc3:0x0] 00010000:00010000:0.0:1787739060.662203:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1acd0a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1512:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662209:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1acd0a lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1512:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662214:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1acd0a lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1512:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662218:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739060.662221:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1acd0a lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1512:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662228:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1acd0a lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1512:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662236:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1512:0x0].0x0 (ffffa07305d56300) 00010000:00010000:0.0:1787739060.662239:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739060.662244:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1acd0a lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1512:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662250:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1acd0a lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1512:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662266:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1accfc lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662273:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1accfc lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662278:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1accfc lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662283:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1accfc lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662289:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739060.662291:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1accfc lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662296:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1accfc lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662304:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235100) 00010000:00010000:0.0:1787739060.662306:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739060.662309:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1accfc lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662314:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1accfc lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739060.662330:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000dff685e0 reply state 0000000086716a6e transno 4294983378 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1acd03 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662341:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1acd03 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.662349:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xeb6b201b (ffffa07305d56a00) 00010000:00010000:0.0:1787739060.662351:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739060.662353:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787739060.662441:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17566:x1874579652465536:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 27595us (27827us total) trans 4294983378 rc 0/0 00000100:00100000:0.0:1787739060.662450:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23398 00010000:00010000:1.0:1787739060.663529:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739060.663539:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d200/0x7ee4453e4a1acd18 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1acd1f expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.663549:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739060.663554:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d200/0x7ee4453e4a1acd18 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1acd1f expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739060.663562:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d200/0x7ee4453e4a1acd18 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1acd1f expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.663594:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d200/0x7ee4453e4a1acd18 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acd1f expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.663602:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1787739060.663652:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665651072:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739060.663815:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.663921:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665651072 00000100:00100000:2.0:1787739060.663928:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665651072 00000100:00100000:2.0:1787739060.663972:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1292 00000100:00100000:2.0:1787739060.663979:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+7:3653:x1874579665651072:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739060.663992:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739060.664009:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1acd1f lrc: 0/0,0 mode: --/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1acd18 expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.664027:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc3:0x0].0x0 (ffffa073068a9900) 00010000:00010000:2.0:1787739060.664071:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739060.664116:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d200/0x7ee4453e4a1acd18 lrc: 1/0,0 mode: --/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acd1f expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.664125:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d200/0x7ee4453e4a1acd18 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acd1f expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.664142:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739060.664144:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739060.664147:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739060.665627:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+6:3653:x1874579665651072:12345-0@lo:103:ldlm_bl.0 Request processed in 1642us (1806us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.665639:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1292 00000100:00100000:0.0:1787739060.665921:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665651072:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739060.678244:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739060.678347:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652465920 00000100:00100000:1.0:1787739060.678357:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652465920 00000100:00100000:1.0:1787739060.678442:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23399 00000100:00100000:1.0:1787739060.678454:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17567:x1874579652465920:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739060.678477:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739060.678500:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1acd26 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094eeb expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.678561:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acd2d lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.678574:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acd2d lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.678585:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acd2d lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.678598:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1acd34 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.678605:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1acd34 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.678616:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1acd34 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.678656:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acd2d lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.678663:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acd2d lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.678669:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acd2d lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.678676:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acd2d lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.678700:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739060.678704:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acd2d lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.678712:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acd2d lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.678724:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235300) 00010000:00010000:1.0:1787739060.678727:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739060.678732:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acd2d lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.678739:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acd2d lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.678752:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1acd34 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.678758:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1acd34 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.678764:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1acd34 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.678770:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739060.678773:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1acd34 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.678778:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1acd34 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.678786:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xc1169e19 (ffffa07433235600) 00010000:00010000:1.0:1787739060.678788:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739060.678791:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1acd34 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.678796:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1acd34 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.678837:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1acd26 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094eeb expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.678847:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235300) 00010000:00010000:1.0:1787739060.678849:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739060.678853:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1acd26 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094eeb expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.678865:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c200, rc 0) 00000100:00100000:1.0:1787739060.678927:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17567:x1874579652465920:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 476us (691us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739060.678934:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23399 00000100:00100000:2.0:1787739060.685593:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739060.688215:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652466048 00000100:00100000:3.0:1787739060.688301:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652466048 00000100:00100000:2.0:1787739060.688793:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23400 00000100:00100000:2.0:1787739060.688805:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb7c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17567:x1874579652466048:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739060.688824:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739060.688846:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1acd3b lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094ef2 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.688905:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1acd42 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.688920:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1acd42 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.688932:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1acd42 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.688946:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1acd49 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.688953:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1acd49 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.688960:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1acd49 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.689002:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1acd42 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.689010:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1acd42 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.689016:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1acd42 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.689022:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1acd42 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.689028:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739060.689041:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1acd42 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.689049:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1acd42 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.689060:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5e00) 00010000:00010000:2.0:1787739060.689063:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739060.689068:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1acd42 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.689074:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1acd42 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.689086:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1acd49 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.689092:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1acd49 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.689098:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1acd49 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.689102:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739060.689105:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1acd49 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.689111:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1acd49 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.689119:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xc1169e19 (ffffa074419d5700) 00010000:00010000:2.0:1787739060.689121:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739060.689124:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1acd49 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.689130:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1acd49 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.689159:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1acd3b lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ef2 expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.689168:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5e00) 00010000:00010000:2.0:1787739060.689171:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739060.689174:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1acd3b lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ef2 expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.689185:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633cc00, rc 0) 00000100:00100000:2.0:1787739060.689259:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb7c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17567:x1874579652466048:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 457us (3674us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739060.689271:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23400 00000100:00100000:3.0:1787739060.696198:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739060.696283:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652466176 00000100:00100000:2.0:1787739060.696293:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652466176 00000100:00100000:1.0:1787739060.696735:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23401 00000100:00100000:1.0:1787739060.696746:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17567:x1874579652466176:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739060.696765:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739060.696849:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1acd50 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094ef9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.696960:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acd57 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.696974:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acd57 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.696985:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acd57 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739060.697029:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:1.0:1787739060.697043:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739060.697078:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1acd50 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ef9 expref: 19 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.697093:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acd57 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ef9 expref: 18 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.697103:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235300) 00010000:00010000:1.0:1787739060.697106:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739060.697110:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c600, rc 0) 00000100:00100000:1.0:1787739060.697162:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17567:x1874579652466176:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 418us (973us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739060.697168:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23401 00000100:00100000:1.0:1787739060.710147:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739060.710229:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652466432 00000100:00100000:2.0:1787739060.710238:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652466432 00000100:00100000:2.0:1787739060.710297:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23402 00000100:00100000:2.0:1787739060.710317:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a312c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17567:x1874579652466432:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:2.0:1787739060.710754:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb70c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665651200:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739060.710814:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.710847:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739060.710934:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665651200 00000100:00100000:3.0:1787739060.710940:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665651200 00000100:00100000:3.0:1787739060.710980:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3030 00000100:00100000:3.0:1787739060.710989:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb5680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665651200:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739060.711079:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb5680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665651200:12345-0@lo:1000:mdt00_001.0 Request processed in 91us (264us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739060.711087:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3030 00000100:00100000:1.0:1787739060.711132:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb70c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665651200:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739060.711183:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a30f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665651328:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739060.711208:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739060.711227:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0efb40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.711273:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665651328 00000100:00100000:2.0:1787739060.711277:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665651328 00000100:00100000:2.0:1787739060.711708:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3031 00000100:00100000:2.0:1787739060.711720:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665651328:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739060.711798:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665651328:12345-0@lo:1000:mdt00_001.0 Request processed in 80us (587us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739060.716754:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a30f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665651328:0@lo:1000:mdt00_001.0 00010000:00010000:1.0:1787739060.716839:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1acd5e lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.716850:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1acd5e lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.716861:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1acd5e lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739060.716881:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a30f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665651456:0@lo:101:mdt00_001.0 00000100:00100000:1.0:1787739060.716939:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.717617:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3031 00000100:00100000:1.0:1787739060.717853:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.724292:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665651456 00000100:00100000:2.0:1787739060.724375:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665651456 00000100:00100000:2.0:1787739060.724444:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3032 00000100:00100000:2.0:1787739060.724452:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665651456:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739060.724470:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739060.724491:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1acd65 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1acd5e expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.724522:0:8448:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acd57 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ef9 expref: 17 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.724532:0:8448:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1acd65 lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acd5e expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.724564:0:8448:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acd57 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ef9 expref: 17 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.724577:0:8448:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acd57 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ef9 expref: 17 pid: 7944 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.724590:0:8448:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb70c0 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8448:1874579665651584:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739060.724672:0:8448:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413b40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.732060:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739060.732095:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652466560 00000100:00100000:2.0:1787739060.732104:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652466560 00000100:00100000:0.0:1787739060.732134:0:8448:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb70c0 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8448:1874579665651584:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739060.732144:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acd57 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ef9 expref: 18 pid: 7944 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.732169:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1acd65 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acd5e expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.732180:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235300) 00000100:00100000:2.0:1787739060.732182:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1488 00010000:00010000:0.0:1787739060.732185:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739060.732192:0:8448:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1acd65 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acd5e expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.732197:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2409:x1874579652466560:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739060.732209:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633cc00, rc 0) 00010000:00010000:2.0:1787739060.732217:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739060.732226:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acd57 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ef9 expref: 18 pid: 7944 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.732240:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acd57 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ef9 expref: 18 pid: 7944 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739060.732268:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6524:x1874579665651456:12345-0@lo:101:mdt00_001.0 Request processed in 7816us (15328us total) trans 0 rc 0/0 00010000:00010000:2.0:1787739060.732272:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1acd57 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ef9 expref: 18 pid: 7944 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739060.732280:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3032 00010000:00010000:2.0:1787739060.732292:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235300) 00010000:00010000:2.0:1787739060.732300:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1acd65 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acd5e expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.732311:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1acd65 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1acd5e expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.732320:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1acd65 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1acd5e expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739060.732345:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a30f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665651456:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739060.732362:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1acd5e lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acd65 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.732363:0:6508:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1acd65 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acd5e expref: 7 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.732383:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1acd5e lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acd65 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.732390:0:6508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f71680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665651712:0@lo:105: 00000100:00100000:2.0:1787739060.732454:0:6508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.732492:0:6508:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.732783:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665651712 00000100:00100000:0.0:1787739060.732787:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665651712 00000100:00100000:0.0:1787739060.732831:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1138 00000100:00100000:0.0:1787739060.732838:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f712c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665651712:12345-0@lo:105: 00010000:00010000:0.0:1787739060.732847:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1acd5e lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acd65 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.732856:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1acd5e lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acd65 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.732864:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1acd5e lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acd65 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.732872:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1acd5e lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1acd65 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.732878:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1acd5e lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1acd65 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.732886:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1acd5e lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1acd65 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.732902:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07441a1dc00) 00000100:00100000:0.0:1787739060.732935:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f712c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665651712:12345-0@lo:105: Request processed in 98us (481us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.732941:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1138 00010000:00010000:0.0:1787739060.732972:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1acd5e lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acd65 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.732982:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1acd5e lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acd65 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739060.733009:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739060.733049:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1acd6c lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.733056:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1acd6c lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.733065:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1acd6c lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739060.733080:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383da40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665651840:0@lo:101:mdt00_001.0 00000100:00100000:0.0:1787739060.733105:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739060.733124:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241540 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.733138:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665651840 00000100:00100000:0.0:1787739060.733141:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665651840 00000100:00100000:0.0:1787739060.733167:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3033 00000100:00100000:0.0:1787739060.733173:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb70c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6524:x1874579665651840:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739060.733186:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739060.733200:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1acd73 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1acd6c expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.733218:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1acd73 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acd6c expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.733235:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1acd73 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acd6c expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.733242:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xc1169e19 (ffffa07307064f00) 00010000:00010000:0.0:1787739060.733245:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739060.733247:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a800, rc 0) 00000100:00100000:0.0:1787739060.733274:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb70c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665651840:12345-0@lo:101:mdt00_001.0 Request processed in 100us (168us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.733279:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3033 00000100:00100000:0.0:1787739060.733319:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383da40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665651840:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739060.733329:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1acd6c lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acd73 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.733338:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1acd6c lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acd73 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739060.733350:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:0.0:1787739060.733379:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383da40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665651968:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739060.733404:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739060.733438:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.733450:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665651968 00000100:00100000:0.0:1787739060.733454:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665651968 00000100:00100000:0.0:1787739060.733476:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3034 00000100:00100000:0.0:1787739060.733482:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665651968:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739060.733531:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665651968:12345-0@lo:1000:mdt00_001.0 Request processed in 49us (126us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.733537:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3034 00000100:00100000:0.0:1787739060.733557:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383da40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665651968:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739060.733590:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252a940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665652096:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739060.733608:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739060.733619:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.733629:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665652096 00000100:00100000:0.0:1787739060.733632:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665652096 00000100:00100000:0.0:1787739060.733649:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3035 00000100:00100000:0.0:1787739060.733654:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665652096:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739060.733711:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665652096:12345-0@lo:1000:mdt00_001.0 Request processed in 56us (101us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739060.733717:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3035 00000100:00100000:0.0:1787739060.733739:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252a940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665652096:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739060.733800:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252a940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665652224:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739060.733823:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739060.733837:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.733848:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665652224 00000100:00100000:0.0:1787739060.733851:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665652224 00000100:00100000:0.0:1787739060.733871:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3036 00000100:00100000:0.0:1787739060.733876:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665652224:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739060.733920:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665652224:12345-0@lo:1000:mdt00_001.0 Request processed in 44us (96us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739060.733926:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3036 00000100:00100000:0.0:1787739060.733948:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252a940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665652224:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739060.733990:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252a940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665652352:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739060.734012:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739060.734025:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.734046:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665652352 00000100:00100000:0.0:1787739060.734049:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665652352 00000100:00100000:0.0:1787739060.734067:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3037 00000100:00100000:0.0:1787739060.734072:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665652352:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739060.734112:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665652352:12345-0@lo:1000:mdt00_001.0 Request processed in 39us (98us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.734117:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3037 00000100:00100000:0.0:1787739060.734139:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252a940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665652352:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739060.734175:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561ed00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665652480:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739060.734195:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739060.734206:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.734214:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665652480 00000100:00100000:0.0:1787739060.734217:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665652480 00000100:00100000:0.0:1787739060.734233:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3038 00000100:00100000:0.0:1787739060.734237:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665652480:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739060.734282:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665652480:12345-0@lo:1000:mdt00_001.0 Request processed in 44us (86us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.734287:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3038 00000100:00100000:0.0:1787739060.734306:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561ed00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665652480:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739060.734335:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561ed00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665652608:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739060.734357:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739060.734369:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.734379:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665652608 00000100:00100000:0.0:1787739060.734382:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665652608 00000100:00100000:0.0:1787739060.734402:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3039 00000100:00100000:0.0:1787739060.734407:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665652608:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739060.734478:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665652608:12345-0@lo:1000:mdt00_001.0 Request processed in 70us (120us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.734483:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3039 00000100:00100000:0.0:1787739060.734507:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561ed00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665652608:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739060.734533:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561ed00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665652736:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739060.734553:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739060.734566:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.734575:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665652736 00000100:00100000:0.0:1787739060.734578:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665652736 00000100:00100000:0.0:1787739060.734596:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3040 00000100:00100000:0.0:1787739060.734601:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665652736:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739060.734644:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665652736:12345-0@lo:1000:mdt00_001.0 Request processed in 42us (90us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.734650:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3040 00000100:00100000:0.0:1787739060.734673:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561ed00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665652736:0@lo:1000:mdt00_001.0 00000040:00080000:0.0:1787739060.735464:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].354, 2168 off538072 00000040:00080000:0.0:1787739060.735541:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].354, 2368 off588008 00000004:00080000:0.0:1787739060.735546:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07306241540 with version 675 00000100:00100000:2.0:1787739060.737702:0:6508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f71680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665651712:0@lo:105: 00010000:00010000:2.0:1787739060.737729:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739060.737800:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652466560:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 5606us (5747us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.737812:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1488 00000004:00080000:2.0:1787739060.738464:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 675 rpc_version 675 00000020:00100000:2.0:1787739060.738505:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739060.738514:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303f71680 x1874579665652992/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739060.738536:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f71680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665652992:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739060.738602:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.738635:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.738655:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665652992 00000100:00100000:2.0:1787739060.738662:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665652992 00000100:00100000:2.0:1787739060.738701:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3041 00000100:00100000:2.0:1787739060.738709:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356afc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665652992:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739060.740784:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356afc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665652992:12345-0@lo:1000:osp_up0-1.0 Request processed in 2071us (2177us total) trans 4294983379 rc 0/0 00000100:00100000:2.0:1787739060.740798:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3041 00000100:00100000:2.0:1787739060.740887:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f71680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665652992:0@lo:1000:osp_up0-1.0 00010000:00010000:2.0:1787739060.743303:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1acd7a lrc: 3/0,1 mode: --/PW res: [0x240000402:0x173:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.743364:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1acd7a lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x173:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.743376:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1acd7a lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x173:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.743396:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1acd81 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.743404:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1acd81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.743411:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1acd81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.743456:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1acd88 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.743462:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1acd88 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.743472:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1acd88 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.743494:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356acb40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665653120:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1787739060.743553:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.743585:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d840 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.743608:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665653120 00000100:00100000:2.0:1787739060.743613:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665653120 00000100:00100000:2.0:1787739060.743655:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3042 00000100:00100000:2.0:1787739060.743663:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ada40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:6524:x1874579665653120:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739060.743681:0:17834:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739060.743695:0:17834:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1acd8f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1acd88 expref: -99 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.743715:0:17834:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1acd8f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acd88 expref: 11 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.743726:0:17834:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1acd8f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acd88 expref: 11 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.743733:0:17834:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc1:0x0].0x0 (ffffa074419d5300) 00010000:00010000:2.0:1787739060.743736:0:17834:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739060.743740:0:17834:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c600, rc 0) 00000100:00100000:2.0:1787739060.743774:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ada40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+11:6524:x1874579665653120:12345-0@lo:101:mdt00_001.0 Request processed in 112us (221us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.743780:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3042 00000100:00100000:2.0:1787739060.743813:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356acb40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665653120:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739060.743825:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1acd88 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acd8f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.743834:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1acd88 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acd8f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739060.743851:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b4d66144 reply state 000000005f1d79b4 transno 4294968367 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1acd81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.743862:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1acd81 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.743868:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc1:0x0].0x0 (ffffa074419d5700) 00010000:00010000:2.0:1787739060.743870:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739060.743873:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787739060.743879:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1acd88 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acd8f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.743886:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1acd88 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acd8f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739060.743895:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xc1:0x0] 00000004:00010000:2.0:1787739060.743900:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b4d66144 reply state 000000005f1d79b4 transno 4294968367 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1acd7a lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x173:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.743911:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1acd7a lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x173:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.743916:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x173:0x0].0x0 (ffffa074419d5e00) 00010000:00010000:2.0:1787739060.743918:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739060.743919:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:2.0:1787739060.743924:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x173:0x0]: returning size 4096 00000004:00080000:2.0:1787739060.743931:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739060.743939:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1acd5e lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acd65 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.743946:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1acd5e lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acd65 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.743953:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1acd5e lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acd65 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.743978:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739060.743982:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739060.743985:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1acd6c lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acd73 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.743990:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1acd6c lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acd73 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.744689:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739060.744698:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1acd5e lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1acd65 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.744706:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1dc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739060.744712:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1acd5e lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1acd65 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739060.744720:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1acd5e lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1acd65 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.744747:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1acd5e lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acd65 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.744754:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739060.744772:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1acd5e lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acd65 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.744779:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1dc00/0x7ee4453e4a1acd5e lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acd65 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.744796:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739060.744798:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739060.744799:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739060.744815:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665653248:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739060.744855:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.747209:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a312c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17567:x1874579652466432:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 36893us (37063us total) trans 4294968367 rc 0/0 00000100:00100000:2.0:1787739060.747220:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23402 00000100:00100000:0.0:1787739060.747569:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665653248 00000100:00100000:0.0:1787739060.747577:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665653248 00000100:00100000:0.0:1787739060.747734:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1293 00000100:00100000:0.0:1787739060.747744:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41da40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+11:3650:x1874579665653248:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739060.747761:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739060.747784:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1acd65 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1acd5e expref: 11 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.747801:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235300) 00010000:00010000:0.0:1787739060.747810:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739060.747864:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41da40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+10:3650:x1874579665653248:12345-0@lo:103:ldlm_bl.0 Request processed in 122us (3008us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.747871:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1293 00000100:00100000:3.0:1787739060.748217:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665653248:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739060.803163:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739060.805701:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652466816 00000100:00100000:0.0:1787739060.805715:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652466816 00000100:00100000:0.0:1787739060.805795:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23403 00000100:00100000:0.0:1787739060.805806:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356acb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17568:x1874579652466816:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:0.0:1787739060.805976:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356acb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17568:x1874579652466816:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 171us (2821us total) trans 0 rc -17/-17 00000100:00100000:0.0:1787739060.805983:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23403 00000100:00100000:3.0:1787739060.824691:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739060.824767:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652466944 00000100:00100000:1.0:1787739060.824777:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652466944 00000100:00100000:1.0:1787739060.824833:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23404 00000100:00100000:1.0:1787739060.824844:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17569:x1874579652466944:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739060.824864:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739060.824885:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1acd96 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094f00 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.824951:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1acd9d lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.824975:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1acd9d lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.824987:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1acd9d lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.825007:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1acda4 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.825018:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1acd73 lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acd6c expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.825026:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1acda4 lrc: 4/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.825067:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1acd73 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1acd6c expref: 9 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739060.825084:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665653376:0@lo:104: 00000100:00100000:1.0:1787739060.825129:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739060.825181:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0efb40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.825304:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665653376 00000100:00100000:2.0:1787739060.825309:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665653376 00000100:00100000:2.0:1787739060.825348:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1139 00000100:00100000:2.0:1787739060.825355:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665653376:12345-0@lo:104: 00010000:00010000:2.0:1787739060.825367:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1acd6c lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1acd73 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.825436:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739060.825442:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665653376:12345-0@lo:104: Request processed in 88us (312us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.825448:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1139 00010000:00010000:2.0:1787739060.825468:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739060.825474:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1acd6c lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1acd73 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.825482:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739060.825487:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1acd6c lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1acd73 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739060.825496:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1acd6c lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1acd73 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.825519:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1acd6c lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1acd73 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.825524:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739060.825543:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1acd6c lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1acd73 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.825555:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a600/0x7ee4453e4a1acd6c lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1acd73 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.825572:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739060.825574:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739060.825575:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739060.825585:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356af0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665653504:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739060.825613:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.825646:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665653504 00000100:00100000:2.0:1787739060.825650:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665653504 00010000:00010000:2.0:1787739060.825668:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1acd73 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1acd6c expref: 10 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.825690:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1489 00000100:00100000:2.0:1787739060.825697:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ae940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+10:3651:x1874579665653504:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739060.825706:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787739060.827233:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665653376:0@lo:104: 00010000:00010000:1.0:1787739060.827257:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1acda4 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787739060.831183:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983379 is committed 00000001:00080000:2.0:1787739060.831213:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983378 is committed 00000020:00080000:2.0:1787739060.831221:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 368 batchid updated 00000020:00080000:2.0:1787739060.831233:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07303c6e9c0 refcount 2 committed 0 result 0 batchid 369 00000020:00080000:2.0:1787739060.831238:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743563fb00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07303c6ed80 00000020:00080000:2.0:1787739060.831242:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].366 00000020:00080000:2.0:1787739060.831245:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743563f000 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073040e9800 00000020:00080000:2.0:1787739060.831248:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].367 00000020:00080000:2.0:1787739060.831266:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 367 batchid updated 00010000:00010000:2.0:1787739060.831340:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1acd73 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1acd6c expref: 7 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.831356:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xc1169e19 (ffffa07307064f00) 00010000:00010000:2.0:1787739060.831361:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1acda4 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.831373:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1acda4 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.831381:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1acda4 lrc: 4/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.831406:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739060.831502:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ae940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+6:3651:x1874579665653504:12345-0@lo:103:ldlm_bl.0 Request processed in 5804us (5887us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.831512:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1489 00000100:00100000:2.0:1787739060.831566:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356af0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665653504:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739060.831642:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1acca1 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831651:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1acca1 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831656:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1acca1 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831662:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aae00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739060.831666:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1acca1 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831673:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1acca1 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831693:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xeb6b201b (ffffa07305d56a00) 00010000:00010000:1.0:1787739060.831696:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739060.831700:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1acca1 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831706:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1acca1 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831717:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1acca8 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1512:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831724:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1acca8 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1512:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831729:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1acca8 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1512:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831733:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739060.831737:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1acca8 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1512:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831742:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1acca8 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1512:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831750:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1512:0x0].0x0 (ffffa07305d56300) 00010000:00010000:1.0:1787739060.831752:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739060.831755:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1acca8 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1512:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831761:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1acca8 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1512:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831775:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1accaf lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831782:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1accaf lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831787:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1accaf lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831792:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739060.831794:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1accaf lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831800:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1accaf lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831807:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc3:0x0].0x0 (ffffa07305d56d00) 00010000:00010000:1.0:1787739060.831809:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739060.831813:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1accaf lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831824:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1accaf lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831839:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1acd03 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831845:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1acd03 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831871:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1acd03 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831877:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1cc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739060.831880:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1acd03 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831885:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1acd03 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831892:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xeb6b201b (ffffa07305d56a00) 00010000:00010000:1.0:1787739060.831895:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739060.831897:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1acd03 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831907:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1acd03 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xeb6b201b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831930:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1acda4 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.831941:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1acda4 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739060.832068:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665653632:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739060.832124:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739060.832155:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef240 going to sleep for 11 seconds 00000020:00080000:3.0:1787739060.832285:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 369 committed batchid 368 00000004:00080000:3.0:1787739060.832380:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07303c6e300 with version 700 00000020:00080000:3.0:1787739060.832389:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 367 cancel update log [0x240000401:0x3:0x0].364: rc = 0 00000020:00080000:3.0:1787739060.832453:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 367 cancel update log [0x200000400:0x3:0x0].365: rc = 0 00000004:00080000:3.0:1787739060.832478:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07303c6e900 with version 701 00000020:00080000:3.0:1787739060.832481:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 368 cancel update log [0x240000401:0x3:0x0].365: rc = 0 00000020:00080000:3.0:1787739060.832500:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 368 cancel update log [0x200000400:0x3:0x0].366: rc = 0 00000020:00080000:3.0:1787739060.832507:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 368 00000100:00100000:2.0:1787739060.832698:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665653632 00000100:00100000:2.0:1787739060.832705:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665653632 00000100:00100000:2.0:1787739060.832755:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3043 00000100:00100000:2.0:1787739060.832763:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7944:x1874579665653632:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739060.832842:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7944:x1874579665653632:12345-0@lo:1000:mdt00_003.0 Request processed in 80us (718us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.832849:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3043 00000004:00080000:1.0:1787739060.833600:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 700 rpc_version 700 00000020:00100000:1.0:1787739060.833627:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739060.833635:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074074f7c00 x1874579665653888/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739060.833654:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665653888:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739060.833710:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739060.833741:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739060.833837:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665653632:0@lo:1000:mdt00_003.0 00010000:00010000:1.0:1787739060.833891:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1acdab lrc: 3/1,0 mode: --/PR res: [0x240000402:0x173:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.833905:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1acdab lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x173:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.833917:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1acdab lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x173:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739060.833941:0:7944:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1acdab lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x173:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.833955:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1acd9d lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.833976:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1acd9d lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.833982:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1acd9d lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.833988:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1acd9d lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.833993:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ca00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739060.833997:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1acd9d lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.834079:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1acd9d lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.834091:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235c00) 00010000:00010000:1.0:1787739060.834094:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739060.834098:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1acd9d lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.834104:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1acd9d lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.834117:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1acda4 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.834123:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1acda4 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.834129:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1acda4 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.834135:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739060.834137:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1acda4 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.834143:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1acda4 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.834150:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xc1169e19 (ffffa07307064f00) 00010000:00010000:1.0:1787739060.834152:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739060.834155:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1acda4 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.834160:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1acda4 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.834196:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1acd96 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f00 expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.834213:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1acdab lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x173:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f00 expref: 14 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.834220:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x173:0x0].0x0 (ffffa07433235700) 00010000:00010000:1.0:1787739060.834222:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739060.834224:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c000, rc 0) 00000100:00100000:1.0:1787739060.834273:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17569:x1874579652466944:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 9431us (9602us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739060.834281:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23404 00000100:00100000:3.0:1787739060.834958:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665653888 00000100:00100000:3.0:1787739060.834977:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665653888 00000100:00100000:3.0:1787739060.835040:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3044 00000100:00100000:3.0:1787739060.835049:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665653888:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739060.835209:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665653888:12345-0@lo:1000:osp_up1-0.0 Request processed in 161us (1498us total) trans 4294968368 rc 0/0 00000100:00100000:3.0:1787739060.835216:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3044 00000100:00100000:2.0:1787739060.835356:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665653888:0@lo:1000:osp_up1-0.0 00000004:00080000:2.0:1787739060.835366:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 701 rpc_version 701 00000020:00100000:2.0:1787739060.835384:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739060.835390:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074356af0c0 x1874579665654144/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739060.835407:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356af0c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665654144:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739060.835472:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.835501:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739060.835778:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665654144 00000100:00100000:1.0:1787739060.835784:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665654144 00000100:00100000:1.0:1787739060.835825:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3045 00000100:00100000:1.0:1787739060.835832:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665654144:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739060.835925:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665654144:12345-0@lo:1000:osp_up1-0.0 Request processed in 92us (452us total) trans 4294968369 rc 0/0 00000100:00100000:1.0:1787739060.835932:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3045 00000100:00100000:1.0:1787739060.835974:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356af0c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665654144:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739060.839910:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739060.840009:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652467072 00000100:00100000:2.0:1787739060.840018:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652467072 00000100:00100000:2.0:1787739060.840083:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23405 00000100:00100000:2.0:1787739060.840094:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17569:x1874579652467072:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739060.840118:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739060.840138:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acdb2 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x173:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094f07 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.840198:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acdb9 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x173:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.840211:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acdb9 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x173:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.840222:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acdb9 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x173:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739060.840309:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x173:0x0]: returning size 4096 00000004:00200000:2.0:1787739060.840314:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739060.840347:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acdb2 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x173:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f07 expref: 11 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.840364:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acdb9 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x173:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f07 expref: 10 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.840376:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x173:0x0].0x0 (ffffa074419d5e00) 00010000:00010000:2.0:1787739060.840379:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739060.840382:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633de00, rc 0) 00000100:00100000:2.0:1787739060.840465:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17569:x1874579652467072:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 373us (563us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.840472:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23405 00000100:00100000:2.0:1787739060.850097:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739060.850217:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652467200 00000100:00100000:0.0:1787739060.850227:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652467200 00000100:00100000:0.0:1787739060.850321:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23406 00000100:00100000:0.0:1787739060.850332:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356af840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17569:x1874579652467200:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739060.850351:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739060.850370:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1acdc0 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094f0e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.850451:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acdc7 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.850471:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acdc7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.850482:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acdc7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739060.850524:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xc1:0x0]: returning size 4096 00000004:00200000:0.0:1787739060.850530:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739060.850562:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1acdc0 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f0e expref: 12 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.850577:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acdc7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f0e expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.850585:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc1:0x0].0x0 (ffffa074419d5700) 00010000:00010000:0.0:1787739060.850588:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739060.850591:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a800, rc 0) 00000100:00100000:0.0:1787739060.850646:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356af840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17569:x1874579652467200:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 316us (558us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.850653:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23406 00000100:00100000:0.0:1787739060.858676:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739060.860825:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652467328 00000100:00100000:2.0:1787739060.860838:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652467328 00000100:00100000:2.0:1787739060.860899:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23407 00000100:00100000:2.0:1787739060.860909:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17569:x1874579652467328:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739060.860929:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739060.860951:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1acdce lrc: 2/0,0 mode: --/CR res: [0x200000401:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094f15 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.861010:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1acdd5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.861026:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1acd8f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acd88 expref: 5 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.861044:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1acdd5 lrc: 4/1,0 mode: --/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.861075:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1acd8f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1acd88 expref: 5 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.861095:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ac780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665654272:0@lo:104: 00000100:00100000:2.0:1787739060.861155:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.861195:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.861220:0:6507:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:2.0:1787739060.861230:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665654272 00000100:00100000:2.0:1787739060.861233:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665654272 00000100:00100000:2.0:1787739060.861259:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1140 00000100:00100000:2.0:1787739060.861265:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f4ec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665654272:12345-0@lo:104: 00010000:00010000:2.0:1787739060.861276:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1acd88 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1acd8f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.863785:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739060.863792:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ac780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665654272:0@lo:104: 00000100:00100000:2.0:1787739060.863797:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f4ec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665654272:12345-0@lo:104: Request processed in 2531us (2639us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.863807:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1140 00010000:00010000:3.0:1787739060.864028:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1acdd5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.864058:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739060.864065:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1acd88 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1acd8f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.864072:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633da00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739060.864078:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1acd88 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1acd8f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739060.864087:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1acd88 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1acd8f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.864123:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1acd88 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1acd8f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.864130:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1787739060.864188:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665654400:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739060.864239:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787739060.864285:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1acd88 lrc: 1/0,0 mode: --/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1acd8f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.864292:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1acd88 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1acd8f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.864319:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739060.864322:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739060.864323:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739060.864492:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665654400 00000100:00100000:3.0:1787739060.864498:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665654400 00010000:00010000:3.0:1787739060.864523:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1acd8f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1acd88 expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.864550:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1490 00000100:00100000:3.0:1787739060.864558:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356acf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+6:3652:x1874579665654400:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739060.864570:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00080000:1.0:1787739060.869008:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 369 batchid updated 00000020:00080000:0.0:1787739060.869058:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 369 00000004:00080000:0.0:1787739060.869133:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306241900 with version 702 00000020:00080000:0.0:1787739060.869145:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 369 cancel update log [0x240000401:0x3:0x0].366: rc = 0 00010000:00010000:1.0:1787739060.869152:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1acd8f lrc: 0/0,0 mode: --/PW res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1acd88 expref: 6 pid: 17834 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.869171:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc1:0x0].0x0 (ffffa074419d5300) 00010000:00010000:1.0:1787739060.869176:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1acdd5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.869187:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1acdd5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.869196:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1acdd5 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:0.0:1787739060.869217:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 369 cancel update log [0x200000400:0x3:0x0].367: rc = 0 00010000:00010000:1.0:1787739060.869220:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000020:00080000:0.0:1787739060.869226:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 369 00000100:00100000:1.0:1787739060.869294:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356acf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+5:3652:x1874579665654400:12345-0@lo:103:ldlm_bl.0 Request processed in 4735us (5053us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739060.869305:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1490 00000004:00080000:1.0:1787739060.869334:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 702 rpc_version 702 00010000:00010000:3.0:1787739060.869340:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1acdd5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.869351:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1acdd5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787739060.869363:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739060.869372:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303a30f00 x1874579665654656/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739060.869398:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a30f00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665654656:0@lo:1000:osp_up1-0.0 00000004:00200000:3.0:1787739060.869409:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xc1:0x0]: returning size 4096 00000004:00200000:3.0:1787739060.869473:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739060.869515:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1acdce lrc: 0/0,0 mode: --/CR res: [0x200000401:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f15 expref: 16 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.869532:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1acdd5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f15 expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.869540:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc1:0x0].0x0 (ffffa074419d5300) 00010000:00010000:3.0:1787739060.869542:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739060.869545:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c200, rc 0) 00000100:00100000:3.0:1787739060.869648:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17569:x1874579652467328:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 8740us (10980us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739060.869655:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23407 00000100:00100000:2.0:1787739060.869728:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665654400:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739060.871935:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739060.871978:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef000 going to sleep for 11 seconds 00000100:00100000:3.0:1787739060.872200:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665654656 00000100:00100000:3.0:1787739060.872210:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665654656 00000100:00100000:3.0:1787739060.872262:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3046 00000100:00100000:3.0:1787739060.872271:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665654656:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739060.873894:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665654656:12345-0@lo:1000:osp_up1-0.0 Request processed in 1615us (1955us total) trans 4294968370 rc 0/0 00000100:00100000:3.0:1787739060.873908:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3046 00000100:00100000:3.0:1787739060.875175:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739060.875929:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652467456 00000100:00100000:1.0:1787739060.875938:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652467456 00000100:00100000:2.0:1787739060.875960:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a30f00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665654656:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739060.876008:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1294 00000100:00100000:1.0:1787739060.876019:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074184c3850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2408:x1874579652467456:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:1.0:1787739060.876048:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739060.876074:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1acdab lrc: 0/0,0 mode: --/PR res: [0x240000402:0x173:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f00 expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.876091:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x173:0x0].0x0 (ffffa07433235700) 00010000:00010000:1.0:1787739060.876103:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739060.876158:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074184c3850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2408:x1874579652467456:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 141us (988us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739060.876165:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1294 00000100:00100000:2.0:1787739060.882132:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739060.882250:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652467712 00000100:00100000:3.0:1787739060.882259:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652467712 00000100:00100000:3.0:1787739060.882317:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23408 00000100:00100000:3.0:1787739060.882326:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17569:x1874579652467712:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:3.0:1787739060.882378:0:6525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739060.882404:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acdb9 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x173:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f07 expref: 11 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.882453:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x173:0x0].0x0 (ffffa074419d5e00) 00010000:00010000:3.0:1787739060.882458:0:6525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739060.882505:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1acddc lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.882513:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1acddc lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.882523:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1acddc lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.882544:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665654784:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1787739060.882594:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739060.882630:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6eb40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.882695:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665654784 00000100:00100000:2.0:1787739060.882699:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665654784 00000100:00100000:2.0:1787739060.882733:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3047 00000100:00100000:2.0:1787739060.882740:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+5:6525:x1874579665654784:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787739060.882749:0:8448:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739060.882764:0:8448:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1acde3 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1acddc expref: -99 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.882786:0:8448:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1acde3 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acddc expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.882799:0:8448:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1acde3 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acddc expref: 6 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.882816:0:8448:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78000) 00010000:00010000:2.0:1787739060.882819:0:8448:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739060.882822:0:8448:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633ce00, rc 0) 00000100:00100000:2.0:1787739060.882861:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:6525:x1874579665654784:12345-0@lo:101:mdt00_002.0 Request processed in 122us (266us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739060.882868:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3047 00000100:00100000:3.0:1787739060.883112:0:6525:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983379 gen 2 00000004:00080000:3.0:1787739060.883118:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983375 committed transno 4294983379 00000020:00080000:3.0:1787739060.883134:0:6525:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07302e89180 refcount 2 committed 0 result 0 batchid 360 00000020:00080000:3.0:1787739060.883138:0:6525:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2a00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07302e89600 00000020:00080000:3.0:1787739060.883141:0:6525:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].353 00000020:00080000:3.0:1787739060.883143:0:6525:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2d00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073069be800 00000020:00080000:3.0:1787739060.883146:0:6525:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].353 00000004:00080000:3.0:1787739060.883175:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983377 committed transno 4294983379 00000004:00080000:3.0:1787739060.883188:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983379 committed transno 4294983379 00000100:00100000:3.0:1787739060.883207:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665654784:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739060.883222:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1acddc lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acde3 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.883232:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1acddc lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acde3 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739060.883245:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739060.883265:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa600/0x7ee4453e4a1acdea lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.883272:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa600/0x7ee4453e4a1acdea lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.883279:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa600/0x7ee4453e4a1acdea lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.883290:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665654912:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1787739060.883315:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739060.883332:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6eb40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739060.883406:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665654912 00000100:00100000:1.0:1787739060.883424:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665654912 00000100:00100000:1.0:1787739060.883453:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3048 00000100:00100000:1.0:1787739060.883461:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+6:6525:x1874579665654912:12345-0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787739060.883470:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000020:00080000:0.0:1787739060.883477:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 360 committed batchid 358 00010000:00010000:1.0:1787739060.883485:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1acdf1 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1acdea expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.883505:0:8447:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1acdf1 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acdea expref: 7 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.883521:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1acdf1 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acdea expref: 7 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.883528:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xc1169e19 (ffffa07433235c00) 00010000:00010000:1.0:1787739060.883531:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000020:00080000:0.0:1787739060.883533:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 358 00010000:00010000:1.0:1787739060.883534:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1ce00, rc 0) 00000100:00100000:1.0:1787739060.883582:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6525:x1874579665654912:12345-0@lo:101:mdt00_002.0 Request processed in 122us (266us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739060.883589:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3048 00000100:00100000:0.0:1787739060.883826:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665654912:0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787739060.883838:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa600/0x7ee4453e4a1acdea lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acdf1 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.883848:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa600/0x7ee4453e4a1acdea lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acdf1 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739060.883861:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:0.0:1787739060.883893:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561e940 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665655040:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739060.883923:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739060.883951:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739060.884079:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665655040 00000100:00100000:3.0:1787739060.884083:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665655040 00000100:00100000:3.0:1787739060.884120:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3049 00000100:00100000:3.0:1787739060.884127:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6525:x1874579665655040:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739060.884187:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a940 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6525:x1874579665655040:12345-0@lo:1000:mdt00_002.0 Request processed in 60us (262us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739060.884194:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3049 00000100:00100000:3.0:1787739060.884217:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561e940 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665655040:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739060.884252:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c3c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665655168:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739060.884272:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739060.884287:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6ed80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.884383:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665655168 00000100:00100000:0.0:1787739060.884387:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665655168 00000100:00100000:0.0:1787739060.884443:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3050 00000100:00100000:0.0:1787739060.884450:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6525:x1874579665655168:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739060.884517:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6525:x1874579665655168:12345-0@lo:1000:mdt00_002.0 Request processed in 67us (243us total) trans 0 rc 1/1 00000100:00100000:0.0:1787739060.884524:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3050 00000100:00100000:2.0:1787739060.884600:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c3c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665655168:0@lo:1000:mdt00_002.0 00010000:00010000:2.0:1787739060.884644:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acdf8 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x173:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.884652:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acdf8 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x173:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.884661:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acdf8 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x173:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.884673:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ad680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665655296:0@lo:101:mdt00_002.0 00000100:00100000:2.0:1787739060.884709:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.884731:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.884870:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665655296 00000100:00100000:0.0:1787739060.884875:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665655296 00000100:00100000:0.0:1787739060.884902:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3051 00000100:00100000:0.0:1787739060.884909:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356aed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+7:6525:x1874579665655296:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787739060.884918:0:18535:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739060.884930:0:18535:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acdff lrc: 2/0,0 mode: --/EX res: [0x240000402:0x173:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1acdf8 expref: -99 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.884952:0:18535:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acdff lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x173:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acdf8 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.884963:0:18535:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acdff lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x173:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acdf8 expref: 8 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.884972:0:18535:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x173:0x0].0x0 (ffffa07307064100) 00010000:00010000:0.0:1787739060.884975:0:18535:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739060.884978:0:18535:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609aa00, rc 0) 00000100:00100000:0.0:1787739060.885014:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356aed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6525:x1874579665655296:12345-0@lo:101:mdt00_002.0 Request processed in 106us (304us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.885021:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3051 00000100:00100000:2.0:1787739060.885607:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ad680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665655296:0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787739060.885622:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acdf8 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x173:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acdff expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.885632:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acdf8 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x173:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acdff expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.885658:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ace06 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x173:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.885666:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ace06 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x173:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.885674:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ace06 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x173:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.885703:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ace0d lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.885715:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acdc7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f0e expref: 10 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.885724:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ace0d lrc: 4/0,1 mode: --/EX res: [0x240000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.885747:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acdc7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f0e expref: 10 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.885761:0:6525:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acdc7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f0e expref: 10 pid: 6523 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.885772:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ad680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665655424:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739060.885845:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d900 going to sleep for 11 seconds 00000100:00100000:2.0:1787739060.889167:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ad680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665655424:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739060.889194:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ace0d lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.891550:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739060.891627:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652467840 00000100:00100000:0.0:1787739060.891637:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652467840 00010000:00010000:0.0:1787739060.891685:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acdc7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f0e expref: 11 pid: 6523 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739060.891719:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1491 00000100:00100000:0.0:1787739060.891727:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ad680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2408:x1874579652467840:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739060.891738:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739060.891743:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acdc7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f0e expref: 11 pid: 6523 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.891752:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acdc7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f0e expref: 11 pid: 6523 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.891774:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acdc7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f0e expref: 11 pid: 6523 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.891785:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc1:0x0].0x0 (ffffa074419d5700) 00010000:00010000:0.0:1787739060.891789:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ace0d lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.891795:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ace0d lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.891808:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ace0d lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.891827:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739060.891870:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ad680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652467840:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 144us (329us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.891875:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1491 00010000:00010000:2.0:1787739060.891946:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ace0d lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.891952:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ace0d lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.891992:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ace14 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.891999:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ace14 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.892009:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ace14 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739060.892028:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ad2c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665655552:0@lo:101:mdt00_002.0 00000100:00100000:2.0:1787739060.892080:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739060.892116:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.892159:0:8447:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665655552 00000100:00100000:0.0:1787739060.892164:0:8447:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665655552 00000100:00100000:0.0:1787739060.892199:0:8447:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3052 00000100:00100000:0.0:1787739060.892206:0:8447:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ae580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+8:6525:x1874579665655552:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787739060.892221:0:8447:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739060.892234:0:8447:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ace1b lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ace14 expref: -99 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.892250:0:8447:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1acdd5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f15 expref: 13 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.892259:0:8447:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ace1b lrc: 4/0,0 mode: --/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ace14 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.892275:0:8447:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1acdd5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f15 expref: 13 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.892283:0:8447:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1acdd5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f15 expref: 13 pid: 7944 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739060.892291:0:8447:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7a940 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8447:1874579665655680:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739060.892334:0:8447:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241600 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.905345:0:8447:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7a940 pname:cluuid:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000_UUID:8447:1874579665655680:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739060.905386:0:8447:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ace1b lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ace14 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.905400:0:8447:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc1:0x0].0x0 (ffffa074419d5300) 00010000:00010000:0.0:1787739060.905403:0:8447:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739060.905448:0:8447:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ace1b lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ace14 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.905458:0:8447:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ace1b lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ace14 expref: 9 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.905466:0:8447:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b200, rc 0) 00000100:00100000:0.0:1787739060.905526:0:8447:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ae580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6525:x1874579665655552:12345-0@lo:101:mdt00_002.0 Request processed in 13320us (13445us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.905535:0:8447:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3052 00000100:00100000:2.0:1787739060.906051:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ad2c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665655552:0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787739060.906072:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ace14 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ace1b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739060.906083:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ace14 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ace1b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.908267:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739060.908306:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652467968 00000100:00100000:3.0:1787739060.908315:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652467968 00010000:00010000:3.0:1787739060.908352:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1acdd5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f15 expref: 14 pid: 7944 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.908385:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1492 00000100:00100000:3.0:1787739060.908395:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652467968:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739060.908409:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739060.908431:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1acdd5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f15 expref: 14 pid: 7944 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.908444:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1acdd5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f15 expref: 14 pid: 7944 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.908467:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1acdd5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f15 expref: 14 pid: 7944 timeout: 747 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.908490:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc1:0x0].0x0 (ffffa074419d5300) 00010000:00010000:3.0:1787739060.908497:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ace1b lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ace14 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.908506:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ace1b lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ace14 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.908515:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ace1b lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ace14 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.908547:0:6508:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ace1b lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ace14 expref: 8 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.908560:0:6508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561e940 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665655808:0@lo:105: 00000100:00100000:3.0:1787739060.908604:0:6508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739060.908646:0:6508:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e000 going to sleep for 11 seconds 00000100:00100000:3.0:1787739060.908662:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665655808 00000100:00100000:3.0:1787739060.908665:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665655808 00000100:00100000:3.0:1787739060.908698:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1141 00000100:00100000:3.0:1787739060.908706:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415700b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665655808:12345-0@lo:105: 00010000:00010000:3.0:1787739060.908715:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ace14 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ace1b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.908724:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ace14 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ace1b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.908735:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ace14 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ace1b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.908741:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ace14 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ace1b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.908747:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ace14 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ace1b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.908754:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ace14 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ace1b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.908773:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633dc00) 00000100:00100000:3.0:1787739060.908816:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415700b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665655808:12345-0@lo:105: Request processed in 112us (212us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739060.908823:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1141 00000100:00100000:3.0:1787739060.908863:0:6508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561e940 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665655808:0@lo:105: 00010000:00010000:3.0:1787739060.908877:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739060.908917:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:2409:x1874579652467968:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 524us (668us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739060.908923:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1492 00010000:00010000:3.0:1787739060.911513:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ace14 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ace1b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.911536:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ace14 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ace1b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.915508:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415701e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665655936:0@lo:403:mdt00_002.0 00000100:00100000:3.0:1787739060.915588:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739060.915629:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e000 going to sleep for 11 seconds 00000100:00100000:0.0:1787739060.916221:0:18535:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665655936 00000100:00100000:0.0:1787739060.916229:0:18535:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665655936 00000100:00100000:0.0:1787739060.916296:0:18535:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3053 00000100:00100000:0.0:1787739060.916307:0:18535:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415701a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6525:x1874579665655936:12345-0@lo:403:mdt00_002.0 00000100:00100000:0.0:1787739060.920107:0:18535:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415701a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:6525:x1874579665655936:12345-0@lo:403:mdt00_002.0 Request processed in 3792us (4510us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.920123:0:18535:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3053 00000100:00100000:3.0:1787739060.920271:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415701e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665655936:0@lo:403:mdt00_002.0 00000040:00080000:3.0:1787739060.921629:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].355, 664 off538736 00000040:00080000:3.0:1787739060.922130:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].355, 744 off588752 00000004:00080000:3.0:1787739060.922137:0:6525:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07303c6e000 with version 676 00000004:00080000:0.0:1787739060.922231:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 676 rpc_version 676 00000020:00100000:0.0:1787739060.922331:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739060.922342:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07305f7a940 x1874579665656192/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1808/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739060.922365:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7a940 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665656192:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739060.922545:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739060.922589:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241600 going to sleep for 11 seconds 00000100:00100000:1.0:1787739060.922643:0:8448:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665656192 00000100:00100000:1.0:1787739060.922650:0:8448:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665656192 00000100:00100000:1.0:1787739060.922714:0:8448:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3054 00000100:00100000:1.0:1787739060.922724:0:8448:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665656192:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739060.925503:0:8448:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665656192:12345-0@lo:1000:osp_up0-1.0 Request processed in 2775us (2952us total) trans 4294983380 rc 0/0 00000100:00100000:1.0:1787739060.925518:0:8448:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3054 00000100:00100000:0.0:1787739060.926001:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7a940 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665656192:0@lo:1000:osp_up0-1.0 00000004:00200000:3.0:1787739060.926067:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x173:0x0]: returning size 4096 00010000:00010000:3.0:1787739060.926090:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ace0d lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.926100:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ace0d lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.926107:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ace0d lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.926114:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739060.926119:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ace0d lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.926127:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ace0d lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.926141:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc1:0x0].0x0 (ffffa074419d5700) 00010000:00010000:3.0:1787739060.926144:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739060.926148:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ace0d lrc: 2/0,0 mode: --/EX res: [0x240000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.926157:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ace0d lrc: 0/0,0 mode: --/EX res: [0x240000400:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.926171:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ace14 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ace1b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.926179:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ace14 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ace1b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.926185:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ace14 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ace1b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.926234:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739060.926244:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xc1:0x0] 00010000:00010000:3.0:1787739060.926250:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ace06 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x173:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.926266:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ace06 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x173:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.926272:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ace06 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x173:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.926277:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739060.926279:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ace06 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x173:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.926285:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ace06 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x173:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.926297:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x173:0x0].0x0 (ffffa074419d5e00) 00010000:00010000:3.0:1787739060.926299:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739060.926302:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ace06 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x173:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.926307:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ace06 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x173:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.926316:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acdf8 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x173:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acdff expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.926324:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acdf8 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x173:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acdff expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.926331:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acdf8 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x173:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acdff expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.926402:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1787739060.926408:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ace14 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ace1b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.926437:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633dc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739060.926442:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ace14 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ace1b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739060.926450:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ace14 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ace1b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.926495:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ace14 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ace1b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.926502:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739060.926533:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ace14 lrc: 1/0,0 mode: --/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ace1b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.926540:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633dc00/0x7ee4453e4a1ace14 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ace1b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.926557:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739060.926560:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acdf8 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x173:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1acdff expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.926567:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739060.926570:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acdf8 lrc: 5/0,0 mode: EX/EX res: [0x240000402:0x173:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1acdff expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739060.926577:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acdf8 lrc: 5/0,0 mode: EX/EX res: [0x240000402:0x173:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1acdff expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.926594:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acdf8 lrc: 4/0,0 mode: --/EX res: [0x240000402:0x173:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acdff expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.926600:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739060.926616:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acdf8 lrc: 3/0,0 mode: --/EX res: [0x240000402:0x173:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acdff expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.926639:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739060.926640:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739060.926642:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739060.926651:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787739060.926673:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665656320:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739060.926727:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739060.926762:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665656448:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739060.926784:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739060.926840:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665656320 00000100:00100000:1.0:1787739060.926845:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665656320 00000100:00100000:1.0:1787739060.926881:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665656448 00000100:00100000:1.0:1787739060.926884:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665656448 00000100:00100000:1.0:1787739060.926902:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1295 00000100:00100000:1.0:1787739060.926910:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+11:3652:x1874579665656320:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739060.926922:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739060.926939:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ace1b lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ace14 expref: 11 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.926950:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc1:0x0].0x0 (ffffa074419d5300) 00010000:00010000:1.0:1787739060.926957:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739060.926987:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+10:3652:x1874579665656320:12345-0@lo:103:ldlm_bl.0 Request processed in 77us (259us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739060.926994:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1295 00000100:00100000:1.0:1787739060.927009:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1296 00000100:00100000:1.0:1787739060.927015:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3650:x1874579665656448:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739060.927022:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739060.927044:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acdff lrc: 0/0,0 mode: --/EX res: [0x240000402:0x173:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1acdf8 expref: 9 pid: 18535 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.927053:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x173:0x0].0x0 (ffffa07307064100) 00010000:00010000:1.0:1787739060.927060:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739060.927085:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+8:3650:x1874579665656448:12345-0@lo:103:ldlm_bl.0 Request processed in 71us (300us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739060.927092:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1296 00000100:00100000:1.0:1787739060.927128:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665656448:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739060.927152:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665656320:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739060.928118:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:3.0:1787739060.928127:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1acdf8 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x173:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acdff expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739060.928161:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739060.928166:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1acddc lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acde3 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.928173:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1acddc lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acde3 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.928180:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1acddc lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acde3 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.928211:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739060.928218:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1acddc lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1acde3 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.928226:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739060.928231:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1acddc lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1acde3 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739060.928239:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1acddc lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1acde3 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.928343:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1acddc lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acde3 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.928351:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739060.928376:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1acddc lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acde3 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.928386:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739060.928387:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739060.928389:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739060.928449:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665656576:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739060.928495:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787739060.928619:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:3.0:1787739060.928625:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1acddc lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acde3 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739060.928640:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739060.928646:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa600/0x7ee4453e4a1acdea lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acdf1 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.928662:0:6525:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa600/0x7ee4453e4a1acdea lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acdf1 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739060.928738:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17569:x1874579652467712:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 46411us (46613us total) trans 4294968371 rc 0/0 00000100:00100000:3.0:1787739060.928746:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23408 00010000:00010000:1.0:1787739060.929083:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739060.935014:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665656576 00000100:00100000:0.0:1787739060.935028:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665656576 00000100:00100000:0.0:1787739060.935110:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1297 00000100:00100000:0.0:1787739060.935120:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3651:x1874579665656576:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739060.935136:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739060.935160:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1acde3 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1acddc expref: 8 pid: 8448 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.935181:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78000) 00010000:00010000:0.0:1787739060.935203:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739060.935274:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+7:3651:x1874579665656576:12345-0@lo:103:ldlm_bl.0 Request processed in 156us (6778us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739060.935282:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1297 00000100:00100000:1.0:1787739060.935442:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665656576:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739060.964184:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739060.964242:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652468096 00000100:00100000:0.0:1787739060.964252:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652468096 00000100:00100000:0.0:1787739060.964310:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23409 00000100:00100000:0.0:1787739060.964319:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17570:x1874579652468096:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739060.964341:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739060.964365:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ace22 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094f1c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.964461:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ace29 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.964475:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ace29 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.964486:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ace29 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.964498:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ace30 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.964504:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ace30 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.964512:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ace30 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.964549:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ace29 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.964556:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ace29 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.964563:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ace29 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.964569:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ace29 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.964587:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739060.964591:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ace29 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.964598:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ace29 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.964610:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07307064000) 00010000:00010000:0.0:1787739060.964613:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739060.964617:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ace29 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.964622:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ace29 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.964632:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ace30 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.964638:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ace30 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.964643:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ace30 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.964648:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739060.964650:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ace30 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.964782:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ace30 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.964796:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x8f0f038c (ffffa07307064800) 00010000:00010000:0.0:1787739060.964798:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739060.964802:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ace30 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.964808:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ace30 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.964839:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ace22 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f1c expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.964850:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07307064000) 00010000:00010000:0.0:1787739060.964852:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739060.964855:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ace22 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f1c expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739060.964865:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609ba00, rc 0) 00000100:00100000:0.0:1787739060.964925:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17570:x1874579652468096:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 606us (749us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739060.964933:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23409 00000100:00100000:3.0:1787739060.973544:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739060.976238:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652468224 00000100:00100000:1.0:1787739060.976252:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652468224 00000100:00100000:1.0:1787739060.976317:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23410 00000100:00100000:1.0:1787739060.976330:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415701e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17570:x1874579652468224:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739060.976350:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739060.976629:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ace37 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094f23 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.976812:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ace3e lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.976826:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ace3e lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.976838:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ace3e lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.976851:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ace45 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.976858:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ace45 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.976864:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ace45 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.976906:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ace3e lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.976914:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ace3e lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.976919:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ace3e lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.976926:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ace3e lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.976932:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739060.976936:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ace3e lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.976943:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ace3e lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.976955:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235700) 00010000:00010000:1.0:1787739060.977020:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739060.977087:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ace3e lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.977095:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ace3e lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.977108:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ace45 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.977114:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ace45 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.977120:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ace45 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.977126:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739060.977129:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ace45 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.977134:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ace45 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.977142:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x8f0f038c (ffffa07433235800) 00010000:00010000:1.0:1787739060.977145:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739060.977148:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ace45 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.977153:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ace45 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.977180:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ace37 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f23 expref: 14 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.977278:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235700) 00010000:00010000:1.0:1787739060.977280:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739060.977284:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ace37 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f23 expref: 14 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739060.977295:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c200, rc 0) 00000100:00100000:1.0:1787739060.981362:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415701e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17570:x1874579652468224:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 5023us (7813us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739060.981386:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23410 00000100:00100000:0.0:1787739060.992912:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739060.993490:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652468352 00000100:00100000:3.0:1787739060.993502:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652468352 00000100:00100000:3.0:1787739060.993554:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23411 00000100:00100000:3.0:1787739060.993564:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17570:x1874579652468352:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739060.993594:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739060.993616:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ace4c lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094f2a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.993684:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ace53 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.993697:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ace53 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.993708:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ace53 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739060.993755:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:3.0:1787739060.993761:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739060.993793:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ace4c lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f2a expref: 15 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.993811:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ace53 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f2a expref: 14 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739060.993819:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305d56f00) 00010000:00010000:3.0:1787739060.993822:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739060.993826:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab000, rc 0) 00000100:00100000:3.0:1787739060.993883:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17570:x1874579652468352:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 320us (980us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739060.993890:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23411 00000100:00100000:3.0:1787739061.003059:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739061.003096:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652468608 00000100:00100000:3.0:1787739061.003107:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652468608 00000100:00100000:3.0:1787739061.003186:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23412 00000100:00100000:3.0:1787739061.003198:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17570:x1874579652468608:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:3.0:1787739061.003281:0:6524:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739061.003319:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ace53 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f2a expref: 14 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.003337:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305d56f00) 00010000:00010000:3.0:1787739061.003353:0:6524:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739061.003439:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ace5a lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787739061.003445:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1787739061.003453:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ace5a lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.003463:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ace5a lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.003475:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ace61 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.003490:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ace61 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.003497:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ace61 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787739061.005082:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].369, 2168 off564888 00000040:00080000:3.0:1787739061.005162:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].368, 2368 off611152 00000004:00080000:3.0:1787739061.005170:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07303c6e9c0 with version 703 00000004:00080000:0.0:1787739061.005325:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 703 rpc_version 703 00000020:00100000:0.0:1787739061.005362:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739061.005385:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743562f840 x1874579665656832/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 4232/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739061.005407:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562f840 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665656832:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739061.005491:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739061.005540:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241180 going to sleep for 11 seconds 00000100:00100000:3.0:1787739061.007221:0:17834:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665656832 00000100:00100000:3.0:1787739061.007233:0:17834:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665656832 00000100:00100000:3.0:1787739061.007294:0:17834:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3055 00000100:00100000:3.0:1787739061.007304:0:17834:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c42cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665656832:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739061.008514:0:17834:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c42cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_out00_00:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665656832:12345-0@lo:1000:osp_up1-0.0 Request processed in 1208us (3018us total) trans 4294968372 rc 0/0 00000100:00100000:3.0:1787739061.008525:0:17834:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3055 00000100:00100000:0.0:1787739061.008812:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562f840 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665656832:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787739061.008923:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ace68 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1515:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.008939:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ace68 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1515:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.008950:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ace68 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1515:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.008968:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ace6f lrc: 3/0,1 mode: --/PW res: [0x200000400:0xc4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.008974:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ace6f lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.008981:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ace6f lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.009011:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ace76 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.009018:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ace76 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.009043:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ace76 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739061.009062:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665656960:0@lo:101:mdt00_001.0 00000100:00100000:0.0:1787739061.009097:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739061.009129:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241180 going to sleep for 11 seconds 00000100:00100000:0.0:1787739061.009146:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665656960 00000100:00100000:0.0:1787739061.009151:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665656960 00000100:00100000:0.0:1787739061.009184:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3084 00000100:00100000:0.0:1787739061.009205:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:6524:x1874579665656960:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739061.009221:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739061.009236:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ace7d lrc: 2/0,0 mode: --/PW res: [0x240000401:0xc4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ace76 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.009255:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ace7d lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ace76 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.009267:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ace7d lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ace76 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.009276:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc4:0x0].0x0 (ffffa07307064600) 00010000:00010000:0.0:1787739061.009279:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739061.009282:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b000, rc 0) 00000100:00100000:0.0:1787739061.009311:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+11:6524:x1874579665656960:12345-0@lo:101:mdt00_001.0 Request processed in 121us (214us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739061.009317:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3084 00000100:00100000:0.0:1787739061.009348:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665656960:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739061.009359:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ace76 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ace7d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.009367:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ace76 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ace7d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739061.009388:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c3ae7c07 reply state 000000003ce659c1 transno 4294983381 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ace6f lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.009399:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ace6f lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.009406:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc4:0x0].0x0 (ffffa07307064900) 00010000:00010000:0.0:1787739061.009409:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739061.009429:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739061.009435:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ace76 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ace7d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.009442:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ace76 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ace7d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739061.009450:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xc4:0x0] 00000004:00010000:0.0:1787739061.009456:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c3ae7c07 reply state 000000003ce659c1 transno 4294983381 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ace68 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1515:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.009463:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ace68 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1515:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.009467:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1515:0x0].0x0 (ffffa07307064800) 00010000:00010000:0.0:1787739061.009469:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739061.009470:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787739061.009475:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1515:0x0]: returning size 4096 00010000:00010000:0.0:1787739061.009482:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ace5a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.009488:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ace5a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.009493:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ace5a lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.009499:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ace5a lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.009505:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aba00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739061.009509:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ace5a lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.009515:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ace5a lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.009524:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305d56000) 00010000:00010000:0.0:1787739061.009526:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739061.009529:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ace5a lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.009533:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ace5a lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739061.009548:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c3ae7c07 reply state 000000003ce659c1 transno 4294983381 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ace61 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.009555:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ace61 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.009561:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x8f0f038c (ffffa07305d56500) 00010000:00010000:0.0:1787739061.009563:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739061.009564:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 2 00000100:00100000:0.0:1787739061.009795:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17570:x1874579652468608:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 6597us (6744us total) trans 4294983381 rc 0/0 00000100:00100000:0.0:1787739061.009803:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23412 00000100:00100000:1.0:1787739061.051011:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739061.051112:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652468864 00000100:00100000:3.0:1787739061.051123:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652468864 00000100:00100000:3.0:1787739061.051172:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23413 00000100:00100000:3.0:1787739061.051179:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17571:x1874579652468864:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:3.0:1787739061.051767:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17571:x1874579652468864:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 586us (761us total) trans 0 rc -17/-17 00000100:00100000:3.0:1787739061.051775:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23413 00000100:00100000:0.0:1787739061.084482:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739061.084646:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652468992 00000100:00100000:3.0:1787739061.084656:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652468992 00000100:00100000:3.0:1787739061.084712:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23414 00000100:00100000:3.0:1787739061.084722:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17572:x1874579652468992:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739061.084746:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739061.084769:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ace84 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094f31 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.084828:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ace8b lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.084841:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ace8b lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.084854:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ace8b lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.084864:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ace92 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.084872:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ace92 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.084879:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ace92 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.084953:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ace99 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1515:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.084962:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ace99 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1515:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.084968:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ace99 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1515:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739061.085066:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1515:0x0]: returning size 4096 00000004:00200000:3.0:1787739061.085072:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:3.0:1787739061.085090:0:6523:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ace99 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1515:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.085102:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ace8b lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.085109:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ace8b lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.085115:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ace8b lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.085122:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ace8b lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.085128:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739061.085132:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ace8b lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.085140:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ace8b lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.085153:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305d56f00) 00010000:00010000:3.0:1787739061.085156:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739061.085160:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ace8b lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.085167:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ace8b lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.085180:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ace92 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.085186:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ace92 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.085191:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ace92 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.085195:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739061.085197:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ace92 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.085203:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ace92 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.085209:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x8f0f038c (ffffa07305d56500) 00010000:00010000:3.0:1787739061.085211:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739061.085214:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ace92 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.085219:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ace92 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.085246:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ace84 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f31 expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.085262:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ace99 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1515:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f31 expref: 16 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.085269:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1515:0x0].0x0 (ffffa07307064800) 00010000:00010000:3.0:1787739061.085271:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787739061.085274:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab800, rc 0) 00000100:00100000:3.0:1787739061.085335:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17572:x1874579652468992:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 614us (860us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.085343:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23414 00000100:00100000:1.0:1787739061.090206:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739061.090278:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652469120 00000100:00100000:0.0:1787739061.090288:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652469120 00000100:00100000:0.0:1787739061.090349:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23415 00000100:00100000:0.0:1787739061.090360:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17572:x1874579652469120:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739061.090392:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739061.090446:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1acea0 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xc4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094f38 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.090530:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1acea7 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xc4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.090544:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1acea7 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.090556:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1acea7 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739061.090608:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xc4:0x0]: returning size 4096 00000004:00200000:0.0:1787739061.090614:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739061.090647:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1acea0 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f38 expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.090663:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1acea7 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f38 expref: 17 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.090672:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc4:0x0].0x0 (ffffa07307064900) 00010000:00010000:0.0:1787739061.090675:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739061.090693:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609be00, rc 0) 00000100:00100000:0.0:1787739061.090748:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17572:x1874579652469120:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 391us (552us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739061.090755:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23415 00000100:00100000:3.0:1787739061.096995:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739061.097186:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652469248 00000100:00100000:1.0:1787739061.097195:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652469248 00000100:00100000:1.0:1787739061.097271:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23416 00000100:00100000:1.0:1787739061.097282:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17572:x1874579652469248:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739061.097305:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739061.097326:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1aceae lrc: 2/0,0 mode: --/CR res: [0x240000401:0xc4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094f3f expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.097384:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aceb5 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.097398:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ace7d lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ace76 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.097407:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aceb5 lrc: 4/1,0 mode: --/PR res: [0x240000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.097468:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ace7d lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ace76 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739061.097482:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383de00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665657088:0@lo:104: 00000100:00100000:1.0:1787739061.097524:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739061.097562:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef240 going to sleep for 11 seconds 00000100:00100000:1.0:1787739061.097576:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665657088 00000100:00100000:1.0:1787739061.097580:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665657088 00000100:00100000:1.0:1787739061.097601:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1142 00000100:00100000:1.0:1787739061.097607:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383da40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665657088:12345-0@lo:104: 00010000:00010000:1.0:1787739061.097618:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ace76 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ace7d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.097674:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00080000:2.0:1787739061.097759:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787739061.097836:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383da40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665657088:12345-0@lo:104: Request processed in 227us (310us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739061.097843:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1142 00000100:00100000:1.0:1787739061.097884:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383de00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665657088:0@lo:104: 00010000:00010000:1.0:1787739061.097899:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aceb5 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.097967:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739061.097974:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ace76 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ace7d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.097981:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ac00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739061.097987:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ace76 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xc4:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ace7d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739061.097996:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ace76 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xc4:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ace7d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.098020:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ace76 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xc4:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ace7d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.098137:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739061.098163:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ace76 lrc: 1/0,0 mode: --/PW res: [0x240000401:0xc4:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ace7d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.098171:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609ac00/0x7ee4453e4a1ace76 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xc4:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ace7d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.098190:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739061.098191:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739061.098193:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739061.098221:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787739061.098232:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665657216:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739061.098263:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739061.098324:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665657216 00000100:00100000:3.0:1787739061.098329:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665657216 00010000:00010000:3.0:1787739061.098347:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ace7d lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ace76 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739061.098405:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1493 00000100:00100000:3.0:1787739061.098429:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ade00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+11:3653:x1874579665657216:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739061.098438:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739061.112221:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa600/0x7ee4453e4a1acdea lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1acdf1 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739061.112240:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa600/0x7ee4453e4a1acdea lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1acdf1 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.112755:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa600/0x7ee4453e4a1acdea lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acdf1 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.112763:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1787739061.112829:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665657344:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:2.0:1787739061.112899:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739061.113138:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665657344 00000100:00100000:1.0:1787739061.113146:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665657344 00000100:00100000:1.0:1787739061.113197:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1298 00000100:00100000:1.0:1787739061.113205:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074184c1850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+7:3653:x1874579665657344:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:1.0:1787739061.113218:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739061.113237:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1acdf1 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1acdea expref: 7 pid: 8447 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.113253:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xc1169e19 (ffffa07433235c00) 00010000:00010000:1.0:1787739061.113275:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739061.113330:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa600/0x7ee4453e4a1acdea lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xc1169e19 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acdf1 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787739061.113345:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968372 is committed 00000001:00080000:3.0:1787739061.113361:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968371 is committed 00000020:00080000:3.0:1787739061.113380:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 360 batchid updated 00000020:00080000:3.0:1787739061.113391:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07306241a80 refcount 2 committed 0 result 0 batchid 361 00000020:00080000:3.0:1787739061.113395:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2d00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07306241540 00000020:00080000:3.0:1787739061.113398:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].354 00000020:00080000:3.0:1787739061.113401:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2700 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304277e00 00000020:00080000:3.0:1787739061.113404:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].354 00000020:00080000:3.0:1787739061.113434:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 359 batchid updated 00010000:00010000:3.0:1787739061.113457:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1acd7a lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x173:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.113465:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1acd7a lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x173:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.113472:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1acd7a lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x173:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.113477:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739061.113481:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1acd7a lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x173:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.113488:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1acd7a lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x173:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.113499:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x173:0x0].0x0 (ffffa074419d5e00) 00010000:00010000:3.0:1787739061.113501:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739061.113506:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1acd7a lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x173:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.113511:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1acd7a lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x173:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.113525:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1acd81 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.113530:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1acd81 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.113536:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1acd81 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.113541:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739061.113543:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1acd81 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.113549:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1acd81 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.113556:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc1:0x0].0x0 (ffffa074419d5700) 00010000:00010000:3.0:1787739061.113558:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739061.113561:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1acd81 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.113565:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1acd81 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1787739061.113584:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 361 committed batchid 360 00000004:00080000:3.0:1787739061.113877:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa074104046c0 with version 677 00000020:00080000:3.0:1787739061.113889:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 359 cancel update log [0x200000401:0x3:0x0].352: rc = 0 00000020:00080000:3.0:1787739061.113931:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 359 cancel update log [0x240000400:0x3:0x0].352: rc = 0 00000004:00080000:3.0:1787739061.113954:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07410404000 with version 678 00000020:00080000:3.0:1787739061.113957:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 360 cancel update log [0x200000401:0x3:0x0].353: rc = 0 00000020:00080000:3.0:1787739061.113977:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 360 cancel update log [0x240000400:0x3:0x0].353: rc = 0 00000020:00080000:3.0:1787739061.113983:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 360 00010000:00010000:3.0:1787739061.114092:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ace7d lrc: 0/0,0 mode: --/PW res: [0x240000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ace76 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.114106:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc4:0x0].0x0 (ffffa07307064600) 00010000:00010000:3.0:1787739061.114110:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aceb5 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.114119:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aceb5 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.114126:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aceb5 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.114150:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739061.116676:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561cf00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665657344:0@lo:103:jbd2/dm-1-8.0 00010000:00010000:0.0:1787739061.119510:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aceb5 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.119527:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aceb5 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739061.119592:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xc4:0x0]: returning size 4096 00000004:00200000:0.0:1787739061.119598:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739061.119657:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1aceae lrc: 0/0,0 mode: --/CR res: [0x240000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f3f expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.119678:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aceb5 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f3f expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.119688:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc4:0x0].0x0 (ffffa07307064600) 00010000:00010000:0.0:1787739061.119691:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739061.119694:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1de00, rc 0) 00000100:00100000:1.0:1787739061.123382:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074184c1850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+6:3653:x1874579665657344:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 10169us (10473us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739061.123398:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1298 00000100:00100000:3.0:1787739061.123610:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ade00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+5:3653:x1874579665657216:12345-0@lo:103:ldlm_bl.0 Request processed in 25197us (25344us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.123619:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1493 00000004:00080000:3.0:1787739061.124898:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 677 rpc_version 677 00000020:00100000:3.0:1787739061.124931:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739061.124940:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0730561e1c0 x1874579665657600/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739061.124961:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561e1c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665657600:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739061.125166:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739061.125203:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404180 going to sleep for 11 seconds 00000100:00100000:2.0:1787739061.125836:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665657216:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739061.125872:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665657600 00000100:00100000:2.0:1787739061.125877:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665657600 00000100:00100000:2.0:1787739061.125955:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3085 00000100:00100000:2.0:1787739061.125964:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:7467:x1874579665657600:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739061.126308:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665657600:12345-0@lo:1000:osp_up0-1.0 Request processed in 344us (1141us total) trans 4294983382 rc 0/0 00000100:00100000:2.0:1787739061.126316:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3085 00000100:00100000:0.0:1787739061.126565:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17572:x1874579652469248:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 29282us (29576us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739061.126573:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23416 00000100:00100000:2.0:1787739061.126955:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561e1c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665657600:0@lo:1000:osp_up0-1.0 00000004:00080000:2.0:1787739061.126968:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 678 rpc_version 678 00000020:00100000:2.0:1787739061.126989:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739061.126995:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074356ac3c0 x1874579665657856/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739061.127135:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ac3c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665657856:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739061.127176:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739061.127203:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739061.127243:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665657856 00000100:00100000:3.0:1787739061.127248:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665657856 00000100:00100000:3.0:1787739061.127287:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3086 00000100:00100000:3.0:1787739061.127294:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665657856:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739061.127547:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665657856:12345-0@lo:1000:osp_up0-1.0 Request processed in 253us (369us total) trans 4294983383 rc 0/0 00000100:00100000:3.0:1787739061.127555:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3086 00000100:00100000:2.0:1787739061.127637:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ac3c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665657856:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739061.142477:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739061.142702:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652469504 00000100:00100000:2.0:1787739061.142711:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652469504 00000100:00100000:2.0:1787739061.142767:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23417 00000100:00100000:2.0:1787739061.142777:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17572:x1874579652469504:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:2.0:1787739061.142842:0:13793:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739061.142871:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ace99 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1515:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f31 expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.142887:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1515:0x0].0x0 (ffffa07307064800) 00010000:00010000:2.0:1787739061.142892:0:13793:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739061.142915:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1acebc lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.142927:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1acebc lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.142940:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1acebc lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.142950:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1acec3 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.142958:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1acec3 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.142964:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1acec3 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.143057:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1aceca lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1515:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.143066:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1aceca lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1515:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.143074:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1aceca lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1515:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.143090:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1aced1 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.143100:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1acea7 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f38 expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.143108:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1aced1 lrc: 4/0,1 mode: --/EX res: [0x200000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.143138:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1acea7 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f38 expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.143151:0:13793:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1acea7 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f38 expref: 16 pid: 6524 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739061.143165:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ade00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665657984:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739061.143248:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739061.149603:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ade00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665657984:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739061.149634:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1aced1 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739061.166343:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739061.166580:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652469632 00000100:00100000:3.0:1787739061.166589:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652469632 00010000:00010000:3.0:1787739061.166636:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1acea7 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f38 expref: 17 pid: 6524 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739061.166686:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1494 00000100:00100000:3.0:1787739061.166696:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652469632:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739061.166710:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739061.166716:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1acea7 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f38 expref: 17 pid: 6524 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.166729:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1acea7 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f38 expref: 17 pid: 6524 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.166750:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1acea7 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f38 expref: 17 pid: 6524 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.166764:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc4:0x0].0x0 (ffffa07307064900) 00010000:00010000:3.0:1787739061.166769:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1aced1 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.166779:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1aced1 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.166786:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1aced1 lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.166817:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739061.166862:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:2409:x1874579652469632:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 167us (527us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.166869:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1494 00010000:00010000:0.0:1787739061.167108:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1aced1 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.167121:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1aced1 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.167162:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1aced8 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.167170:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1aced8 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.167179:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1aced8 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739061.167198:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562cb40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665658112:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739061.167250:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739061.167279:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241180 going to sleep for 11 seconds 00000100:00100000:2.0:1787739061.167715:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665658112 00000100:00100000:2.0:1787739061.167721:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665658112 00000100:00100000:2.0:1787739061.167758:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3087 00000100:00100000:2.0:1787739061.167766:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:13793:x1874579665658112:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739061.167781:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739061.167796:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1acedf lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aced8 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.167828:0:12410:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aceb5 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f3f expref: 7 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.167837:0:12410:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1acedf lrc: 4/0,0 mode: --/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aced8 expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.167858:0:12410:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aceb5 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f3f expref: 7 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.167870:0:12410:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aceb5 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f3f expref: 7 pid: 7944 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739061.167880:0:12410:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ade00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001_UUID:12410:1874579665658240:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739061.167938:0:12410:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739061.173142:0:12410:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ade00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001_UUID:12410:1874579665658240:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739061.173182:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1acedf lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aced8 expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.173195:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc4:0x0].0x0 (ffffa07307064600) 00010000:00010000:2.0:1787739061.173197:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739061.173202:0:12410:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1acedf lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aced8 expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.173210:0:12410:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1acedf lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aced8 expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.173216:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c400, rc 0) 00000100:00100000:2.0:1787739061.173276:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:13793:x1874579665658112:12345-0@lo:101:mdt00_004.0 Request processed in 5509us (6024us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739061.173284:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3087 00000100:00100000:3.0:1787739061.173338:0:13793:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968372 gen 2 00000004:00080000:3.0:1787739061.173344:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968366 committed transno 4294968372 00000020:00080000:3.0:1787739061.173351:0:13793:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa0743c0ef540 refcount 2 committed 0 result 0 batchid 370 00000020:00080000:3.0:1787739061.173355:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2100 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa0743c0ef480 00000020:00080000:3.0:1787739061.173358:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].367 00000020:00080000:3.0:1787739061.173361:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2900 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306940400 00000020:00080000:3.0:1787739061.173364:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].368 00000100:00100000:0.0:1787739061.173383:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000004:00080000:3.0:1787739061.173395:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968368 committed transno 4294968372 00000004:00080000:3.0:1787739061.173430:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968369 committed transno 4294968372 00000004:00080000:3.0:1787739061.173440:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968370 committed transno 4294968372 00000004:00080000:3.0:1787739061.173449:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968372 committed transno 4294968372 00000100:00100000:3.0:1787739061.173462:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562cb40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665658112:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739061.173481:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1aced8 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acedf expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.173492:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1aced8 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acedf expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00080000:1.0:1787739061.173499:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 370 committed batchid 369 00000020:00080000:1.0:1787739061.173565:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 369 00000100:00100000:2.0:1787739061.173618:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652469760 00000100:00100000:2.0:1787739061.173625:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652469760 00010000:00010000:2.0:1787739061.173649:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aceb5 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f3f expref: 8 pid: 7944 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739061.173681:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1495 00000100:00100000:2.0:1787739061.173690:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562d680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2408:x1874579652469760:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739061.173704:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739061.173709:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aceb5 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f3f expref: 8 pid: 7944 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.173721:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aceb5 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f3f expref: 8 pid: 7944 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.173741:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1aceb5 lrc: 0/0,0 mode: --/PR res: [0x240000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f3f expref: 8 pid: 7944 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.173754:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc4:0x0].0x0 (ffffa07307064600) 00010000:00010000:2.0:1787739061.173759:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1acedf lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aced8 expref: 5 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.173769:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1acedf lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1aced8 expref: 5 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.173777:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1acedf lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1aced8 expref: 5 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.173803:0:6508:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1acedf lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aced8 expref: 5 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739061.173816:0:6508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ade00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001_UUID:6508:1874579665658368:0@lo:105: 00000100:00100000:2.0:1787739061.173849:0:6508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739061.173885:0:6508:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739061.173906:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665658368 00000100:00100000:3.0:1787739061.173911:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665658368 00000100:00100000:3.0:1787739061.173952:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1143 00000100:00100000:3.0:1787739061.173959:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356afc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6508:x1874579665658368:12345-0@lo:105: 00010000:00010000:3.0:1787739061.173969:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1aced8 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acedf expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.173978:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1aced8 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acedf expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.173985:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1aced8 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acedf expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.173991:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1aced8 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1acedf expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.173997:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1aced8 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1acedf expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.174004:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1aced8 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1acedf expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.174036:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609b800) 00010000:00010000:1.0:1787739061.174067:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1aced8 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acedf expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.174076:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1aced8 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acedf expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739061.174085:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356afc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6508:x1874579665658368:12345-0@lo:105: Request processed in 126us (234us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.174093:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1143 00000100:00100000:2.0:1787739061.174165:0:6508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ade00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001_UUID:6508:1874579665658368:0@lo:105: 00010000:00010000:2.0:1787739061.174179:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739061.174227:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562d680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652469760:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 538us (845us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739061.174234:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1495 00000100:00100000:1.0:1787739061.174759:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383de00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665658496:0@lo:403:mdt00_004.0 00000100:00100000:1.0:1787739061.174804:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739061.174830:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739061.174878:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665658496 00000100:00100000:2.0:1787739061.174883:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665658496 00000100:00100000:2.0:1787739061.174919:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3088 00000100:00100000:2.0:1787739061.174926:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:13793:x1874579665658496:12345-0@lo:403:mdt00_004.0 00000100:00100000:2.0:1787739061.176675:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:13793:x1874579665658496:12345-0@lo:403:mdt00_004.0 Request processed in 1741us (1862us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739061.176688:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3088 00000100:00100000:1.0:1787739061.176930:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383de00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665658496:0@lo:403:mdt00_004.0 00000040:00080000:1.0:1787739061.179321:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].370, 664 off565552 00000040:00080000:1.0:1787739061.179843:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].369, 744 off611896 00000004:00080000:1.0:1787739061.179851:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0743c0ef240 with version 704 00000004:00080000:0.0:1787739061.179944:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 704 rpc_version 704 00000020:00100000:0.0:1787739061.179977:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739061.179988:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743562ed00 x1874579665658752/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1448/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739061.180011:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562ed00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665658752:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739061.180234:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739061.181186:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739061.182211:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665658752 00000100:00100000:2.0:1787739061.182223:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665658752 00000100:00100000:2.0:1787739061.183085:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3089 00000100:00100000:2.0:1787739061.183101:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665658752:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739061.184126:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665658752:12345-0@lo:1000:osp_up1-0.0 Request processed in 1027us (3888us total) trans 4294968373 rc 0/0 00000100:00100000:2.0:1787739061.184136:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3089 00000100:00100000:0.0:1787739061.184335:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562ed00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665658752:0@lo:1000:osp_up1-0.0 00000004:00200000:1.0:1787739061.184390:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1515:0x0]: returning size 4096 00010000:00010000:1.0:1787739061.184480:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1aced1 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.184491:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1aced1 lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.184503:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1aced1 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.184509:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739061.184514:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1aced1 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.184521:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1aced1 lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.184535:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc4:0x0].0x0 (ffffa07307064900) 00010000:00010000:1.0:1787739061.184538:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739061.184543:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1aced1 lrc: 2/0,0 mode: --/EX res: [0x200000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.184549:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1aced1 lrc: 0/0,0 mode: --/EX res: [0x200000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.184565:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1aced8 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acedf expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.184572:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1aced8 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acedf expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.184578:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1aced8 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acedf expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.184932:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739061.184942:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xc4:0x0] 00010000:00010000:1.0:1787739061.184951:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1aceca lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1515:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.184957:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1aceca lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1515:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.184962:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1aceca lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1515:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.184967:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739061.184970:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1aceca lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1515:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.184975:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1aceca lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1515:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.184983:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1515:0x0].0x0 (ffffa07307064800) 00010000:00010000:1.0:1787739061.184986:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739061.184988:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1aceca lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1515:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.184994:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1aceca lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1515:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.185011:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1acebc lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.185016:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1acebc lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.185022:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1acebc lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.185036:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1acebc lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.185041:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633cc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739061.185043:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1acebc lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.185048:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1acebc lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.185056:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78800) 00010000:00010000:1.0:1787739061.185058:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739061.185060:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1acebc lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.185066:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1acebc lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739061.185083:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e7001cb8 reply state 000000004a009558 transno 4294983384 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1acec3 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.185094:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1acec3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.185101:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x8f0f038c (ffffa07305d56500) 00010000:00010000:1.0:1787739061.185103:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739061.185106:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787739061.185192:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17572:x1874579652469504:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 42415us (42721us total) trans 4294983384 rc 0/0 00000100:00100000:1.0:1787739061.185201:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23417 00010000:00010000:0.0:1787739061.185577:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739061.185651:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1aced8 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1acedf expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.185666:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739061.185742:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1aced8 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1acedf expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739061.185753:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1aced8 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1acedf expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.185786:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1aced8 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acedf expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.185793:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1787739061.185841:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665658880:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739061.185886:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739061.185955:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665658880 00000100:00100000:3.0:1787739061.185962:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665658880 00000100:00100000:3.0:1787739061.186013:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1299 00000100:00100000:3.0:1787739061.186021:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383de00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+7:3650:x1874579665658880:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739061.186043:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739061.186061:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1acedf lrc: 0/0,0 mode: --/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aced8 expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.186075:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc4:0x0].0x0 (ffffa07307064600) 00010000:00010000:3.0:1787739061.186083:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739061.186127:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1aced8 lrc: 1/0,0 mode: --/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acedf expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.186134:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1aced8 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acedf expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.186148:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739061.186150:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739061.186152:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739061.186360:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383de00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+6:3650:x1874579665658880:12345-0@lo:103:ldlm_bl.0 Request processed in 338us (472us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.186366:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1299 00000100:00100000:1.0:1787739061.193966:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665658880:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739061.217998:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739061.218107:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652469888 00000100:00100000:3.0:1787739061.218116:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652469888 00000100:00100000:3.0:1787739061.218181:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23418 00000100:00100000:3.0:1787739061.218193:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17573:x1874579652469888:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739061.218217:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739061.218240:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1acee6 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094f46 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.218325:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1aceed lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.218339:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1aceed lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.218350:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1aceed lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.218381:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1acef4 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.218389:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1acef4 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.218395:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1acef4 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.218473:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1aceed lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.218480:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1aceed lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.218485:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1aceed lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.218491:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1aceed lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.218496:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739061.218500:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1aceed lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.218507:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1aceed lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.218519:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9300) 00010000:00010000:3.0:1787739061.218521:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739061.218527:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1aceed lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.218533:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1aceed lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.218545:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1acef4 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.218550:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1acef4 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.218556:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1acef4 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.218560:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739061.218562:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1acef4 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.218568:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1acef4 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.218574:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xb75d5c1a (ffffa074039f2800) 00010000:00010000:3.0:1787739061.218577:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739061.218579:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1acef4 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.218589:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1acef4 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.218622:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1acee6 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f46 expref: 8 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.218647:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9300) 00010000:00010000:3.0:1787739061.218650:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739061.218655:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1acee6 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f46 expref: 8 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.218666:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa600, rc 0) 00000100:00100000:3.0:1787739061.218732:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17573:x1874579652469888:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 542us (746us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739061.218740:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23418 00000100:00100000:2.0:1787739061.227351:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739061.227391:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652470016 00000100:00100000:2.0:1787739061.227401:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652470016 00000100:00100000:2.0:1787739061.227472:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23419 00000100:00100000:2.0:1787739061.227483:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ada40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17573:x1874579652470016:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739061.227506:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739061.227531:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1acefb lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094f4d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.227590:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1acf02 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.227604:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1acf02 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.227616:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1acf02 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739061.227661:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:2.0:1787739061.227666:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739061.227698:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1acefb lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f4d expref: 19 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.227716:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1acf02 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f4d expref: 18 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.227724:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78100) 00010000:00010000:2.0:1787739061.227727:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739061.227730:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c600, rc 0) 00000100:00100000:2.0:1787739061.227783:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ada40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17573:x1874579652470016:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 302us (442us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739061.227790:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23419 00000100:00100000:0.0:1787739061.248835:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739061.248876:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652470272 00000100:00100000:0.0:1787739061.248885:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652470272 00000100:00100000:0.0:1787739061.248939:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23420 00000100:00100000:0.0:1787739061.248949:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17573:x1874579652470272:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:0.0:1787739061.249006:0:13793:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739061.249041:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1acc93 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ecf expref: 7 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.249063:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9300) 00010000:00010000:0.0:1787739061.249075:0:13793:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739061.249137:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf09 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.249151:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf09 lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.249160:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf09 lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.249171:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1acf10 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.249177:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1acf10 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.249183:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1acf10 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1787739061.250167:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].356, 2168 off540904 00000040:00080000:0.0:1787739061.250261:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].357, 2368 off592192 00000004:00080000:0.0:1787739061.250269:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07306241300 with version 679 00000004:00080000:0.0:1787739061.250298:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 679 rpc_version 679 00000020:00100000:0.0:1787739061.250333:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739061.250342:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743562cf00 x1874579665659136/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739061.250365:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562cf00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665659136:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739061.250464:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739061.250513:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241c00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739061.253143:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665659136 00000100:00100000:3.0:1787739061.253158:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665659136 00000100:00100000:3.0:1787739061.253214:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3090 00000100:00100000:3.0:1787739061.253224:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665659136:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739061.253914:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665659136:12345-0@lo:1000:osp_up0-1.0 Request processed in 690us (3447us total) trans 4294983385 rc 0/0 00000100:00100000:3.0:1787739061.253925:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3090 00000100:00100000:3.0:1787739061.254008:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562cf00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665659136:0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1787739061.257239:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1acf17 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x174:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.257258:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1acf17 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x174:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.257270:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1acf17 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x174:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.257289:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acf1e lrc: 3/0,1 mode: --/PW res: [0x240000400:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.257296:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acf1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.257303:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acf1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.257337:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1acf25 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.257345:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1acf25 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.257355:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1acf25 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739061.257378:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562f0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665659264:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739061.257474:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739061.257562:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665659264 00000100:00100000:3.0:1787739061.257569:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665659264 00000100:00100000:3.0:1787739061.257622:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3091 00000100:00100000:3.0:1787739061.257632:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665659264:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739061.257649:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739061.257664:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1acf2c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1acf25 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.257696:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1acf2c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acf25 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.257707:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1acf2c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acf25 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.257715:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc2:0x0].0x0 (ffffa074039f2800) 00010000:00010000:3.0:1787739061.257717:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739061.257720:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab400, rc 0) 00000100:00100000:3.0:1787739061.257773:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:13793:x1874579665659264:12345-0@lo:101:mdt00_004.0 Request processed in 143us (299us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.257779:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3091 00000100:00100000:0.0:1787739061.258081:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739061.258106:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562f0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665659264:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739061.258121:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1acf25 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acf2c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.258131:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1acf25 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acf2c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739061.258152:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e93bf4c5 reply state 00000000e5990254 transno 4294968374 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acf1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.258164:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acf1e lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.258171:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc2:0x0].0x0 (ffffa07307064b00) 00010000:00010000:0.0:1787739061.258174:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739061.258176:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739061.258182:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1acf25 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acf2c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.258189:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1acf25 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acf2c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739061.258199:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xc2:0x0] 00000004:00010000:0.0:1787739061.258205:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e93bf4c5 reply state 00000000e5990254 transno 4294968374 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1acf17 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x174:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.258214:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1acf17 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x174:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.258220:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x174:0x0].0x0 (ffffa07307064200) 00010000:00010000:0.0:1787739061.258222:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739061.258224:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787739061.258229:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x174:0x0]: returning size 4096 00010000:00010000:0.0:1787739061.258238:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf09 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.258243:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf09 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.258248:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf09 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.258254:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf09 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.258260:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739061.258263:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf09 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.258271:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf09 lrc: 5/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.258279:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064e00) 00010000:00010000:0.0:1787739061.258281:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739061.258284:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf09 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.258290:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf09 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739061.258304:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e93bf4c5 reply state 00000000e5990254 transno 4294968374 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1acf10 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.258313:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1acf10 lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.258323:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xb75d5c1a (ffffa07307064700) 00010000:00010000:0.0:1787739061.258326:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739061.258327:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 2 00000100:00100000:0.0:1787739061.258482:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17573:x1874579652470272:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 9533us (9656us total) trans 4294968374 rc 0/0 00000100:00100000:0.0:1787739061.258490:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23420 00000100:00100000:2.0:1787739061.286821:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739061.286910:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652470528 00000100:00100000:1.0:1787739061.286919:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652470528 00000100:00100000:1.0:1787739061.286974:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23421 00000100:00100000:1.0:1787739061.286985:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17574:x1874579652470528:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:1.0:1787739061.287315:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17574:x1874579652470528:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 331us (499us total) trans 0 rc -17/-17 00000100:00100000:1.0:1787739061.287323:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23421 00000100:00100000:2.0:1787739061.316500:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739061.316683:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652470656 00000100:00100000:1.0:1787739061.316693:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652470656 00000100:00100000:1.0:1787739061.316763:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23422 00000100:00100000:1.0:1787739061.316775:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356aed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17575:x1874579652470656:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739061.316799:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739061.316824:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1acf33 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094f54 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.316902:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acf3a lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.316916:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acf3a lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.316927:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acf3a lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.316937:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acf41 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.316944:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acf41 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.316951:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acf41 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.317018:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acf48 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x174:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.317026:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acf48 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x174:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.317046:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acf48 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x174:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739061.317125:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x174:0x0]: returning size 4096 00000004:00200000:1.0:1787739061.317130:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:1.0:1787739061.317150:0:6524:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acf48 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x174:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.317164:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acf3a lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.317170:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acf3a lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.317176:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acf3a lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.317181:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acf3a lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.317187:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739061.317191:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acf3a lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.317198:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acf3a lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.317209:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07433235c00) 00010000:00010000:1.0:1787739061.317211:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739061.317218:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acf3a lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.317224:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acf3a lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.317235:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acf41 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.317241:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acf41 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.317247:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acf41 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.317252:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739061.317254:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acf41 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.317259:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acf41 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.317264:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xb75d5c1a (ffffa07307064700) 00010000:00010000:1.0:1787739061.317266:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739061.317269:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acf41 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.317278:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acf41 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.317305:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1acf33 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f54 expref: 10 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.317322:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acf48 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x174:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f54 expref: 9 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.317543:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x174:0x0].0x0 (ffffa07307064200) 00010000:00010000:1.0:1787739061.317545:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787739061.317548:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1de00, rc 0) 00000100:00100000:1.0:1787739061.317610:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356aed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17575:x1874579652470656:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 837us (1119us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739061.317617:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23422 00000100:00100000:3.0:1787739061.324097:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739061.325263:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652470784 00000100:00100000:0.0:1787739061.325277:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652470784 00000100:00100000:0.0:1787739061.325342:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23423 00000100:00100000:0.0:1787739061.325353:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17575:x1874579652470784:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739061.325372:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739061.325392:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf4f lrc: 2/0,0 mode: --/CR res: [0x240000400:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094f5b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.325467:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1acf56 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.325482:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1acf56 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.325493:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1acf56 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739061.325537:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xc2:0x0]: returning size 4096 00000004:00200000:0.0:1787739061.325542:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739061.325577:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf4f lrc: 0/0,0 mode: --/CR res: [0x240000400:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f5b expref: 11 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.325598:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1acf56 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f5b expref: 10 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.325607:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc2:0x0].0x0 (ffffa07307064b00) 00010000:00010000:0.0:1787739061.325609:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739061.325613:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a400, rc 0) 00000100:00100000:0.0:1787739061.325689:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17575:x1874579652470784:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 337us (1598us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739061.325695:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23423 00000100:00100000:1.0:1787739061.331499:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739061.331578:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652470912 00000100:00100000:3.0:1787739061.331588:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652470912 00000100:00100000:3.0:1787739061.331649:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23424 00000100:00100000:3.0:1787739061.331658:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17575:x1874579652470912:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739061.331677:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739061.331696:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1acf5d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094f62 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.331766:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acf64 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.331782:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1acf2c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acf25 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.331795:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acf64 lrc: 4/1,0 mode: --/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.331936:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1acf2c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1acf25 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739061.331951:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665659392:0@lo:104: 00000100:00100000:3.0:1787739061.331996:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739061.332063:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739061.332130:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665659392 00000100:00100000:0.0:1787739061.332134:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665659392 00000100:00100000:0.0:1787739061.332172:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1144 00000100:00100000:0.0:1787739061.332180:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6523:x1874579665659392:12345-0@lo:104: 00010000:00010000:0.0:1787739061.332192:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1acf25 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1acf2c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.332247:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787739061.332253:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6523:x1874579665659392:12345-0@lo:104: Request processed in 74us (256us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739061.332259:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1144 00010000:00010000:0.0:1787739061.332282:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739061.332289:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1acf25 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1acf2c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.332296:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609aa00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739061.332302:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1acf25 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1acf2c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739061.332312:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1acf25 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1acf2c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.332336:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1acf25 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1acf2c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.332343:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739061.332361:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1acf25 lrc: 1/0,0 mode: --/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1acf2c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.332368:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1acf25 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1acf2c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.332384:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739061.332386:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739061.332388:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739061.332398:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665659520:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739061.332447:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739061.332483:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665659520 00000100:00100000:0.0:1787739061.332487:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665659520 00010000:00010000:0.0:1787739061.332503:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1acf2c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1acf25 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739061.332523:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1496 00000100:00100000:0.0:1787739061.332529:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+10:3651:x1874579665659520:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739061.332538:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1787739061.334572:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665659392:0@lo:104: 00010000:00010000:0.0:1787739061.334596:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acf64 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787739061.342456:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983385 is committed 00000001:00080000:1.0:1787739061.342484:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983384 is committed 00000020:00080000:1.0:1787739061.342492:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 370 batchid updated 00000020:00080000:1.0:1787739061.342506:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07303c6e0c0 refcount 2 committed 0 result 0 batchid 371 00000020:00080000:1.0:1787739061.342511:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743563fa00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07303c6e9c0 00000020:00080000:1.0:1787739061.342514:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].368 00000020:00080000:1.0:1787739061.342517:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743563f200 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073040e8800 00000020:00080000:1.0:1787739061.342520:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].369 00000020:00080000:1.0:1787739061.342538:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 371 committed batchid 370 00000004:00080000:1.0:1787739061.342653:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730623dc00 with version 705 00000020:00080000:1.0:1787739061.342665:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 370 cancel update log [0x240000401:0x3:0x0].367: rc = 0 00000020:00080000:1.0:1787739061.342715:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 370 cancel update log [0x200000400:0x3:0x0].368: rc = 0 00000020:00080000:1.0:1787739061.342722:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 370 00010000:00010000:1.0:1787739061.342780:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1acf2c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1acf25 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.342809:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc2:0x0].0x0 (ffffa074039f2800) 00010000:00010000:1.0:1787739061.342815:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acf64 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.342824:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acf64 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.342832:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acf64 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.342855:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739061.342931:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665659520:12345-0@lo:103:ldlm_bl.0 Request processed in 10400us (10481us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739061.342941:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1496 00010000:00010000:3.0:1787739061.345222:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ace61 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345238:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ace61 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345245:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ace61 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345251:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739061.345255:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ace61 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345263:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ace61 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345279:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x8f0f038c (ffffa07305d56500) 00010000:00010000:3.0:1787739061.345282:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739061.345286:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ace61 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345296:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ace61 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345307:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ace68 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1515:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345312:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ace68 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1515:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345318:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ace68 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1515:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345323:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ba00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739061.345325:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ace68 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1515:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345330:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ace68 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1515:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345336:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1515:0x0].0x0 (ffffa07307064800) 00010000:00010000:3.0:1787739061.345338:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739061.345341:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ace68 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1515:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345346:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ace68 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1515:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345364:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ace6f lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345369:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ace6f lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345375:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ace6f lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345380:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739061.345382:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ace6f lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345388:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ace6f lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345395:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc4:0x0].0x0 (ffffa07307064900) 00010000:00010000:3.0:1787739061.345397:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739061.345400:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ace6f lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345405:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ace6f lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345446:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1acec3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345452:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1acec3 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345457:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1acec3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345462:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739061.345465:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1acec3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345475:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1acec3 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345483:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x8f0f038c (ffffa07305d56500) 00010000:00010000:3.0:1787739061.345485:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739061.345487:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1acec3 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.345493:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1acec3 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x8f0f038c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739061.347459:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 705 rpc_version 705 00000020:00100000:2.0:1787739061.347499:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739061.347510:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074356ae580 x1874579665659776/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739061.347532:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665659776:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739061.347604:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739061.347640:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413480 going to sleep for 11 seconds 00010000:00010000:2.0:1787739061.347666:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acf64 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.347676:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acf64 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739061.347731:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xc2:0x0]: returning size 4096 00000004:00200000:2.0:1787739061.347735:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739061.347784:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1acf5d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f62 expref: 16 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.348657:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acf64 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f62 expref: 15 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.348670:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc2:0x0].0x0 (ffffa074039f2800) 00010000:00010000:2.0:1787739061.348673:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739061.348676:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab000, rc 0) 00000100:00100000:2.0:1787739061.348740:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17575:x1874579652470912:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 17082us (17249us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739061.348754:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23424 00000100:00100000:1.0:1787739061.348917:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665659520:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739061.348951:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665659776 00000100:00100000:1.0:1787739061.348957:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665659776 00000100:00100000:1.0:1787739061.349040:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3092 00000100:00100000:1.0:1787739061.349049:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665659776:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739061.349189:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665659776:12345-0@lo:1000:osp_up1-0.0 Request processed in 141us (1585us total) trans 4294968375 rc 0/0 00000100:00100000:1.0:1787739061.349196:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3092 00000100:00100000:2.0:1787739061.350710:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665659776:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739061.364361:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739061.364481:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652471168 00000100:00100000:0.0:1787739061.364491:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652471168 00000100:00100000:0.0:1787739061.364544:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23425 00000100:00100000:0.0:1787739061.364554:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ae940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17575:x1874579652471168:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:0.0:1787739061.364608:0:6525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739061.364634:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1acf48 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x174:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f54 expref: 10 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.364651:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x174:0x0].0x0 (ffffa07307064200) 00010000:00010000:0.0:1787739061.364656:0:6525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739061.364681:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acf6b lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.364693:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acf6b lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.364703:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acf6b lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.364714:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf72 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.364722:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf72 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.364729:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf72 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.364811:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1acf79 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x174:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.364820:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1acf79 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x174:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.364827:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1acf79 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x174:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.364843:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acf80 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.364853:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1acf56 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f5b expref: 9 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.364862:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acf80 lrc: 4/0,1 mode: --/EX res: [0x240000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.364893:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1acf56 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f5b expref: 9 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.364907:0:6525:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1acf56 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f5b expref: 9 pid: 13793 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739061.365050:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562f840 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665659904:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739061.365132:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241240 going to sleep for 11 seconds 00000100:00100000:1.0:1787739061.384931:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562f840 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665659904:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739061.384960:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acf80 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739061.388559:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739061.388704:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652471296 00000100:00100000:1.0:1787739061.388713:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652471296 00010000:00010000:1.0:1787739061.388750:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1acf56 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f5b expref: 10 pid: 13793 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739061.388792:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1497 00000100:00100000:1.0:1787739061.388801:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356acf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2409:x1874579652471296:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739061.388816:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739061.388823:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1acf56 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f5b expref: 10 pid: 13793 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.388837:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1acf56 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f5b expref: 10 pid: 13793 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.388859:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1acf56 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f5b expref: 10 pid: 13793 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.388872:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc2:0x0].0x0 (ffffa07307064b00) 00010000:00010000:1.0:1787739061.388876:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acf80 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.388886:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acf80 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.388894:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acf80 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.389102:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739061.389156:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356acf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:2409:x1874579652471296:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 355us (605us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739061.389164:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1497 00010000:00010000:3.0:1787739061.389315:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acf80 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.389324:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acf80 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.389370:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acf87 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.389376:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acf87 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.389384:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acf87 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739061.389402:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665660032:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1787739061.389484:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739061.389521:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404180 going to sleep for 11 seconds 00000100:00100000:3.0:1787739061.389537:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665660032 00000100:00100000:3.0:1787739061.389542:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665660032 00000100:00100000:3.0:1787739061.389631:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3093 00000100:00100000:3.0:1787739061.389639:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+5:6525:x1874579665660032:12345-0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739061.389654:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739061.389668:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1acf8e lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1acf87 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.389688:0:6528:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acf64 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f62 expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.389698:0:6528:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1acf8e lrc: 4/0,0 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acf87 expref: 6 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.389720:0:6528:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acf64 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f62 expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.389733:0:6528:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acf64 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f62 expref: 14 pid: 6523 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739061.389743:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33480 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6528:1874579665660160:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739061.389790:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739061.396481:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33480 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6528:1874579665660160:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739061.396518:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1acf8e lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acf87 expref: 6 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.396531:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc2:0x0].0x0 (ffffa074039f2800) 00010000:00010000:1.0:1787739061.396535:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739061.396538:0:6528:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1acf8e lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acf87 expref: 6 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.396548:0:6528:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1acf8e lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acf87 expref: 6 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.396556:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa000, rc 0) 00000100:00100000:1.0:1787739061.396621:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:6525:x1874579665660032:12345-0@lo:101:mdt00_002.0 Request processed in 6983us (7136us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739061.396630:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3093 00000100:00100000:3.0:1787739061.396968:0:6525:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983385 gen 2 00000004:00080000:3.0:1787739061.396975:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983380 committed transno 4294983385 00000020:00080000:3.0:1787739061.396980:0:6525:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07303c6ed80 refcount 2 committed 0 result 0 batchid 362 00000020:00080000:3.0:1787739061.396985:0:6525:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743563f300 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07303c6e000 00000020:00080000:3.0:1787739061.396988:0:6525:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].355 00000020:00080000:3.0:1787739061.396990:0:6525:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743563f100 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073040e8a00 00000020:00080000:3.0:1787739061.396993:0:6525:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].355 00000004:00080000:3.0:1787739061.397021:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983382 committed transno 4294983385 00000004:00080000:3.0:1787739061.397041:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983383 committed transno 4294983385 00000004:00080000:3.0:1787739061.397049:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983385 committed transno 4294983385 00000100:00100000:3.0:1787739061.397068:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665660032:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739061.397085:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acf87 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acf8e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.397095:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acf87 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acf8e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00080000:1.0:1787739061.397251:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 362 committed batchid 360 00080000:02000000:1.0:1787739061.397271:0:7475:0:(osd_handler.c:2218:osd_trans_start()) *** cfs_fail_loc=19a, val=0*** 00000020:00080000:1.0:1787739061.397280:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 362 committed batchid 360 00000100:00100000:1.0:1787739061.400112:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739061.400191:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652471424 00000100:00100000:3.0:1787739061.400201:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652471424 00010000:00010000:3.0:1787739061.400240:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acf64 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f62 expref: 15 pid: 6523 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739061.400288:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1498 00000100:00100000:3.0:1787739061.400299:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304003c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2408:x1874579652471424:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739061.400315:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739061.400322:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acf64 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f62 expref: 15 pid: 6523 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.400336:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acf64 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f62 expref: 15 pid: 6523 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.400359:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1acf64 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc2:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f62 expref: 15 pid: 6523 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.400379:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc2:0x0].0x0 (ffffa074039f2800) 00010000:00010000:3.0:1787739061.400386:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1acf8e lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acf87 expref: 5 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.400398:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1acf8e lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1acf87 expref: 5 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.400407:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1acf8e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1acf87 expref: 5 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.400462:0:6509:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1acf8e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acf87 expref: 5 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739061.400475:0:6509:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665660288:0@lo:105: 00000100:00100000:3.0:1787739061.400521:0:6509:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739061.400566:0:6509:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404180 going to sleep for 11 seconds 00000100:00100000:2.0:1787739061.401839:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665660288 00000100:00100000:2.0:1787739061.401849:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665660288 00000100:00100000:2.0:1787739061.402017:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1145 00000100:00100000:2.0:1787739061.402041:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665660288:12345-0@lo:105: 00010000:00010000:2.0:1787739061.402055:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acf87 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acf8e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.402067:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acf87 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acf8e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.402076:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acf87 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acf8e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.402091:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acf87 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1acf8e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.402099:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acf87 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1acf8e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.402108:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acf87 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1acf8e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.402132:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069ab600) 00010000:00010000:3.0:1787739061.402153:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acf87 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acf8e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.402165:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acf87 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acf8e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739061.402196:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665660288:12345-0@lo:105: Request processed in 170us (1674us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739061.402205:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1145 00000100:00100000:2.0:1787739061.402257:0:6509:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665660288:0@lo:105: 00010000:00010000:2.0:1787739061.402270:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739061.402312:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304003c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2408:x1874579652471424:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 2015us (2209us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739061.402320:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1498 00000100:00100000:3.0:1787739061.412567:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665660416:0@lo:403:mdt00_002.0 00000100:00100000:3.0:1787739061.412646:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739061.412766:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665660416 00000100:00100000:2.0:1787739061.412773:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665660416 00000100:00100000:2.0:1787739061.412875:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3094 00000100:00100000:2.0:1787739061.412885:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:6525:x1874579665660416:12345-0@lo:403:mdt00_002.0 00000100:00100000:3.0:1787739061.413931:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739061.419520:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:6525:x1874579665660416:12345-0@lo:403:mdt00_002.0 Request processed in 6630us (6867us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739061.419536:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3094 00000100:00100000:1.0:1787739061.420532:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665660416:0@lo:403:mdt00_002.0 00000040:00080000:1.0:1787739061.422763:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].357, 664 off541568 00000040:00080000:1.0:1787739061.423772:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].358, 744 off592936 00000004:00080000:1.0:1787739061.423780:0:6525:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730623d840 with version 680 00000004:00080000:1.0:1787739061.425517:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 680 rpc_version 680 00000020:00100000:1.0:1787739061.425569:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739061.425581:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304001680 x1874579665660672/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1448/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739061.425607:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304001680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665660672:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739061.425675:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739061.425711:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739061.425735:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665660672 00000100:00100000:1.0:1787739061.425741:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665660672 00000100:00100000:1.0:1787739061.425791:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3095 00000100:00100000:1.0:1787739061.425800:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304000b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:7467:x1874579665660672:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739061.426683:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304000b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665660672:12345-0@lo:1000:osp_up0-1.0 Request processed in 879us (1004us total) trans 4294983386 rc 0/0 00000100:00100000:1.0:1787739061.426694:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3095 00000100:00100000:1.0:1787739061.426773:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304001680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665660672:0@lo:1000:osp_up0-1.0 00000004:00200000:1.0:1787739061.427772:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x174:0x0]: returning size 4096 00010000:00010000:1.0:1787739061.427797:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acf80 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.427808:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acf80 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.427816:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acf80 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.427822:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739061.427827:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acf80 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.427836:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acf80 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.427850:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc2:0x0].0x0 (ffffa07307064b00) 00010000:00010000:1.0:1787739061.427854:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739061.427859:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acf80 lrc: 2/0,0 mode: --/EX res: [0x240000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.427865:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acf80 lrc: 0/0,0 mode: --/EX res: [0x240000400:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.427890:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acf87 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acf8e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.427898:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acf87 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acf8e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.427905:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acf87 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acf8e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.428748:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739061.428761:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xc2:0x0] 00010000:00010000:1.0:1787739061.428773:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739061.428779:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acf87 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1acf8e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.428788:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739061.428793:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acf87 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1acf8e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739061.428807:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acf87 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1acf8e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.428842:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acf87 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acf8e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.428849:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739061.428873:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acf87 lrc: 1/0,0 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acf8e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.428881:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1acf87 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acf8e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.428901:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739061.428903:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739061.428905:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739061.428987:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073040012c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665660800:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739061.429067:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739061.429127:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665660800 00000100:00100000:1.0:1787739061.429133:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665660800 00000100:00100000:1.0:1787739061.429180:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1300 00000100:00100000:1.0:1787739061.429189:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074184c2050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+7:3652:x1874579665660800:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739061.429203:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739061.429222:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1acf8e lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1acf87 expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.429236:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc2:0x0].0x0 (ffffa074039f2800) 00010000:00010000:1.0:1787739061.429246:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739061.429307:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074184c2050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+6:3652:x1874579665660800:12345-0@lo:103:ldlm_bl.0 Request processed in 118us (238us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739061.429315:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1300 00010000:00010000:1.0:1787739061.429372:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1acf79 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x174:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.429381:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1acf79 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x174:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.429388:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1acf79 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x174:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.429393:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609bc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739061.429397:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1acf79 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x174:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.429405:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1acf79 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x174:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.429434:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x174:0x0].0x0 (ffffa07307064200) 00010000:00010000:1.0:1787739061.429438:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739061.429442:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1acf79 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x174:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.429449:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1acf79 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x174:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.429469:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acf6b lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.429476:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acf6b lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.429482:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acf6b lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.429487:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acf6b lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.429493:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609aa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739061.429496:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acf6b lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.429502:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acf6b lrc: 5/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.429509:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064d00) 00010000:00010000:1.0:1787739061.429512:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739061.429514:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acf6b lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.429520:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1acf6b lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739061.429536:0:6525:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b5fb99af reply state 00000000359eb097 transno 4294968376 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf72 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.429548:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf72 lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.429555:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xb75d5c1a (ffffa07307064700) 00010000:00010000:1.0:1787739061.429558:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739061.429560:0:6525:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787739061.429618:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ae940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17575:x1874579652471168:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 65064us (65274us total) trans 4294968376 rc 0/0 00000100:00100000:1.0:1787739061.429626:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23425 00000100:00100000:3.0:1787739061.429890:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073040012c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665660800:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739061.468182:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739061.468259:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652471552 00000100:00100000:2.0:1787739061.468268:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652471552 00000100:00100000:2.0:1787739061.468330:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23426 00000100:00100000:2.0:1787739061.468341:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17578:x1874579652471552:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739061.468365:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739061.468386:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1acf95 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094f69 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.468508:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acf9c lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.468521:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acf9c lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.468532:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acf9c lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.468545:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfa3 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.468552:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfa3 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.468558:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfa3 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.468596:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acf9c lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.468603:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acf9c lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.468613:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acf9c lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.468619:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acf9c lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.468625:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633de00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739061.468629:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acf9c lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.468654:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acf9c lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.468665:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07303c78400) 00010000:00010000:2.0:1787739061.468668:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739061.468689:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acf9c lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.468696:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acf9c lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.468709:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfa3 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.468715:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfa3 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.468721:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfa3 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.468726:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739061.468730:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfa3 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.468735:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfa3 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.468742:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x62d06ae6 (ffffa07303c78500) 00010000:00010000:2.0:1787739061.468744:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739061.468747:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfa3 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.468752:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfa3 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.468782:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1acf95 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f69 expref: 11 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.468792:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07303c78400) 00010000:00010000:2.0:1787739061.468794:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739061.468798:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1acf95 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f69 expref: 11 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.468809:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c200, rc 0) 00000100:00100000:2.0:1787739061.468872:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17578:x1874579652471552:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 533us (699us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739061.468879:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23426 00000100:00100000:3.0:1787739061.479398:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739061.486362:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652471680 00000100:00100000:2.0:1787739061.486377:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652471680 00000100:00100000:2.0:1787739061.486491:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23427 00000100:00100000:2.0:1787739061.486500:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17578:x1874579652471680:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739061.486520:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739061.486540:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1acfaa lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094f70 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.486600:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfb1 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.486614:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfb1 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.486625:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfb1 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.486636:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acfb8 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.486643:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acfb8 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.486650:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acfb8 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.486685:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfb1 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.486692:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfb1 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.486698:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfb1 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.486705:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfb1 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.486712:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739061.486716:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfb1 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.486724:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfb1 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.486735:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07303c78400) 00010000:00010000:2.0:1787739061.486738:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739061.486742:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfb1 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.486748:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfb1 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.486759:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acfb8 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.486766:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acfb8 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.486772:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acfb8 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.486777:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633de00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739061.486779:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acfb8 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.486785:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acfb8 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.486795:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x62d06ae6 (ffffa07303c78500) 00010000:00010000:2.0:1787739061.486797:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739061.486800:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acfb8 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.486806:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acfb8 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.486836:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1acfaa lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f70 expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.486846:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07303c78400) 00010000:00010000:2.0:1787739061.486848:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739061.486852:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1acfaa lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f70 expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.486863:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c200, rc 0) 00000100:00100000:2.0:1787739061.486924:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17578:x1874579652471680:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 425us (7535us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739061.486932:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23427 00000100:00100000:0.0:1787739061.510479:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739061.510603:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652471808 00000100:00100000:1.0:1787739061.510612:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652471808 00000100:00100000:1.0:1787739061.510670:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23428 00000100:00100000:1.0:1787739061.510689:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17578:x1874579652471808:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739061.510710:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739061.510732:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acfbf lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094f77 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.510807:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acfc6 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.510822:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acfc6 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.510833:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acfc6 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739061.510883:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x5:0x0]: returning size 4096 00000004:00200000:1.0:1787739061.510888:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739061.510924:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1acfbf lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f77 expref: 12 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.510942:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acfc6 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f77 expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.510950:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07433235c00) 00010000:00010000:1.0:1787739061.510953:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739061.510957:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c800, rc 0) 00000100:00100000:1.0:1787739061.511022:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17578:x1874579652471808:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 343us (550us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739061.511039:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23428 00000100:00100000:3.0:1787739061.544310:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739061.544400:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652472064 00000100:00100000:0.0:1787739061.544410:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652472064 00000100:00100000:0.0:1787739061.544500:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23429 00000100:00100000:0.0:1787739061.544512:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17578:x1874579652472064:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:0.0:1787739061.544624:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562d680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665660928:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739061.544690:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739061.544737:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062413c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739061.548788:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665660928 00000100:00100000:2.0:1787739061.548802:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665660928 00000100:00100000:2.0:1787739061.548873:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3096 00000100:00100000:2.0:1787739061.548883:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665660928:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739061.549001:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665660928:12345-0@lo:1000:mdt00_001.0 Request processed in 119us (4310us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739061.549008:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3096 00000100:00100000:2.0:1787739061.549067:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562d680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665660928:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739061.549128:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ac780 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665661056:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739061.549160:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739061.549192:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413180 going to sleep for 11 seconds 00000100:00100000:0.0:1787739061.550182:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665661056 00000100:00100000:0.0:1787739061.550190:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665661056 00000100:00100000:0.0:1787739061.550221:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3097 00000100:00100000:0.0:1787739061.550228:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665661056:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739061.550314:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665661056:12345-0@lo:1000:mdt00_001.0 Request processed in 86us (1152us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739061.550321:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3097 00000100:00100000:2.0:1787739061.550461:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ac780 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665661056:0@lo:1000:mdt00_001.0 00010000:00010000:2.0:1787739061.550519:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1acfcd lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.550530:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1acfcd lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.550540:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1acfcd lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739061.550554:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ac780 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665661184:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1787739061.550584:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739061.550612:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739061.550682:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665661184 00000100:00100000:0.0:1787739061.550686:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665661184 00000100:00100000:0.0:1787739061.550717:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3098 00000100:00100000:0.0:1787739061.550723:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665661184:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739061.550732:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739061.550747:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acfd4 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1acfcd expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.550772:0:6528:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acfc6 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f77 expref: 10 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.550782:0:6528:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acfd4 lrc: 4/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acfcd expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.550806:0:6528:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acfc6 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f77 expref: 10 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.550821:0:6528:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acfc6 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f77 expref: 10 pid: 6525 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739061.550832:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562fc00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001_UUID:6528:1874579665661312:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739061.550886:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739061.557738:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562fc00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001_UUID:6528:1874579665661312:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739061.557776:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acfd4 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acfcd expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.557790:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07433235c00) 00010000:00010000:1.0:1787739061.557793:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739061.557797:0:6528:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acfd4 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acfcd expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.557808:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a600, rc 0) 00000100:00100000:1.0:1787739061.557878:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6524:x1874579665661184:12345-0@lo:101:mdt00_001.0 Request processed in 7153us (7291us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739061.557887:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3098 00000100:00100000:0.0:1787739061.558022:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ac780 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665661184:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739061.558052:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1acfcd lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acfd4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.558064:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1acfcd lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acfd4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739061.564018:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739061.564099:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652472192 00000100:00100000:2.0:1787739061.564108:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652472192 00010000:00010000:2.0:1787739061.564145:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acfc6 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f77 expref: 11 pid: 6525 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739061.564188:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1499 00000100:00100000:2.0:1787739061.564196:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f7c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2409:x1874579652472192:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739061.564210:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739061.564216:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acfc6 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f77 expref: 11 pid: 6525 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.564235:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acfc6 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f77 expref: 11 pid: 6525 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.564257:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1acfc6 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f77 expref: 11 pid: 6525 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.564272:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07433235c00) 00010000:00010000:2.0:1787739061.564279:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acfd4 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acfcd expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.564289:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acfd4 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1acfcd expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.564298:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acfd4 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1acfcd expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.564327:0:10340:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acfd4 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acfcd expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739061.564339:0:10340:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356af0c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665661440:0@lo:105: 00000100:00100000:2.0:1787739061.564381:0:10340:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739061.564522:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665661440 00000100:00100000:3.0:1787739061.564526:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665661440 00000100:00100000:2.0:1787739061.564571:0:10340:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413180 going to sleep for 11 seconds 00000100:00100000:3.0:1787739061.570186:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1146 00000100:00100000:3.0:1787739061.570209:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665661440:12345-0@lo:105: 00010000:00010000:3.0:1787739061.570228:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1acfcd lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acfd4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.570242:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1acfcd lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acfd4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.570252:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1acfcd lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acfd4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.570260:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1acfcd lrc: 5/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1acfd4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.570268:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1acfcd lrc: 6/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1acfd4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.570277:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1acfcd lrc: 6/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1acfd4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.570358:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633c200) 00000100:00100000:3.0:1787739061.570471:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665661440:12345-0@lo:105: Request processed in 274us (6088us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.570480:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1146 00000100:00100000:3.0:1787739061.570532:0:10340:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356af0c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665661440:0@lo:105: 00010000:00010000:3.0:1787739061.570550:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739061.570589:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f7c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2409:x1874579652472192:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 6392us (6579us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.570597:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1499 00010000:00010000:0.0:1787739061.570847:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739061.574447:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1acfcd lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acfd4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.574466:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1acfcd lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acfd4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739061.574492:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739061.574538:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1acfdb lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.574545:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1acfdb lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.574561:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1acfdb lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739061.574584:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304003c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665661568:0@lo:101:mdt00_001.0 00000100:00100000:1.0:1787739061.574647:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739061.574807:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665661568 00000100:00100000:2.0:1787739061.574814:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665661568 00000100:00100000:2.0:1787739061.574868:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3099 00000100:00100000:2.0:1787739061.574877:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304000780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6524:x1874579665661568:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739061.574895:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739061.574911:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acfe2 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1acfdb expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.574935:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acfe2 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acfdb expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.574948:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acfe2 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acfdb expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.574956:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x62d06ae6 (ffffa07303c78500) 00010000:00010000:2.0:1787739061.574960:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739061.574963:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633de00, rc 0) 00000100:00100000:1.0:1787739061.575104:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d900 going to sleep for 11 seconds 00000100:00100000:2.0:1787739061.577949:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304000780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665661568:12345-0@lo:101:mdt00_001.0 Request processed in 3064us (3293us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739061.577965:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3099 00000100:00100000:1.0:1787739061.580603:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304003c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665661568:0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739061.580635:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1acfdb lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acfe2 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.580649:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1acfdb lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acfe2 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739061.580682:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:1.0:1787739061.580739:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304003c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665661696:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739061.580805:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739061.580843:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d540 going to sleep for 11 seconds 00000100:00100000:0.0:1787739061.581101:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665661696 00000100:00100000:0.0:1787739061.581108:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665661696 00000100:00100000:0.0:1787739061.581162:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3100 00000100:00100000:0.0:1787739061.581172:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304000000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665661696:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739061.581257:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304000000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665661696:12345-0@lo:1000:mdt00_001.0 Request processed in 87us (452us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739061.581264:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3100 00000100:00100000:1.0:1787739061.581379:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304003c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665661696:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739061.581461:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304003c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665661824:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739061.581486:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739061.581504:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d540 going to sleep for 11 seconds 00000100:00100000:0.0:1787739061.581661:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665661824 00000100:00100000:0.0:1787739061.581665:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665661824 00000100:00100000:0.0:1787739061.581707:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3101 00000100:00100000:0.0:1787739061.581713:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073040012c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665661824:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739061.581825:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073040012c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665661824:12345-0@lo:1000:mdt00_001.0 Request processed in 111us (336us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739061.581832:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3101 00000100:00100000:2.0:1787739061.581992:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304003c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665661824:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739061.582069:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561c000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665661952:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739061.582095:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739061.582147:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739061.582180:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665661952 00000100:00100000:0.0:1787739061.582184:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665661952 00000100:00100000:0.0:1787739061.582238:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3102 00000100:00100000:0.0:1787739061.582245:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665661952:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739061.582336:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665661952:12345-0@lo:1000:mdt00_001.0 Request processed in 91us (239us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739061.582342:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3102 00000100:00100000:1.0:1787739061.582509:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561c000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665661952:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739061.582553:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304003c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665662080:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739061.582577:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739061.582628:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739061.582729:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665662080 00000100:00100000:0.0:1787739061.582734:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665662080 00000100:00100000:0.0:1787739061.582763:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3103 00000100:00100000:0.0:1787739061.582769:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665662080:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739061.582825:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665662080:12345-0@lo:1000:mdt00_001.0 Request processed in 55us (247us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739061.582831:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3103 00000100:00100000:3.0:1787739061.582916:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304003c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665662080:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739061.582955:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304003c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665662208:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739061.582979:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739061.583007:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074104046c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739061.583056:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665662208 00000100:00100000:0.0:1787739061.583060:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665662208 00000100:00100000:0.0:1787739061.583109:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3104 00000100:00100000:0.0:1787739061.583115:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665662208:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739061.583176:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665662208:12345-0@lo:1000:mdt00_001.0 Request processed in 61us (195us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739061.583182:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3104 00000100:00100000:3.0:1787739061.583236:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304003c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665662208:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739061.583269:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356af0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665662336:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739061.583454:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739061.583637:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074104046c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739061.583680:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665662336 00000100:00100000:1.0:1787739061.583685:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665662336 00000100:00100000:1.0:1787739061.583704:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3105 00000100:00100000:1.0:1787739061.583710:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304003c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665662336:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739061.583825:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304003c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665662336:12345-0@lo:1000:mdt00_001.0 Request processed in 114us (370us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739061.583831:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3105 00000100:00100000:3.0:1787739061.583868:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356af0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665662336:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739061.583899:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356af0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665662464:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739061.583919:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739061.583940:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074104046c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739061.583982:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665662464 00000100:00100000:0.0:1787739061.583986:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665662464 00000100:00100000:0.0:1787739061.584493:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3106 00000100:00100000:0.0:1787739061.584501:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665662464:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739061.584555:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665662464:12345-0@lo:1000:mdt00_001.0 Request processed in 55us (634us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739061.584561:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3106 00000100:00100000:0.0:1787739061.589216:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356af0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665662464:0@lo:1000:mdt00_001.0 00000040:00080000:0.0:1787739061.590024:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].371, 2168 off567720 00000040:00080000:0.0:1787739061.590115:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].370, 2368 off614264 00000004:00080000:0.0:1787739061.590120:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306241f00 with version 706 00000004:00080000:3.0:1787739061.590321:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 706 rpc_version 706 00000020:00100000:0.0:1787739061.590426:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739061.590436:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743562cf00 x1874579665662720/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739061.590460:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562cf00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665662720:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739061.590532:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739061.590569:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739061.590631:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665662720 00000100:00100000:2.0:1787739061.590638:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665662720 00000100:00100000:2.0:1787739061.590690:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3107 00000100:00100000:2.0:1787739061.590699:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665662720:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739061.591213:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665662720:12345-0@lo:1000:osp_up1-0.0 Request processed in 512us (678us total) trans 4294968377 rc 0/0 00000100:00100000:2.0:1787739061.591225:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3107 00000100:00100000:3.0:1787739061.591343:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562cf00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665662720:0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1787739061.591490:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfe9 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1517:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.591505:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfe9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1517:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.591516:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfe9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1517:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.591533:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1acff0 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.591540:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1acff0 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.591557:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1acff0 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.591583:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1acff7 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.591590:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1acff7 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.591599:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1acff7 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739061.591611:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561c000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665662848:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1787739061.591637:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739061.591659:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739061.591716:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665662848 00000100:00100000:0.0:1787739061.591720:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665662848 00000100:00100000:0.0:1787739061.591753:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3108 00000100:00100000:0.0:1787739061.591760:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060061c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+10:6524:x1874579665662848:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739061.591773:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739061.591786:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1acffe lrc: 2/0,0 mode: --/PW res: [0x240000401:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1acff7 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.591804:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1acffe lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1acff7 expref: 11 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.591815:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1acffe lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acff7 expref: 11 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.591829:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc5:0x0].0x0 (ffffa07307064e00) 00010000:00010000:0.0:1787739061.591833:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739061.591836:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b200, rc 0) 00000100:00100000:0.0:1787739061.591864:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060061c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+11:6524:x1874579665662848:12345-0@lo:101:mdt00_001.0 Request processed in 105us (227us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739061.591869:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3108 00000100:00100000:0.0:1787739061.591892:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561c000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665662848:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739061.591903:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1acff7 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acffe expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.591912:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1acff7 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acffe expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739061.591929:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000004589e14 reply state 00000000210df725 transno 4294983387 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1acff0 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.591940:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1acff0 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.591946:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc5:0x0].0x0 (ffffa07303c78600) 00010000:00010000:0.0:1787739061.591949:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739061.591951:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739061.591957:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1acff7 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acffe expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.591965:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1acff7 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acffe expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739061.591980:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xc5:0x0] 00000004:00010000:0.0:1787739061.591984:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000004589e14 reply state 00000000210df725 transno 4294983387 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfe9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1517:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.591992:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfe9 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1517:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.591997:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1517:0x0].0x0 (ffffa07303c78b00) 00010000:00010000:0.0:1787739061.591999:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739061.592001:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787739061.592005:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1517:0x0]: returning size 4096 00000004:00080000:0.0:1787739061.592010:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739061.592016:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1acfcd lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1acfd4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.592023:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1acfcd lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acfd4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.592040:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1acfcd lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1acfd4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.592056:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739061.592059:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739061.592063:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1acfdb lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acfe2 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.592070:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1acfdb lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1acfe2 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739061.592136:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17578:x1874579652472064:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 47625us (47833us total) trans 4294983387 rc 0/0 00000100:00100000:0.0:1787739061.592143:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23429 00010000:00010000:0.0:1787739061.592163:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739061.592169:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1acfcd lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1acfd4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.592176:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739061.592181:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1acfcd lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1acfd4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739061.592190:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1acfcd lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1acfd4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.592211:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1acfcd lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acfd4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.592217:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739061.592233:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1acfcd lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acfd4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.592240:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c200/0x7ee4453e4a1acfcd lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1acfd4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.592256:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739061.592257:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739061.592259:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739061.592268:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665662976:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739061.592297:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739061.592483:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665662976 00000100:00100000:2.0:1787739061.592487:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665662976 00000100:00100000:2.0:1787739061.592521:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1301 00000100:00100000:2.0:1787739061.592528:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562fc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+11:3653:x1874579665662976:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739061.592536:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739061.592549:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1acfd4 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1acfcd expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.592560:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07433235c00) 00010000:00010000:2.0:1787739061.592567:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739061.592590:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562fc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+10:3653:x1874579665662976:12345-0@lo:103:ldlm_bl.0 Request processed in 63us (293us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739061.592596:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1301 00000100:00100000:0.0:1787739061.592739:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665662976:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739061.611093:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739061.611180:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652472448 00000100:00100000:0.0:1787739061.611189:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652472448 00000100:00100000:0.0:1787739061.611269:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23430 00000100:00100000:0.0:1787739061.611278:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17586:x1874579652472448:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:0.0:1787739061.611628:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17586:x1874579652472448:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 350us (543us total) trans 0 rc -17/-17 00000100:00100000:0.0:1787739061.611636:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23430 00000100:00100000:0.0:1787739061.645631:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739061.660548:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652472576 00000100:00100000:3.0:1787739061.660563:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652472576 00000100:00100000:3.0:1787739061.660645:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23431 00000100:00100000:3.0:1787739061.660658:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17587:x1874579652472576:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739061.660689:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739061.660713:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad005 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094f7e expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.660784:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad00c lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.660954:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad00c lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.660967:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad00c lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.660982:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad013 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.660992:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acfe2 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acfdb expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.661001:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad013 lrc: 4/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.661045:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acfe2 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1acfdb expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739061.661060:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c3c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665663104:0@lo:104: 00000100:00100000:3.0:1787739061.661127:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739061.661190:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739061.661249:0:15907:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:2.0:1787739061.661261:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665663104 00000100:00100000:2.0:1787739061.661266:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665663104 00000100:00100000:2.0:1787739061.661302:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1147 00000100:00100000:2.0:1787739061.661309:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f911450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665663104:12345-0@lo:104: 00010000:00010000:2.0:1787739061.661322:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1acfdb lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1acfe2 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.661384:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739061.661390:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f911450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665663104:12345-0@lo:104: Request processed in 82us (263us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739061.661396:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1147 00010000:00010000:1.0:1787739061.665266:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739061.665281:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1acfdb lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1acfe2 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.665294:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1da00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739061.665301:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1acfdb lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1acfe2 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739061.665311:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1acfdb lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1acfe2 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.665356:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1acfdb lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1acfe2 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.665364:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739061.665391:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1acfdb lrc: 1/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1acfe2 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.665398:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1da00/0x7ee4453e4a1acfdb lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1acfe2 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.665432:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739061.665434:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739061.665436:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739061.665452:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665663232:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739061.665516:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739061.665571:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665663232 00000100:00100000:1.0:1787739061.665577:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665663232 00010000:00010000:1.0:1787739061.665605:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acfe2 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1acfdb expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739061.665635:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1500 00000100:00100000:1.0:1787739061.665645:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+10:3650:x1874579665663232:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739061.665658:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787739061.667118:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c3c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665663104:0@lo:104: 00010000:00010000:3.0:1787739061.667144:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad013 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787739061.669861:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968377 is committed 00000001:00080000:1.0:1787739061.669884:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968376 is committed 00000020:00080000:1.0:1787739061.669891:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 362 batchid updated 00000020:00080000:1.0:1787739061.669898:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07306241000 refcount 2 committed 0 result 0 batchid 363 00000020:00080000:1.0:1787739061.669903:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2600 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07306241300 00000020:00080000:1.0:1787739061.669906:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].357 00000020:00080000:1.0:1787739061.669909:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2a00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304277e00 00000020:00080000:1.0:1787739061.669911:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].356 00000020:00080000:1.0:1787739061.669931:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 361 batchid updated 00010000:00010000:1.0:1787739061.669994:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1acfe2 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1acfdb expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.670011:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x62d06ae6 (ffffa07303c78500) 00010000:00010000:1.0:1787739061.670017:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad013 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.670042:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad013 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.670050:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad013 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.670076:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739061.670145:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+6:3650:x1874579665663232:12345-0@lo:103:ldlm_bl.0 Request processed in 4500us (4627us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739061.670154:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1500 00010000:00010000:1.0:1787739061.670186:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad013 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.670193:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad013 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739061.670290:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f711e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665663360:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739061.670336:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739061.670358:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d780 going to sleep for 11 seconds 00000100:00100000:1.0:1787739061.670396:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665663232:0@lo:103:ldlm_bl.0 00000020:00080000:0.0:1787739061.670562:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 363 committed batchid 362 00000004:00080000:0.0:1787739061.670685:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07306241c00 with version 681 00000020:00080000:0.0:1787739061.670699:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 361 cancel update log [0x200000401:0x3:0x0].354: rc = 0 00000020:00080000:0.0:1787739061.670739:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 361 cancel update log [0x240000400:0x3:0x0].354: rc = 0 00000004:00080000:0.0:1787739061.670759:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07306241a80 with version 682 00000020:00080000:0.0:1787739061.670762:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 362 cancel update log [0x200000401:0x3:0x0].355: rc = 0 00000020:00080000:0.0:1787739061.670780:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 362 cancel update log [0x240000400:0x3:0x0].355: rc = 0 00000020:00080000:0.0:1787739061.670788:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 362 00000100:00100000:0.0:1787739061.670814:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665663360 00000100:00100000:0.0:1787739061.670819:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665663360 00000100:00100000:0.0:1787739061.670860:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3109 00000100:00100000:0.0:1787739061.670868:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:7944:x1874579665663360:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739061.670945:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:7944:x1874579665663360:12345-0@lo:1000:mdt00_003.0 Request processed in 77us (609us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739061.670955:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3109 00000100:00100000:0.0:1787739061.670986:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f711e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665663360:0@lo:1000:mdt00_003.0 00010000:00010000:0.0:1787739061.671038:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad01a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1517:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.671050:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad01a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1517:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.671059:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad01a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1517:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739061.671078:0:7944:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad01a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1517:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.671092:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad00c lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.671099:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad00c lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.671105:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad00c lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.671111:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad00c lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.671117:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739061.671121:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad00c lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.671129:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad00c lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.671140:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305d56d00) 00010000:00010000:0.0:1787739061.671143:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739061.671147:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad00c lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.671154:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad00c lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.671167:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad013 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.671174:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad013 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.671179:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad013 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.671184:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aaa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739061.671186:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad013 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.671192:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad013 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.671199:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x62d06ae6 (ffffa07303c78500) 00010000:00010000:0.0:1787739061.671201:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739061.671204:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad013 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.671209:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad013 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.671249:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad005 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f7e expref: 10 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.671264:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad01a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1517:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f7e expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.671272:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1517:0x0].0x0 (ffffa07307064c00) 00010000:00010000:0.0:1787739061.671274:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739061.671276:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609be00, rc 0) 00000100:00100000:0.0:1787739061.671321:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17587:x1874579652472576:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 10666us (25699us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739061.671328:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23431 00000004:00080000:1.0:1787739061.671886:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 681 rpc_version 681 00000020:00100000:1.0:1787739061.671912:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739061.671921:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743f710000 x1874579665663616/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739061.671940:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665663616:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739061.671981:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739061.672015:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623da80 going to sleep for 11 seconds 00010000:00010000:3.0:1787739061.672184:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1acf10 lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672193:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1acf10 lrc: 2/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672199:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1acf10 lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672205:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739061.672209:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1acf10 lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672216:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1acf10 lrc: 4/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672228:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xb75d5c1a (ffffa07307064700) 00010000:00010000:3.0:1787739061.672231:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739061.672235:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1acf10 lrc: 2/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672241:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1acf10 lrc: 0/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672253:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1acf17 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x174:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672260:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1acf17 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x174:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672270:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1acf17 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x174:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672275:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739061.672278:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1acf17 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x174:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672284:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1acf17 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x174:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672292:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x174:0x0].0x0 (ffffa07307064200) 00010000:00010000:3.0:1787739061.672294:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739061.672297:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1acf17 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x174:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672303:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1acf17 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x174:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672317:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acf1e lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672323:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acf1e lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672330:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acf1e lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672334:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739061.672337:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acf1e lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672347:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acf1e lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672356:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc2:0x0].0x0 (ffffa07307064b00) 00010000:00010000:3.0:1787739061.672358:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739061.672361:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acf1e lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672367:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1acf1e lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xc2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672381:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf72 lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672387:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf72 lrc: 2/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672393:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf72 lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672398:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739061.672400:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf72 lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672407:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf72 lrc: 4/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672452:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xb75d5c1a (ffffa07307064700) 00010000:00010000:3.0:1787739061.672455:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739061.672458:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf72 lrc: 2/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.672465:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1acf72 lrc: 0/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0xb75d5c1a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739061.673214:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665663616 00000100:00100000:0.0:1787739061.673225:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665663616 00000100:00100000:0.0:1787739061.673280:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3110 00000100:00100000:0.0:1787739061.673288:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f711e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:7467:x1874579665663616:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739061.673577:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f711e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665663616:12345-0@lo:1000:osp_up0-1.0 Request processed in 288us (1593us total) trans 4294983388 rc 0/0 00000100:00100000:0.0:1787739061.673584:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3110 00000100:00100000:3.0:1787739061.673656:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665663616:0@lo:1000:osp_up0-1.0 00000004:00080000:3.0:1787739061.673667:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 682 rpc_version 682 00000020:00100000:3.0:1787739061.673695:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739061.673701:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743c41c3c0 x1874579665663872/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739061.673720:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c3c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665663872:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739061.673754:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739061.673790:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739061.674167:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665663872 00000100:00100000:0.0:1787739061.674172:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665663872 00000100:00100000:0.0:1787739061.674209:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3111 00000100:00100000:0.0:1787739061.674216:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665663872:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739061.674313:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665663872:12345-0@lo:1000:osp_up0-1.0 Request processed in 97us (557us total) trans 4294983389 rc 0/0 00000100:00100000:0.0:1787739061.674319:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3111 00000100:00100000:1.0:1787739061.674597:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c3c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665663872:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739061.681247:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739061.681324:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652472704 00000100:00100000:2.0:1787739061.681334:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652472704 00000100:00100000:2.0:1787739061.681401:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23432 00000100:00100000:2.0:1787739061.681531:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17587:x1874579652472704:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739061.681560:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739061.681582:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad021 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1517:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094f85 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.681642:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad028 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1517:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.681656:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad028 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1517:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.681667:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad028 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1517:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739061.681764:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1517:0x0]: returning size 4096 00000004:00200000:2.0:1787739061.681769:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739061.683219:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad021 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1517:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f85 expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.683245:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad028 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1517:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f85 expref: 17 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.683255:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1517:0x0].0x0 (ffffa07303c78b00) 00010000:00010000:2.0:1787739061.683258:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739061.683261:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d600, rc 0) 00000100:00100000:2.0:1787739061.683325:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17587:x1874579652472704:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 1914us (2087us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739061.683333:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23432 00000100:00100000:3.0:1787739061.693549:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739061.693587:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652472832 00000100:00100000:3.0:1787739061.693596:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652472832 00000100:00100000:3.0:1787739061.693646:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23433 00000100:00100000:3.0:1787739061.693658:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17587:x1874579652472832:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739061.693679:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739061.693699:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad02f lrc: 2/0,0 mode: --/CR res: [0x200000400:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094f8c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.693767:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad036 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.693781:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad036 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.693798:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad036 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739061.693846:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xc5:0x0]: returning size 4096 00000004:00200000:3.0:1787739061.693852:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739061.693889:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad02f lrc: 0/0,0 mode: --/CR res: [0x200000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f8c expref: 19 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.693906:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad036 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f8c expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.693915:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc5:0x0].0x0 (ffffa07303c78600) 00010000:00010000:3.0:1787739061.693919:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739061.693922:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa800, rc 0) 00000100:00100000:3.0:1787739061.693976:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17587:x1874579652472832:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 321us (436us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.693983:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23433 00000100:00100000:0.0:1787739061.705240:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739061.705363:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652472960 00000100:00100000:3.0:1787739061.705372:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652472960 00000100:00100000:3.0:1787739061.705446:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23434 00000100:00100000:3.0:1787739061.705456:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17587:x1874579652472960:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739061.705476:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739061.705495:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad03d lrc: 2/0,0 mode: --/CR res: [0x240000401:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094f93 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.705551:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad044 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.705572:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1acffe lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1acff7 expref: 5 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.705582:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad044 lrc: 4/1,0 mode: --/PR res: [0x240000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.705613:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1acffe lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1acff7 expref: 5 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739061.705629:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665664000:0@lo:104: 00000100:00100000:0.0:1787739061.705672:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787739061.705685:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665664128:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1787739061.705694:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739061.705746:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739061.705764:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665664000 00000100:00100000:3.0:1787739061.705767:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665664000 00000100:00100000:3.0:1787739061.705802:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1148 00000100:00100000:3.0:1787739061.705809:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415700b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665664000:12345-0@lo:104: 00010000:00010000:3.0:1787739061.705820:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1acff7 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1acffe expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739061.706055:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739061.706564:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787739061.706575:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356afc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665664256:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787739061.706609:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787739061.711278:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739061.711296:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415700b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665664000:12345-0@lo:104: Request processed in 5482us (5597us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.711310:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1148 00010000:00010000:3.0:1787739061.711355:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739061.711364:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1acff7 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1acffe expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.711375:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739061.711383:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1acff7 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xc5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1acffe expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739061.711393:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1acff7 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xc5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1acffe expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.711456:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1acff7 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xc5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1acffe expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.711463:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739061.711485:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1acff7 lrc: 1/0,0 mode: --/PW res: [0x240000401:0xc5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1acffe expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.711493:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1acff7 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xc5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1acffe expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.711513:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739061.711516:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739061.711518:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739061.711553:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665664000:0@lo:104: 00000100:00100000:1.0:1787739061.711557:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787739061.711575:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665664384:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739061.711576:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad044 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739061.711624:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739061.711798:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665664384 00000100:00100000:3.0:1787739061.711805:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665664384 00010000:00010000:3.0:1787739061.711828:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1acffe lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1acff7 expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739061.711836:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304261a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3653:1874579665664896:0@lo:400:kworker.0 00000100:00100000:3.0:1787739061.711860:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1501 00000100:00100000:3.0:1787739061.711872:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+6:3652:x1874579665664384:12345-0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739061.711880:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787739061.711892:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1787739061.711903:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304262940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3653:1874579665665408:0@lo:400:kworker.0 00000100:00100000:0.0:1787739061.711933:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739061.711956:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1787739061.711964:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665664896 00000100:00100000:0.0:1787739061.711964:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3653:1874579665664768:0@lo:400:kworker.0 00000100:00100000:3.0:1787739061.711970:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665664896 00000100:00100000:0.0:1787739061.711991:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739061.712008:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042612c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665665280:0@lo:400:kworker.0 00000100:00100000:0.0:1787739061.712042:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739061.712064:0:14659:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665665280 00000100:00100000:0.0:1787739061.712067:0:14659:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665665280 00000100:00100000:0.0:1787739061.712099:0:14659:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 348 00000100:00100000:0.0:1787739061.712106:0:14659:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3653:x1874579665665280:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787739061.712147:0:14659:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3653:x1874579665665280:12345-0@lo:400:kworker.0 Request processed in 42us (104us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739061.712154:0:14659:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 348 00000100:00100000:0.0:1787739061.712183:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042612c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665665280:0@lo:400:kworker.0 00000100:00100000:2.0:1787739061.712264:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665664128 00000100:00100000:2.0:1787739061.712269:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665664128 00000100:00100000:2.0:1787739061.712309:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665665408 00000100:00100000:2.0:1787739061.712312:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665665408 00000100:00100000:3.0:1787739061.712796:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665664768 00000100:00100000:3.0:1787739061.712801:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665664768 00000100:00100000:0.0:1787739061.712874:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665664256 00000100:00100000:0.0:1787739061.712879:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665664256 00000100:00100000:0.0:1787739061.712901:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 356 00000100:00100000:0.0:1787739061.712908:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1874579665664128:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1787739061.712931:0:8441:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 25431488 pending 0 free 3923496960 avail 3691974656 00000100:00100000:0.0:1787739061.712976:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3650:x1874579665664128:12345-0@lo:13:osp-pre-1-1.0 Request processed in 69us (6921us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739061.712983:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 356 00000100:00100000:0.0:1787739061.712996:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 357 00000100:00100000:0.0:1787739061.713002:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356af0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874579665664256:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787739061.713012:0:8441:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 25431488 pending 0 free 3923496960 avail 3691974656 00000100:00100000:0.0:1787739061.713052:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356af0c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874579665664256:12345-0@lo:13:osp-pre-1-0.0 Request processed in 50us (6442us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739061.713058:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 357 00000100:00100000:0.0:1787739061.713104:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356afc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665664256:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787739061.719559:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 654 00000100:00100000:3.0:1787739061.719569:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 655 00000100:00100000:2.0:1787739061.719575:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874579665664896:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787739061.719583:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3653:x1874579665665408:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739061.720004:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874579665664896:12345-0@lo:400:kworker.0 Request processed in 435us (8127us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739061.720011:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 654 00000100:00100000:2.0:1787739061.720039:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 656 00000100:00100000:2.0:1787739061.720045:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3653:x1874579665664768:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787739061.721258:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3653:x1874579665665408:12345-0@lo:400:kworker.0 Request processed in 1672us (9317us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.721269:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 655 00000100:00100000:3.0:1787739061.721319:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3651:1874579665664640:0@lo:400:kworker.0 00000100:00100000:3.0:1787739061.721368:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739061.721399:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042621c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665665152:0@lo:400:kworker.0 00000100:00100000:3.0:1787739061.721440:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739061.721469:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:3.0:1787739061.721476:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2:3651:1874579665664512:0@lo:400:kworker.0 00000100:00100000:3.0:1787739061.721503:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739061.721532:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304263840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3651:1874579665665024:0@lo:400:kworker.0 00000100:00100000:3.0:1787739061.721567:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739061.721586:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3651:1874579665665536:0@lo:400:kworker.0 00000100:00100000:3.0:1787739061.721609:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739061.721638:0:8439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665665152 00000100:00100000:3.0:1787739061.721646:0:8439:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665665152 00000100:00100000:3.0:1787739061.721685:0:8439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 349 00000100:00100000:3.0:1787739061.721693:0:8439:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f77c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665665152:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787739061.721734:0:8439:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f77c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665665152:12345-0@lo:400:kworker.0 Request processed in 42us (293us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.721741:0:8439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 349 00000100:00100000:3.0:1787739061.721760:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665664640 00000100:00100000:3.0:1787739061.721763:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665664640 00000100:00100000:3.0:1787739061.721780:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665665024 00000100:00100000:3.0:1787739061.721782:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665665024 00000100:00100000:3.0:1787739061.721795:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665665536 00000100:00100000:3.0:1787739061.721798:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665665536 00000100:00100000:3.0:1787739061.721809:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 657 00000100:00100000:3.0:1787739061.721815:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f752c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1874579665664640:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787739061.721847:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f752c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1874579665664640:12345-0@lo:400:kworker.0 Request processed in 32us (478us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.721853:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 657 00000100:00100000:3.0:1787739061.721864:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 658 00000100:00100000:3.0:1787739061.721869:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f76d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1874579665665024:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787739061.721895:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f76d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3651:x1874579665665024:12345-0@lo:400:kworker.0 Request processed in 26us (328us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.721901:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 658 00000100:00100000:3.0:1787739061.721911:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 659 00000100:00100000:3.0:1787739061.721917:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f743c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3651:x1874579665665536:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787739061.721941:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f743c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3651:x1874579665665536:12345-0@lo:400:kworker.0 Request processed in 24us (330us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.721946:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 659 00000100:00100000:3.0:1787739061.722472:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561e1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665664128:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1787739061.722507:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304263840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3651:1874579665665024:0@lo:400:kworker.0 00000100:00100000:3.0:1787739061.722519:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3651:1874579665665536:0@lo:400:kworker.0 00000100:00100000:3.0:1787739061.722529:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7a940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3651:1874579665664640:0@lo:400:kworker.0 00000100:00100000:3.0:1787739061.722538:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042621c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665665152:0@lo:400:kworker.0 00000020:00080000:3.0:1787739061.729884:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 363 batchid updated 00000020:00080000:3.0:1787739061.729916:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 363 00000004:00080000:3.0:1787739061.729971:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07410404240 with version 683 00000020:00080000:3.0:1787739061.729989:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 363 cancel update log [0x200000401:0x3:0x0].357: rc = 0 00000020:00080000:3.0:1787739061.730073:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 363 cancel update log [0x240000400:0x3:0x0].356: rc = 0 00000020:00080000:3.0:1787739061.730082:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 363 00010000:00010000:3.0:1787739061.730140:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1acffe lrc: 0/0,0 mode: --/PW res: [0x240000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1acff7 expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.730158:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc5:0x0].0x0 (ffffa07307064e00) 00010000:00010000:3.0:1787739061.730163:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad044 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.730172:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad044 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.730181:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad044 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.730204:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739061.730835:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+5:3652:x1874579665664384:12345-0@lo:103:ldlm_bl.0 Request processed in 18961us (19208us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.730846:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1501 00010000:00010000:3.0:1787739061.730881:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad044 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.730893:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad044 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739061.730943:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xc5:0x0]: returning size 4096 00000004:00200000:3.0:1787739061.730948:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739061.730986:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad03d lrc: 0/0,0 mode: --/CR res: [0x240000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f93 expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.731004:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad044 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f93 expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.731012:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc5:0x0].0x0 (ffffa07307064e00) 00010000:00010000:3.0:1787739061.731015:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739061.731018:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab000, rc 0) 00000100:00100000:3.0:1787739061.731072:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17587:x1874579652472960:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 25617us (25838us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.731080:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23434 00000100:00100000:2.0:1787739061.733182:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3653:x1874579665664768:12345-0@lo:400:kworker.0 Request processed in 13128us (21182us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739061.733195:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 656 00000100:00080000:2.0:1787739061.733238:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1787739061.733244:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787739061.733246:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1787739061.733250:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787739061.733264:0:6518:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665664512 00000100:00100000:2.0:1787739061.733273:0:6518:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665664512 00000100:00100000:2.0:1787739061.733332:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 448 00000100:00100000:2.0:1787739061.733341:0:6518:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0740ea11050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2+24:3651:x1874579665664512:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739061.733440:0:6518:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0740ea11050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2+24:3651:x1874579665664512:12345-0@lo:400:kworker.0 Request processed in 100us (11935us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739061.733448:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 448 00000100:00100000:2.0:1787739061.733512:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2:3651:1874579665664512:0@lo:400:kworker.0 00000100:00100000:0.0:1787739061.734670:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304261a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3653:1874579665664896:0@lo:400:kworker.0 00000004:00080000:1.0:1787739061.734884:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 683 rpc_version 683 00000020:00100000:1.0:1787739061.734919:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739061.734927:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304d84b40 x1874579665665792/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739061.734950:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d84b40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665665792:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739061.735007:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739061.735043:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739061.735067:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665665792 00000100:00100000:3.0:1787739061.735074:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665665792 00000100:00100000:3.0:1787739061.735118:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3112 00000100:00100000:3.0:1787739061.735126:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d87480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665665792:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739061.735292:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d87480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665665792:12345-0@lo:1000:osp_up0-1.0 Request processed in 166us (283us total) trans 4294983390 rc 0/0 00000100:00100000:3.0:1787739061.735299:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3112 00000100:00100000:3.0:1787739061.735343:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d84b40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665665792:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739061.737675:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304262940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3653:1874579665665408:0@lo:400:kworker.0 00000100:00100000:1.0:1787739061.737706:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7b0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3653:1874579665664768:0@lo:400:kworker.0 00000100:00100000:0.0:1787739061.738766:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665664384:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739061.743820:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739061.743873:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739061.744020:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652473344 00000100:00100000:1.0:1787739061.744039:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652473344 00000100:00100000:1.0:1787739061.744101:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23435 00000100:00100000:1.0:1787739061.744113:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f76940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17587:x1874579652473344:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:1.0:1787739061.744170:0:6525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739061.744196:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad028 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1517:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f85 expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.744215:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1517:0x0].0x0 (ffffa07303c78b00) 00010000:00010000:1.0:1787739061.744220:0:6525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739061.744268:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad04b lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.744281:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad04b lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.744291:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad04b lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739061.744314:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d87840 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665665920:0@lo:101:mdt00_002.0 00000100:00100000:1.0:1787739061.744363:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739061.744401:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623da80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739061.744492:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665665920 00000100:00100000:2.0:1787739061.744497:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665665920 00000100:00100000:2.0:1787739061.744531:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3113 00000100:00100000:2.0:1787739061.744538:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6525:x1874579665665920:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787739061.744547:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739061.744563:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad052 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad04b expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.744585:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad052 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad04b expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.744598:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad052 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad04b expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.744606:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07303c78700) 00010000:00010000:2.0:1787739061.744609:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739061.744611:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d400, rc 0) 00000100:00100000:2.0:1787739061.744655:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6525:x1874579665665920:12345-0@lo:101:mdt00_002.0 Request processed in 118us (292us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739061.744662:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3113 00000100:00100000:0.0:1787739061.744731:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652473088 00000100:00100000:0.0:1787739061.744735:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652473088 00000100:00100000:0.0:1787739061.744768:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1302 00000100:00100000:0.0:1787739061.744775:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2408:x1874579652473088:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:0.0:1787739061.744785:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739061.744796:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad01a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1517:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f7e expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.744807:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1517:0x0].0x0 (ffffa07307064c00) 00010000:00010000:0.0:1787739061.744815:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739061.744854:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652473088:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 80us (1044us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739061.744861:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1302 00000100:00100000:1.0:1787739061.744920:0:6525:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968377 gen 2 00000004:00080000:1.0:1787739061.744926:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968373 committed transno 4294968377 00000020:00080000:1.0:1787739061.744932:0:6525:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa0743c0ef480 refcount 2 committed 0 result 0 batchid 372 00000020:00080000:1.0:1787739061.744936:0:6525:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2a00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa0743c0ef240 00000020:00080000:1.0:1787739061.744939:0:6525:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].369 00000020:00080000:1.0:1787739061.744941:0:6525:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2d00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306941e00 00000020:00080000:1.0:1787739061.744943:0:6525:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].370 00000004:00080000:1.0:1787739061.744975:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968375 committed transno 4294968377 00000004:00080000:1.0:1787739061.744990:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968377 committed transno 4294968377 00000100:00100000:1.0:1787739061.745010:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d87840 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665665920:0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787739061.745033:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad04b lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad052 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.745044:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad04b lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad052 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739061.745057:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739061.745078:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1ad059 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.745086:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1ad059 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.745093:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1ad059 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739061.745105:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d87840 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665666048:0@lo:101:mdt00_002.0 00000100:00100000:1.0:1787739061.745129:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739061.745147:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623da80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739061.745177:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665666048 00000100:00100000:3.0:1787739061.745183:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665666048 00000020:00080000:0.0:1787739061.745221:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 372 committed batchid 370 00000100:00100000:3.0:1787739061.745231:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3114 00000100:00100000:3.0:1787739061.745241:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d852c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6525:x1874579665666048:12345-0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739061.745250:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739061.745263:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad060 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad059 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00080000:0.0:1787739061.745276:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 370 00010000:00010000:3.0:1787739061.745282:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad060 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad059 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.745294:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad060 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad059 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.745301:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x62d06ae6 (ffffa07305d56e00) 00010000:00010000:3.0:1787739061.745304:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739061.745306:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abe00, rc 0) 00000100:00100000:3.0:1787739061.745354:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d852c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6525:x1874579665666048:12345-0@lo:101:mdt00_002.0 Request processed in 115us (224us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.745363:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3114 00000100:00100000:1.0:1787739061.745379:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d87840 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665666048:0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787739061.745391:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1ad059 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad060 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.745400:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1ad059 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad060 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739061.745436:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:1.0:1787739061.745464:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d87840 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665666176:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739061.745492:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739061.745513:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dc00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739061.745648:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665666176 00000100:00100000:3.0:1787739061.745652:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665666176 00000100:00100000:3.0:1787739061.745692:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3115 00000100:00100000:3.0:1787739061.745700:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6525:x1874579665666176:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739061.745769:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6525:x1874579665666176:12345-0@lo:1000:mdt00_002.0 Request processed in 70us (276us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.745776:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3115 00000100:00100000:1.0:1787739061.747080:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d87840 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665666176:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739061.747146:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562ed00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665666304:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739061.747190:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739061.747220:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dc00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739061.747251:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665666304 00000100:00100000:3.0:1787739061.747257:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665666304 00000100:00100000:3.0:1787739061.747302:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3116 00000100:00100000:3.0:1787739061.747310:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d87840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6525:x1874579665666304:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739061.747410:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d87840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6525:x1874579665666304:12345-0@lo:1000:mdt00_002.0 Request processed in 101us (219us total) trans 0 rc 1/1 00000100:00100000:3.0:1787739061.747438:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3116 00000100:00100000:1.0:1787739061.747543:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562ed00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665666304:0@lo:1000:mdt00_002.0 00010000:00010000:1.0:1787739061.747597:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ad067 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.747608:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ad067 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.747618:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ad067 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739061.747629:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562ed00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665666432:0@lo:101:mdt00_002.0 00000100:00100000:1.0:1787739061.747653:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739061.747676:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623da80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739061.748492:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665666432 00000100:00100000:3.0:1787739061.748501:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665666432 00000100:00100000:3.0:1787739061.748557:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3117 00000100:00100000:3.0:1787739061.748565:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6525:x1874579665666432:12345-0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739061.748579:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739061.748595:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad06e lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad067 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.748622:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad06e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad067 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.748637:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad06e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad067 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.748645:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1517:0x0].0x0 (ffffa07305d56100) 00010000:00010000:3.0:1787739061.748648:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739061.748652:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abc00, rc 0) 00000100:00100000:3.0:1787739061.748924:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6525:x1874579665666432:12345-0@lo:101:mdt00_002.0 Request processed in 358us (1267us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.748931:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3117 00000100:00100000:1.0:1787739061.748971:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562ed00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665666432:0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787739061.748990:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ad067 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad06e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.749001:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ad067 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad06e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.749039:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad075 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.749050:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad075 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.749058:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad075 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.749075:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad07c lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.749087:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad036 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f8c expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.749095:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad07c lrc: 4/0,1 mode: --/EX res: [0x200000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.749123:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad036 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f8c expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.749138:0:6525:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad036 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f8c expref: 17 pid: 13793 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739061.749150:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f0f00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6525:1874579665666560:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739061.749222:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623da80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739061.761499:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f0f00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6525:1874579665666560:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739061.761536:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad07c lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739061.765268:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739061.767658:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652473472 00000100:00100000:1.0:1787739061.768463:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652473472 00010000:00010000:1.0:1787739061.768519:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad036 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f8c expref: 18 pid: 13793 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739061.768566:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1502 00000100:00100000:1.0:1787739061.768575:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f75e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2408:x1874579652473472:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739061.768589:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739061.768596:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad036 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f8c expref: 18 pid: 13793 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.768614:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad036 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f8c expref: 18 pid: 13793 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.768637:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad036 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f8c expref: 18 pid: 13793 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.768654:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc5:0x0].0x0 (ffffa07303c78600) 00010000:00010000:1.0:1787739061.768660:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad07c lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.768669:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad07c lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.768688:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad07c lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.768722:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739061.768774:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad07c lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.768785:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad07c lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739061.768791:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f75e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2408:x1874579652473472:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 215us (3529us total) trans 0 rc 0/0 00010000:00010000:2.0:1787739061.768844:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ad083 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.768853:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ad083 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.768863:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ad083 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739061.768884:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665666688:0@lo:101:mdt00_002.0 00000100:00100000:1.0:1787739061.768913:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1502 00000100:00100000:2.0:1787739061.768942:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739061.768975:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c4139c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739061.769007:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665666688 00000100:00100000:3.0:1787739061.769013:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665666688 00000100:00100000:3.0:1787739061.769060:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3118 00000100:00100000:3.0:1787739061.769067:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6525:x1874579665666688:12345-0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739061.769081:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739061.769095:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad08a lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad083 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.769116:0:6529:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad044 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f93 expref: 6 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.769125:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad08a lrc: 4/0,0 mode: --/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad083 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.769145:0:6529:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad044 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f93 expref: 6 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.769157:0:6529:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad044 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f93 expref: 6 pid: 7944 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739061.769167:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f77840 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665666816:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739061.769216:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074104049c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739061.781312:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f77840 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665666816:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739061.781350:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad08a lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad083 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.781363:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc5:0x0].0x0 (ffffa07307064e00) 00010000:00010000:3.0:1787739061.781367:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739061.781371:0:6529:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad08a lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad083 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.781381:0:6529:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad08a lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad083 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.781388:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab200, rc 0) 00000100:00100000:3.0:1787739061.781481:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:6525:x1874579665666688:12345-0@lo:101:mdt00_002.0 Request processed in 12414us (12540us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.781491:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3118 00000100:00100000:0.0:1787739061.781529:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739061.782179:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652473600 00000100:00100000:3.0:1787739061.782187:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652473600 00010000:00010000:3.0:1787739061.782220:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad044 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f93 expref: 7 pid: 7944 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739061.782256:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1503 00000100:00100000:3.0:1787739061.782266:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074184c3050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652473600:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739061.782281:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739061.782288:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad044 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f93 expref: 7 pid: 7944 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.782300:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad044 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f93 expref: 7 pid: 7944 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.782323:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad044 lrc: 0/0,0 mode: --/PR res: [0x240000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f93 expref: 7 pid: 7944 timeout: 748 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.782342:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc5:0x0].0x0 (ffffa07307064e00) 00010000:00010000:3.0:1787739061.782348:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad08a lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad083 expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.782356:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad08a lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad083 expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.782364:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad08a lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad083 expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.782394:0:10340:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad08a lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad083 expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739061.782407:0:10340:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f77840 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665666944:0@lo:105: 00000100:00100000:3.0:1787739061.782492:0:10340:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739061.782587:0:10340:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074104049c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739061.782615:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004780 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665666688:0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787739061.782634:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ad083 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad08a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739061.782645:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ad083 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad08a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739061.787568:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665666944 00000100:00100000:1.0:1787739061.787583:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665666944 00000100:00100000:1.0:1787739061.787786:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1149 00000100:00100000:1.0:1787739061.788998:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f75a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665666944:12345-0@lo:105: 00010000:00010000:1.0:1787739061.789019:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ad083 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad08a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.789043:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ad083 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad08a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.789055:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ad083 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad08a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.789065:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ad083 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad08a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.789074:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ad083 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad08a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.789085:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ad083 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ad08a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.789119:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633c000) 00000100:00100000:1.0:1787739061.789213:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f75a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665666944:12345-0@lo:105: Request processed in 1418us (6722us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739061.789222:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1149 00010000:00010000:3.0:1787739061.789309:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ad083 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad08a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.789320:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ad083 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad08a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739061.789933:0:10340:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f77840 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665666944:0@lo:105: 00010000:00010000:0.0:1787739061.789949:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739061.790001:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074184c3050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2408:x1874579652473600:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 7734us (8473us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739061.790011:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1503 00000100:00100000:3.0:1787739061.791038:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74b40 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665667072:0@lo:403:mdt00_002.0 00000100:00100000:3.0:1787739061.791103:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739061.791140:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404c00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739061.791241:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665667072 00000100:00100000:0.0:1787739061.791248:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665667072 00000100:00100000:0.0:1787739061.791295:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3119 00000100:00100000:0.0:1787739061.791304:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:6525:x1874579665667072:12345-0@lo:403:mdt00_002.0 00000100:00100000:0.0:1787739061.794167:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:6525:x1874579665667072:12345-0@lo:403:mdt00_002.0 Request processed in 2852us (3053us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739061.794182:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3119 00000100:00100000:0.0:1787739061.794265:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74b40 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665667072:0@lo:403:mdt00_002.0 00000040:00080000:0.0:1787739061.795787:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].372, 664 off568384 00000040:00080000:0.0:1787739061.796131:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].371, 744 off615008 00000004:00080000:0.0:1787739061.796137:0:6525:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306241480 with version 707 00000004:00080000:3.0:1787739061.796191:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 707 rpc_version 707 00000020:00100000:3.0:1787739061.796221:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739061.796230:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303f76580 x1874579665667328/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1808/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739061.796255:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f76580 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665667328:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739061.796311:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739061.796350:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404000 going to sleep for 11 seconds 00000100:00100000:0.0:1787739061.796450:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665667328 00000100:00100000:0.0:1787739061.796457:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665667328 00000100:00100000:0.0:1787739061.796496:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3120 00000100:00100000:0.0:1787739061.796503:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665667328:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739061.797595:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665667328:12345-0@lo:1000:osp_up1-0.0 Request processed in 1088us (1278us total) trans 4294968378 rc 0/0 00000100:00100000:0.0:1787739061.797605:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3120 00000100:00100000:3.0:1787739061.800793:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f76580 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665667328:0@lo:1000:osp_up1-0.0 00000004:00200000:0.0:1787739061.811822:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1517:0x0]: returning size 4096 00010000:00010000:0.0:1787739061.811857:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad07c lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.811868:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad07c lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.811877:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad07c lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.811885:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739061.811891:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad07c lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.811900:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad07c lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.811917:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc5:0x0].0x0 (ffffa07303c78600) 00010000:00010000:0.0:1787739061.811921:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739061.811926:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad07c lrc: 2/0,0 mode: --/EX res: [0x200000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.811933:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad07c lrc: 0/0,0 mode: --/EX res: [0x200000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.811947:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ad083 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad08a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.811963:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ad083 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad08a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.811970:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ad083 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad08a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.812001:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739061.812011:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xc5:0x0] 00010000:00010000:0.0:1787739061.812019:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad075 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.812036:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad075 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.812042:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad075 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.812048:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ca00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739061.812051:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad075 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.812057:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad075 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.812066:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1517:0x0].0x0 (ffffa07303c78b00) 00010000:00010000:0.0:1787739061.812069:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739061.812072:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad075 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.812077:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad075 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.812087:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ad067 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad06e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.812095:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ad067 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad06e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.812102:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ad067 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad06e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.812118:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:0.0:1787739061.812131:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739061.812135:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad04b lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad052 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.812143:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad04b lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad052 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.812151:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad04b lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad052 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.812164:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:0.0:1787739061.812168:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739061.812172:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1ad059 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad060 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.812180:0:6525:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1ad059 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad060 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.812442:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1787739061.812449:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ad083 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad08a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.812458:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739061.812463:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ad083 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad08a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739061.812472:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ad083 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad08a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.812506:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ad083 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad08a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.812513:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739061.812546:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ad083 lrc: 1/0,0 mode: --/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad08a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.812554:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633c000/0x7ee4453e4a1ad083 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad08a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.812579:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787739061.812583:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ad067 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad06e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.812590:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739061.812594:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ad067 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad06e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739061.812601:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ad067 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad06e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.812618:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ad067 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad06e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.812625:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739061.812645:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ad067 lrc: 1/0,0 mode: --/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad06e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.812653:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c600/0x7ee4453e4a1ad067 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad06e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.812666:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739061.812669:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad04b lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad052 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.812687:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739061.812691:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad04b lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad052 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739061.812700:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad04b lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad052 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.812716:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad04b lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad052 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.812721:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739061.812727:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad04b lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad052 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.812735:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad04b lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad052 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.812747:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739061.812749:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739061.812751:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739061.812762:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787739061.812782:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665667456:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739061.812853:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739061.812914:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665667456 00000100:00100000:3.0:1787739061.812920:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665667456 00000100:00100000:3.0:1787739061.812963:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1303 00000100:00100000:3.0:1787739061.812972:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+10:3650:x1874579665667456:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739061.812985:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739061.813003:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad08a lrc: 0/0,0 mode: --/EX res: [0x240000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad083 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.813022:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc5:0x0].0x0 (ffffa07307064e00) 00010000:00010000:3.0:1787739061.813044:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739061.813078:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+9:3650:x1874579665667456:12345-0@lo:103:ldlm_bl.0 Request processed in 108us (225us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.813085:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1303 00000100:00100000:3.0:1787739061.813124:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665667456:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739061.813463:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f76940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17587:x1874579652473344:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 69350us (69586us total) trans 4294983391 rc 0/0 00000100:00100000:0.0:1787739061.813471:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23435 00000100:00100000:0.0:1787739061.813496:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f75e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665667584:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739061.813526:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739061.813546:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787739061.813552:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f77840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665667712:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739061.813572:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739061.813598:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665667584 00000100:00100000:0.0:1787739061.813602:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665667584 00000100:00100000:0.0:1787739061.813629:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665667712 00000100:00100000:0.0:1787739061.813632:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665667712 00000100:00100000:0.0:1787739061.813650:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1304 00000100:00100000:0.0:1787739061.813657:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432cf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+10:3652:x1874579665667584:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739061.813666:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739061.813902:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad06e lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1517:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad067 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.813915:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1517:0x0].0x0 (ffffa07305d56100) 00010000:00010000:0.0:1787739061.813922:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739061.813952:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432cf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+9:3652:x1874579665667584:12345-0@lo:103:ldlm_bl.0 Request processed in 295us (425us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739061.813958:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1304 00000100:00100000:0.0:1787739061.813974:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1305 00000100:00100000:0.0:1787739061.813981:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+8:3652:x1874579665667712:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739061.813991:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739061.814000:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad052 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad04b expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.814011:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07303c78700) 00010000:00010000:0.0:1787739061.814017:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739061.814051:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+7:3652:x1874579665667712:12345-0@lo:103:ldlm_bl.0 Request processed in 71us (478us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739061.814058:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1305 00000100:00100000:0.0:1787739061.814086:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f77840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665667712:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739061.814094:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f75e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665667584:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739061.814805:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739061.814813:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739061.836269:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739061.836311:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739061.836377:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739061.836734:0:14683:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652473856 00000100:00100000:3.0:1787739061.836744:0:14683:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652473856 00000100:00100000:3.0:1787739061.836827:0:14683:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652473984 00000100:00100000:3.0:1787739061.836830:0:14683:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652473984 00000100:00100000:3.0:1787739061.836859:0:14683:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 350 00000100:00100000:3.0:1787739061.836869:0:14683:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2407:x1874579652473856:12345-192.168.203.15@tcp:400:kworker.0 00000100:00100000:3.0:1787739061.836944:0:14683:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2407:x1874579652473856:12345-192.168.203.15@tcp:400:kworker.0 Request processed in 76us (693us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.836951:0:14683:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 350 00000100:00100000:3.0:1787739061.836967:0:14683:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 351 00000100:00100000:3.0:1787739061.836973:0:14683:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2407:x1874579652473984:12345-192.168.203.15@tcp:400:kworker.0 00000100:00100000:3.0:1787739061.836999:0:14683:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2407:x1874579652473984:12345-192.168.203.15@tcp:400:kworker.0 Request processed in 26us (621us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.837005:0:14683:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 351 00000100:00080000:0.0:1787739061.838459:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787739061.838469:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787739061.842690:0:6516:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652473728 00000100:00100000:2.0:1787739061.842704:0:6516:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652473728 00000100:00100000:3.0:1787739061.843498:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 449 00000100:00100000:3.0:1787739061.843510:0:6516:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0740ea13450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:53427813-ebb7-48fc-94b6-ef03330d7ee5+10:2407:x1874579652473728:12345-192.168.203.15@tcp:400:kworker.0 00000100:00100000:3.0:1787739061.843589:0:6516:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0740ea13450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:53427813-ebb7-48fc-94b6-ef03330d7ee5+10:2407:x1874579652473728:12345-192.168.203.15@tcp:400:kworker.0 Request processed in 81us (7278us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.843597:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 449 00000100:00100000:3.0:1787739061.866207:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739061.866341:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652474112 00000100:00100000:1.0:1787739061.866351:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652474112 00000100:00100000:1.0:1787739061.866402:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23436 00000100:00100000:1.0:1787739061.866435:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17588:x1874579652474112:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739061.866457:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739061.866478:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad091 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094f9a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.866544:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad098 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.866557:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad098 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.866568:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad098 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.866581:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad09f lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.866588:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad09f lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.866594:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad09f lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.866630:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad098 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.866638:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad098 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.866644:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad098 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.866649:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad098 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.866655:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1cc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739061.866659:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad098 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.866666:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad098 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.866687:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305eae600) 00010000:00010000:1.0:1787739061.866690:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739061.866695:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad098 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.866701:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad098 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.866714:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad09f lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.866719:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad09f lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.866724:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad09f lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.866729:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739061.866732:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad09f lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.866738:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad09f lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.866745:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x82528618 (ffffa07305eaed00) 00010000:00010000:1.0:1787739061.866747:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739061.866750:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad09f lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.866756:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad09f lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.866787:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad091 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f9a expref: 7 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.866808:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305eae600) 00010000:00010000:1.0:1787739061.866811:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739061.866814:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad091 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f9a expref: 7 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.866826:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1dc00, rc 0) 00000100:00100000:1.0:1787739061.866883:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17588:x1874579652474112:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 472us (687us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739061.866890:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23436 00000100:00100000:2.0:1787739061.898073:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739061.898152:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652474240 00000100:00100000:0.0:1787739061.898161:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652474240 00000100:00100000:0.0:1787739061.898220:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23437 00000100:00100000:0.0:1787739061.898232:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17588:x1874579652474240:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739061.898253:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739061.898275:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad0a6 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094fa1 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.898341:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad0ad lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.898358:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad0ad lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.898372:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad0ad lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.898383:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad0b4 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.898390:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad0b4 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.898397:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad0b4 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.898468:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad0ad lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.898476:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad0ad lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.898481:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad0ad lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.898487:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad0ad lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.898494:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739061.898498:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad0ad lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.898506:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad0ad lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.898517:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064c00) 00010000:00010000:0.0:1787739061.898520:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739061.898524:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad0ad lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.898529:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad0ad lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.898541:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad0b4 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.898547:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad0b4 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.898553:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad0b4 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.898558:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739061.898560:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad0b4 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.898566:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad0b4 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.898573:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x82528618 (ffffa07307064400) 00010000:00010000:0.0:1787739061.898575:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739061.898578:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad0b4 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.898583:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad0b4 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.898615:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad0a6 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fa1 expref: 7 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.898624:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064c00) 00010000:00010000:0.0:1787739061.898627:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739061.898630:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad0a6 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fa1 expref: 7 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.898640:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b800, rc 0) 00000100:00100000:0.0:1787739061.898692:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17588:x1874579652474240:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 463us (626us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739061.898699:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23437 00000100:00100000:1.0:1787739061.908774:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739061.908964:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652474368 00000100:00100000:0.0:1787739061.908974:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652474368 00000100:00100000:0.0:1787739061.909035:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23438 00000100:00100000:0.0:1787739061.909046:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17588:x1874579652474368:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739061.909067:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739061.909089:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad0bb lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094fa8 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.909147:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad0c2 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.909160:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad0c2 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.909171:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad0c2 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739061.909219:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x5:0x0]: returning size 4096 00000004:00200000:0.0:1787739061.909225:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739061.909260:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad0bb lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fa8 expref: 8 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.909277:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad0c2 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fa8 expref: 7 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739061.909285:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064a00) 00010000:00010000:0.0:1787739061.909288:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739061.909291:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b000, rc 0) 00000100:00100000:0.0:1787739061.909360:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17588:x1874579652474368:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 316us (595us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739061.909368:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23438 00000100:00100000:0.0:1787739061.941608:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739061.941695:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652474624 00000100:00100000:1.0:1787739061.941706:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652474624 00000100:00100000:1.0:1787739061.941761:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23439 00000100:00100000:1.0:1787739061.941771:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17588:x1874579652474624:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:1.0:1787739061.941840:0:6523:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739061.941866:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad0c2 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fa8 expref: 7 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.941883:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064a00) 00010000:00010000:1.0:1787739061.941894:0:6523:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739061.941946:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad0c9 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.941958:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad0c9 lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.941968:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad0c9 lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.941979:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad0d0 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x82528618 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.941986:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad0d0 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.941992:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad0d0 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1787739061.942815:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].358, 2168 off543736 00000040:00080000:1.0:1787739061.942883:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].359, 2368 off595304 00000004:00080000:1.0:1787739061.942889:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730623da80 with version 684 00000004:00080000:1.0:1787739061.942914:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 684 rpc_version 684 00000020:00100000:1.0:1787739061.942944:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739061.942952:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074080f0b40 x1874579665667968/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 3976/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739061.942975:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f0b40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665667968:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739061.943061:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739061.943110:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739061.943333:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665667968 00000100:00100000:0.0:1787739061.943340:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665667968 00000100:00100000:0.0:1787739061.943394:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3121 00000100:00100000:0.0:1787739061.943402:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665667968:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739061.944106:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665667968:12345-0@lo:1000:osp_up0-1.0 Request processed in 703us (1042us total) trans 4294983392 rc 0/0 00000100:00100000:0.0:1787739061.944115:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3121 00000100:00100000:0.0:1787739061.944248:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f0b40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665667968:0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1787739061.945942:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad0d7 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x177:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.945965:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad0d7 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x177:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.945977:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad0d7 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x177:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.945996:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad0de lrc: 3/0,1 mode: --/PW res: [0x240000400:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.946003:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad0de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.946009:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad0de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.946053:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad0e5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.946060:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad0e5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.946069:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad0e5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739061.946089:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f3480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665668096:0@lo:101:mdt00_000.0 00000100:00100000:1.0:1787739061.946152:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739061.946196:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d600 going to sleep for 11 seconds 00000100:00100000:3.0:1787739061.946782:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665668096 00000100:00100000:3.0:1787739061.946852:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665668096 00000100:00100000:3.0:1787739061.946906:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3122 00000100:00100000:3.0:1787739061.946917:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665668096:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739061.946935:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739061.946953:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad0ec lrc: 2/0,0 mode: --/PW res: [0x200000401:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad0e5 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.946977:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad0ec lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad0e5 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.946990:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad0ec lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad0e5 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739061.946998:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc3:0x0].0x0 (ffffa07305d56b00) 00010000:00010000:3.0:1787739061.947001:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739061.947004:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab000, rc 0) 00000100:00100000:3.0:1787739061.947074:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665668096:12345-0@lo:101:mdt00_000.0 Request processed in 159us (921us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739061.947082:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3122 00000100:00100000:1.0:1787739061.947961:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f3480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665668096:0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739061.947984:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad0e5 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad0ec expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.948002:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad0e5 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad0ec expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739061.948037:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007ccd6852 reply state 00000000e7930645 transno 4294968379 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad0de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.948050:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad0de lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.948058:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc3:0x0].0x0 (ffffa07305eae200) 00010000:00010000:1.0:1787739061.948061:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739061.948064:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787739061.948070:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad0e5 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad0ec expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.948078:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad0e5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad0ec expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739061.948089:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xc3:0x0] 00000004:00010000:1.0:1787739061.948096:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007ccd6852 reply state 00000000e7930645 transno 4294968379 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad0d7 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x177:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.948103:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad0d7 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x177:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.948109:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x177:0x0].0x0 (ffffa07305eaef00) 00010000:00010000:1.0:1787739061.948112:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739061.948113:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:1.0:1787739061.948119:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x177:0x0]: returning size 4096 00010000:00010000:1.0:1787739061.948128:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad0c9 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.948134:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad0c9 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.948140:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad0c9 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.948146:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad0c9 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.948152:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739061.948157:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad0c9 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.948164:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad0c9 lrc: 5/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.948174:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305eae600) 00010000:00010000:1.0:1787739061.948177:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739061.948180:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad0c9 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.948185:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad0c9 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739061.948201:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007ccd6852 reply state 00000000e7930645 transno 4294968379 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad0d0 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.948209:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad0d0 lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739061.948219:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x82528618 (ffffa07305eaed00) 00010000:00010000:1.0:1787739061.948221:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739061.948223:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 2 00000100:00100000:1.0:1787739061.948300:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17588:x1874579652474624:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 6529us (6700us total) trans 4294968379 rc 0/0 00000100:00100000:1.0:1787739061.948308:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23439 00000100:00100000:2.0:1787739061.994252:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739061.994774:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652474880 00000100:00100000:3.0:1787739061.994785:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652474880 00000100:00100000:3.0:1787739061.994849:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23440 00000100:00100000:3.0:1787739061.994874:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17590:x1874579652474880:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:3.0:1787739061.995198:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17590:x1874579652474880:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 339us (995us total) trans 0 rc -17/-17 00000100:00100000:3.0:1787739061.995226:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23440 00000100:00100000:1.0:1787739062.029192:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739062.029274:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652475008 00000100:00100000:2.0:1787739062.029283:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652475008 00000100:00100000:2.0:1787739062.029348:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23441 00000100:00100000:2.0:1787739062.029358:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17591:x1874579652475008:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739062.029377:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739062.029400:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad0f3 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094faf expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029489:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad0fa lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029504:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad0fa lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029521:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad0fa lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029535:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad101 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029543:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad101 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029550:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad101 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029622:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad108 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x177:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029630:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad108 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x177:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029637:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad108 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x177:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739062.029715:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x177:0x0]: returning size 4096 00000004:00200000:2.0:1787739062.029719:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:2.0:1787739062.029739:0:7944:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad108 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x177:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029752:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad0fa lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029758:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad0fa lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029768:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad0fa lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029774:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad0fa lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029781:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739062.029785:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad0fa lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029792:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad0fa lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029804:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07303c78300) 00010000:00010000:2.0:1787739062.029806:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739062.029810:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad0fa lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029817:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad0fa lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029830:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad101 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029836:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad101 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029842:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad101 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029848:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739062.029851:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad101 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029857:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad101 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029864:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x82528618 (ffffa07305eaed00) 00010000:00010000:2.0:1787739062.029866:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739062.029870:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad101 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029875:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad101 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029899:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad0f3 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094faf expref: 10 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029918:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad108 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x177:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094faf expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.029925:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x177:0x0].0x0 (ffffa07305eaef00) 00010000:00010000:2.0:1787739062.029927:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787739062.029930:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c000, rc 0) 00000100:00100000:2.0:1787739062.029989:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17591:x1874579652475008:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 632us (808us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.029996:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23441 00000100:00100000:2.0:1787739062.033076:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739062.033144:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652475136 00000100:00100000:1.0:1787739062.033153:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652475136 00000100:00100000:1.0:1787739062.033208:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23442 00000100:00100000:1.0:1787739062.033218:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17591:x1874579652475136:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739062.033238:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739062.033259:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad10f lrc: 2/0,0 mode: --/CR res: [0x240000400:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094fb6 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.033314:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad116 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.033343:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad116 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.033356:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad116 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739062.033402:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xc3:0x0]: returning size 4096 00000004:00200000:1.0:1787739062.033406:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739062.033465:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad10f lrc: 0/0,0 mode: --/CR res: [0x240000400:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fb6 expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.033487:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad116 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fb6 expref: 10 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.033497:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc3:0x0].0x0 (ffffa07305eae200) 00010000:00010000:1.0:1787739062.033499:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739062.033502:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c000, rc 0) 00000100:00100000:1.0:1787739062.033561:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17591:x1874579652475136:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 345us (494us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.033569:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23442 00000100:00100000:2.0:1787739062.037979:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739062.038517:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652475264 00000100:00100000:2.0:1787739062.038527:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652475264 00000100:00100000:2.0:1787739062.038580:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23443 00000100:00100000:2.0:1787739062.038590:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f7c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17591:x1874579652475264:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739062.038611:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739062.038631:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad11d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094fbd expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.038691:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad124 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.038705:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad0ec lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad0e5 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.038714:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad124 lrc: 4/1,0 mode: --/PR res: [0x200000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.038749:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad0ec lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ad0e5 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739062.038764:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6525:1874579665668224:0@lo:104: 00000100:00100000:2.0:1787739062.038825:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.038869:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413780 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.039254:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665668224 00000100:00100000:3.0:1787739062.039259:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665668224 00000100:00100000:3.0:1787739062.039298:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1150 00000100:00100000:3.0:1787739062.039306:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6525:x1874579665668224:12345-0@lo:104: 00010000:00010000:3.0:1787739062.039319:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad0e5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad0ec expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.039403:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739062.039408:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6525:x1874579665668224:12345-0@lo:104: Request processed in 104us (584us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.039444:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1150 00010000:00010000:3.0:1787739062.039465:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739062.039471:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad0e5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad0ec expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.039479:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739062.039484:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad0e5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ad0ec expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739062.039492:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad0e5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ad0ec expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.039517:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad0e5 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad0ec expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.039524:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739062.039542:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad0e5 lrc: 1/0,0 mode: --/PW res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad0ec expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.039550:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad0e5 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad0ec expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.039567:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739062.039569:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739062.039571:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739062.039581:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f75e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665668352:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739062.039610:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.039646:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665668352 00000100:00100000:3.0:1787739062.039650:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665668352 00010000:00010000:3.0:1787739062.039668:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad0ec lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ad0e5 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739062.039690:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1504 00000100:00100000:3.0:1787739062.039697:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f77840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3650:x1874579665668352:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739062.039706:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787739062.041305:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6525:1874579665668224:0@lo:104: 00010000:00010000:1.0:1787739062.041340:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad124 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.046443:0:6502:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1ad059 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad060 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739062.046463:0:6502:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1ad059 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad060 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.046510:0:6502:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1ad059 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad060 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.046517:0:6502:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739062.046547:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c200/0x7ee4453e4a1ad059 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad060 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787739062.046638:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983392 is committed 00000001:00080000:3.0:1787739062.046656:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983391 is committed 00000020:00080000:3.0:1787739062.046665:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 372 batchid updated 00000020:00080000:3.0:1787739062.046684:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07306241900 refcount 2 committed 0 result 0 batchid 373 00000020:00080000:3.0:1787739062.046688:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2300 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07306241f00 00000020:00080000:3.0:1787739062.046691:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].370 00000020:00080000:3.0:1787739062.046694:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2f00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306940a00 00000020:00080000:3.0:1787739062.046697:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].371 00000020:00080000:3.0:1787739062.046716:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 371 batchid updated 00000020:00080000:0.0:1787739062.046723:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 373 committed batchid 372 00000100:00100000:3.0:1787739062.046741:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787739062.046760:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665668480:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:3.0:1787739062.046829:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787739062.046887:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfe9 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1517:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.046896:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfe9 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1517:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.046903:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfe9 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1517:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.046909:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739062.046913:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfe9 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1517:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.046920:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfe9 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1517:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.046937:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1517:0x0].0x0 (ffffa07303c78b00) 00010000:00010000:3.0:1787739062.046940:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739062.046945:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfe9 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1517:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.046951:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1acfe9 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1517:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.046971:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1acff0 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.046977:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1acff0 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.046982:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1acff0 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.046987:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739062.046990:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1acff0 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.046995:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1acff0 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.047003:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc5:0x0].0x0 (ffffa07303c78600) 00010000:00010000:3.0:1787739062.047005:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739062.047008:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1acff0 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.047014:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1acff0 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.047091:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad0ec lrc: 0/0,0 mode: --/PW res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ad0e5 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.047105:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc3:0x0].0x0 (ffffa07305d56b00) 00010000:00010000:3.0:1787739062.047109:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad124 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.047117:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad124 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.047125:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad124 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.047155:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739062.047213:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f77840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+5:3650:x1874579665668352:12345-0@lo:103:ldlm_bl.0 Request processed in 7515us (7600us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.047222:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1504 00000100:00100000:3.0:1787739062.047244:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665668480 00000100:00100000:3.0:1787739062.047250:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665668480 00000100:00100000:3.0:1787739062.047290:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1306 00000100:00100000:3.0:1787739062.047297:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+7:3650:x1874579665668480:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:3.0:1787739062.047309:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739062.047322:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad060 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x62d06ae6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad059 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.047332:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x62d06ae6 (ffffa07305d56e00) 00010000:00010000:3.0:1787739062.047338:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739062.047366:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+6:3650:x1874579665668480:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 70us (538us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.047373:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1306 00000100:00100000:3.0:1787739062.047435:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665668480:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:3.0:1787739062.047452:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f75e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665668352:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739062.047461:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad124 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.047472:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad124 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739062.047519:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xc3:0x0]: returning size 4096 00000004:00200000:2.0:1787739062.047525:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739062.047567:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad11d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fbd expref: 16 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.047585:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad124 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fbd expref: 15 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.047593:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc3:0x0].0x0 (ffffa07305d56b00) 00010000:00010000:2.0:1787739062.047595:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739062.047598:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d800, rc 0) 00000100:00100000:2.0:1787739062.047648:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f7c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17591:x1874579652475264:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 9060us (9679us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.047656:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23443 00000004:00080000:0.0:1787739062.047749:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07303c6e900 with version 708 00000020:00080000:0.0:1787739062.047805:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 371 cancel update log [0x240000401:0x3:0x0].368: rc = 0 00000020:00080000:0.0:1787739062.047857:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 371 cancel update log [0x200000400:0x3:0x0].369: rc = 0 00000004:00080000:0.0:1787739062.047884:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07303c6e0c0 with version 709 00000004:00080000:2.0:1787739062.048154:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 708 rpc_version 708 00000020:00100000:2.0:1787739062.048179:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739062.048187:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07409fb7840 x1874579665668736/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739062.048206:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb7840 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665668736:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739062.048245:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.048276:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413900 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.048320:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665668736 00000100:00100000:1.0:1787739062.048326:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665668736 00000100:00100000:1.0:1787739062.048365:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3123 00000100:00100000:1.0:1787739062.048373:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665668736:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739062.048526:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665668736:12345-0@lo:1000:osp_up1-0.0 Request processed in 153us (279us total) trans 4294968380 rc 0/0 00000100:00100000:1.0:1787739062.048533:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3123 00000100:00100000:2.0:1787739062.048653:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb7840 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665668736:0@lo:1000:osp_up1-0.0 00000004:00080000:2.0:1787739062.048662:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 709 rpc_version 709 00000020:00080000:0.0:1787739062.052258:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 372 cancel update log [0x240000401:0x3:0x0].369: rc = 0 00000020:00080000:0.0:1787739062.052350:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 372 cancel update log [0x200000400:0x3:0x0].370: rc = 0 00000020:00080000:0.0:1787739062.052360:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 372 00000020:00100000:2.0:1787739062.053821:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739062.053839:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07409fb4b40 x1874579665668992/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739062.053865:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb4b40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665668992:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739062.053933:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.053980:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665668992 00000100:00100000:2.0:1787739062.053987:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665668992 00000100:00100000:2.0:1787739062.054045:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3124 00000100:00100000:2.0:1787739062.054054:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665668992:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739062.054189:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665668992:12345-0@lo:1000:osp_up1-0.0 Request processed in 135us (255us total) trans 4294968381 rc 0/0 00000100:00100000:2.0:1787739062.054196:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3124 00000100:00100000:2.0:1787739062.054222:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413900 going to sleep for 11 seconds 00000100:00100000:2.0:1787739062.054253:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb4b40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665668992:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739062.064567:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739062.064646:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652475520 00000100:00100000:0.0:1787739062.064655:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652475520 00000100:00100000:0.0:1787739062.064712:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23444 00000100:00100000:0.0:1787739062.064722:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17591:x1874579652475520:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:0.0:1787739062.064788:0:6524:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739062.064825:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad108 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x177:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094faf expref: 10 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.064844:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x177:0x0].0x0 (ffffa07305eaef00) 00010000:00010000:0.0:1787739062.064849:0:6524:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739062.064874:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad12b lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.064886:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad12b lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.064895:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad12b lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.064905:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad132 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x82528618 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.064913:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad132 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.064925:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad132 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.065004:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad139 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x177:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.065012:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad139 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x177:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.065020:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad139 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x177:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.065050:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad140 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.065060:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad116 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fb6 expref: 9 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.065068:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad140 lrc: 4/0,1 mode: --/EX res: [0x240000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.065101:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad116 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fb6 expref: 9 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.065117:0:6524:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad116 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fb6 expref: 9 pid: 6523 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739062.065131:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665669120:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739062.065222:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739062.076611:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665669120:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739062.076643:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad140 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739062.077571:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739062.077636:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652475648 00000100:00100000:0.0:1787739062.077644:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652475648 00010000:00010000:0.0:1787739062.077681:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad116 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fb6 expref: 10 pid: 6523 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739062.077722:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1505 00000100:00100000:0.0:1787739062.077732:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb70c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652475648:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739062.077747:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739062.077753:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad116 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fb6 expref: 10 pid: 6523 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.077765:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad116 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fb6 expref: 10 pid: 6523 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.077786:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad116 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fb6 expref: 10 pid: 6523 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.077799:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc3:0x0].0x0 (ffffa07305eae200) 00010000:00010000:0.0:1787739062.077803:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad140 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.077813:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad140 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.077821:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad140 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.077845:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739062.077893:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb70c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:2408:x1874579652475648:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 162us (331us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739062.077900:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1505 00010000:00010000:0.0:1787739062.077926:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad140 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.077933:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad140 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.077974:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ad147 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.077982:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ad147 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.077993:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ad147 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739062.078011:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665669248:0@lo:101:mdt00_001.0 00000100:00100000:0.0:1787739062.078062:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739062.078091:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739062.078107:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665669248 00000100:00100000:0.0:1787739062.078111:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665669248 00000100:00100000:0.0:1787739062.078141:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3125 00000100:00100000:0.0:1787739062.078148:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+5:6524:x1874579665669248:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739062.078163:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739062.078177:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad14e lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad147 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.078202:0:6529:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad124 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fbd expref: 14 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.078210:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad14e lrc: 4/0,0 mode: --/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad147 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.078231:0:6529:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad124 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fbd expref: 14 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.078243:0:6529:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad124 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fbd expref: 14 pid: 6525 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739062.078254:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432f0c0 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6529:1874579665669376:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739062.078294:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6ecc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739062.085356:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739062.085392:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652475776 00000100:00100000:2.0:1787739062.085400:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652475776 00010000:00010000:2.0:1787739062.085462:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad124 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fbd expref: 15 pid: 6525 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739062.085496:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1506 00000100:00100000:2.0:1787739062.085506:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074184c0450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2409:x1874579652475776:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739062.085521:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739062.085528:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad124 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fbd expref: 15 pid: 6525 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.085541:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad124 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fbd expref: 15 pid: 6525 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.085564:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc3:0x0].0x0 (ffffa07305d56b00) 00010000:00010000:2.0:1787739062.085571:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad14e lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad147 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.085588:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad14e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad147 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.085597:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad14e lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad147 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.085630:0:6508:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad14e lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad147 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739062.085645:0:6508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb4f00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665669504:0@lo:105: 00000100:00100000:2.0:1787739062.085689:0:6508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.085723:0:6508:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739062.085738:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665669504 00000100:00100000:2.0:1787739062.085741:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665669504 00000100:00100000:2.0:1787739062.085768:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1151 00000100:00100000:2.0:1787739062.085774:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb61c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665669504:12345-0@lo:105: 00010000:00010000:2.0:1787739062.085785:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ad147 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad14e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.085793:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ad147 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad14e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.085801:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ad147 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad14e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.085808:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ad147 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad14e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.085814:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ad147 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad14e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.085822:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ad147 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ad14e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.085835:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609ba00) 00000100:00100000:2.0:1787739062.085869:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb61c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665669504:12345-0@lo:105: Request processed in 95us (180us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.085875:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1151 00000100:00100000:2.0:1787739062.085915:0:6508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb4f00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665669504:0@lo:105: 00010000:00010000:2.0:1787739062.085931:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739062.085965:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074184c0450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2409:x1874579652475776:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 460us (616us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.085972:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1506 00010000:00010000:0.0:1787739062.087693:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad124 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fbd expref: 14 pid: 6525 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739062.087717:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432f0c0 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6529:1874579665669376:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739062.087744:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad14e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad147 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.087755:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc3:0x0].0x0 (ffffa07305d56b00) 00010000:00010000:0.0:1787739062.087758:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739062.087761:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a600, rc 0) 00000100:00100000:0.0:1787739062.087931:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:6524:x1874579665669248:12345-0@lo:101:mdt00_001.0 Request processed in 9781us (9866us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739062.087941:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3125 00000100:00100000:2.0:1787739062.089150:0:6524:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983392 gen 2 00000004:00080000:2.0:1787739062.089161:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983386 committed transno 4294983392 00000020:00080000:2.0:1787739062.089168:0:6524:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa0730623d000 refcount 2 committed 0 result 0 batchid 364 00000020:00080000:2.0:1787739062.089172:0:6524:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2100 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730623d840 00000020:00080000:2.0:1787739062.089175:0:6524:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].358 00000020:00080000:2.0:1787739062.089178:0:6524:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2900 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306940200 00000020:00080000:2.0:1787739062.089180:0:6524:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].357 00000004:00080000:2.0:1787739062.089217:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983388 committed transno 4294983392 00000004:00080000:2.0:1787739062.089228:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983389 committed transno 4294983392 00000004:00080000:2.0:1787739062.089235:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983390 committed transno 4294983392 00000004:00080000:2.0:1787739062.089244:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983392 committed transno 4294983392 00000100:00100000:2.0:1787739062.089256:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665669248:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739062.089277:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ad147 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad14e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00080000:0.0:1787739062.089363:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 364 committed batchid 363 00000020:00080000:0.0:1787739062.089456:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 363 00000100:00100000:3.0:1787739062.090847:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665669632:0@lo:403:mdt00_001.0 00000100:00100000:3.0:1787739062.090924:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.090973:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404180 going to sleep for 11 seconds 00000100:00100000:2.0:1787739062.091063:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665669632 00000100:00100000:2.0:1787739062.091071:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665669632 00000100:00100000:2.0:1787739062.091163:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3126 00000100:00100000:2.0:1787739062.091172:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f77840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:6524:x1874579665669632:12345-0@lo:403:mdt00_001.0 00000100:00100000:3.0:1787739062.099902:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f77840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:6524:x1874579665669632:12345-0@lo:403:mdt00_001.0 Request processed in 8723us (8970us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.099918:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3126 00000100:00100000:0.0:1787739062.099941:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561e1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665669632:0@lo:403:mdt00_001.0 00000040:00080000:0.0:1787739062.101042:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].359, 664 off544400 00000040:00080000:0.0:1787739062.101223:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].360, 744 off596048 00000004:00080000:0.0:1787739062.101230:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07303c6ecc0 with version 685 00000004:00080000:2.0:1787739062.101328:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 685 rpc_version 685 00000020:00100000:2.0:1787739062.101363:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739062.101371:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07409fb70c0 x1874579665669888/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1704/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739062.101393:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb70c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665669888:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739062.101480:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.101510:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739062.101930:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665669888 00000100:00100000:0.0:1787739062.101938:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665669888 00000100:00100000:0.0:1787739062.102036:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3127 00000100:00100000:0.0:1787739062.102047:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410b0b050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:7467:x1874579665669888:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739062.102295:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410b0b050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665669888:12345-0@lo:1000:osp_up0-1.0 Request processed in 249us (814us total) trans 4294983393 rc 0/0 00000100:00100000:0.0:1787739062.102303:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3127 00000100:00100000:3.0:1787739062.102458:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb70c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665669888:0@lo:1000:osp_up0-1.0 00000004:00200000:0.0:1787739062.110934:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x177:0x0]: returning size 4096 00010000:00010000:0.0:1787739062.110965:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad140 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.110977:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad140 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.110985:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad140 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.110992:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739062.110997:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad140 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.111009:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad140 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.111036:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc3:0x0].0x0 (ffffa07305eae200) 00010000:00010000:0.0:1787739062.111039:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739062.111044:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad140 lrc: 2/0,0 mode: --/EX res: [0x240000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.111051:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad140 lrc: 0/0,0 mode: --/EX res: [0x240000400:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.111065:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ad147 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad14e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.111074:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ad147 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad14e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.111082:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ad147 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad14e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.111110:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739062.111119:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xc3:0x0] 00010000:00010000:0.0:1787739062.111126:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad139 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x177:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.111133:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad139 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x177:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.111139:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad139 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x177:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.111150:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739062.111153:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad139 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x177:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.111158:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad139 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x177:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.111165:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x177:0x0].0x0 (ffffa07305eaef00) 00010000:00010000:0.0:1787739062.111168:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739062.111171:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad139 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x177:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.111176:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad139 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x177:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.111194:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad12b lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.111201:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad12b lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.111208:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad12b lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.111214:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad12b lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.111220:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739062.111222:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad12b lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.111232:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad12b lrc: 5/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.111239:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064c00) 00010000:00010000:0.0:1787739062.111242:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739062.111245:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad12b lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.111251:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad12b lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739062.111266:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d959d449 reply state 00000000d1f3e499 transno 4294968382 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad132 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.111278:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad132 lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.111286:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x82528618 (ffffa07305eaed00) 00010000:00010000:0.0:1787739062.111288:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739062.111291:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787739062.111447:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739062.111455:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ad147 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad14e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.111464:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ba00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739062.111469:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ad147 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad14e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739062.111479:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ad147 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad14e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.111513:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ad147 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad14e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.111521:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739062.111551:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ad147 lrc: 1/0,0 mode: --/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad14e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.111560:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ba00/0x7ee4453e4a1ad147 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad14e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.111588:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739062.111590:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739062.111593:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739062.111602:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787739062.111621:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f75e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665670016:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739062.111686:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739062.112935:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17591:x1874579652475520:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 48208us (48369us total) trans 4294968382 rc 0/0 00000100:00100000:0.0:1787739062.112948:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23444 00000100:00100000:0.0:1787739062.112997:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665670016 00000100:00100000:0.0:1787739062.113004:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665670016 00000100:00100000:0.0:1787739062.113062:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1307 00000100:00100000:0.0:1787739062.113071:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+7:3653:x1874579665670016:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739062.113085:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739062.113107:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad14e lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad147 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.113125:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc3:0x0].0x0 (ffffa07305d56b00) 00010000:00010000:0.0:1787739062.113134:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739062.113191:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+6:3653:x1874579665670016:12345-0@lo:103:ldlm_bl.0 Request processed in 120us (1503us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739062.113198:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1307 00000100:00100000:0.0:1787739062.113250:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f75e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665670016:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739062.164008:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739062.164153:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652475904 00000100:00100000:2.0:1787739062.164163:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652475904 00000100:00100000:2.0:1787739062.164230:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23445 00000100:00100000:2.0:1787739062.164240:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f75e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17592:x1874579652475904:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739062.164265:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739062.164290:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad155 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094fc4 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.164360:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad15c lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.164382:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad15c lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.164395:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad15c lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.164436:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad163 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.164445:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad163 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.164452:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad163 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.164489:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad15c lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.164496:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad15c lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.164503:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad15c lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.164509:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad15c lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.164515:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739062.164519:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad15c lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.164527:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad15c lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.164538:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07303c78300) 00010000:00010000:2.0:1787739062.164542:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739062.164546:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad15c lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.164553:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad15c lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.164566:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad163 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.164572:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad163 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.164578:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad163 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.164583:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633cc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739062.164586:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad163 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.164592:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad163 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.164598:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xda636067 (ffffa07303c78f00) 00010000:00010000:2.0:1787739062.164601:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739062.164604:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad163 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.164610:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad163 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.164651:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad155 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fc4 expref: 11 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.164661:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07303c78300) 00010000:00010000:2.0:1787739062.164664:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739062.164730:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad155 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fc4 expref: 11 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.164745:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d200, rc 0) 00000100:00100000:2.0:1787739062.164877:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f75e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17592:x1874579652475904:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 638us (878us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739062.164885:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23445 00000100:00100000:0.0:1787739062.177685:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739062.177894:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652476032 00000100:00100000:2.0:1787739062.177903:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652476032 00000100:00100000:2.0:1787739062.177958:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23446 00000100:00100000:2.0:1787739062.177970:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17592:x1874579652476032:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739062.177990:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739062.178011:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad16a lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094fcb expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.178086:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad171 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.178100:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad171 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.178110:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad171 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.178123:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad178 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.178130:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad178 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.178136:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad178 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.178176:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad171 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.178184:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad171 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.178190:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad171 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.178197:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad171 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.178203:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633cc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739062.178207:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad171 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.178214:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad171 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.178227:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07303c78300) 00010000:00010000:2.0:1787739062.178230:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739062.178235:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad171 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.178241:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad171 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.178256:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad178 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.178261:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad178 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.178267:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad178 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.178272:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739062.178275:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad178 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.178280:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad178 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.178287:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xda636067 (ffffa07303c78f00) 00010000:00010000:2.0:1787739062.178289:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739062.178292:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad178 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.178297:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad178 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.178328:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad16a lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fcb expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.178339:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07303c78300) 00010000:00010000:2.0:1787739062.178341:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739062.178345:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad16a lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fcb expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.178358:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d200, rc 0) 00000100:00100000:2.0:1787739062.178475:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17592:x1874579652476032:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 507us (807us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739062.178481:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23446 00000100:00100000:3.0:1787739062.186367:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739062.187213:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652476160 00000100:00100000:0.0:1787739062.187225:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652476160 00000100:00100000:1.0:1787739062.187357:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23447 00000100:00100000:1.0:1787739062.187368:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17592:x1874579652476160:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739062.187397:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739062.187453:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad17f lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094fd2 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.187514:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad186 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.187529:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad186 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.187541:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad186 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739062.187585:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x5:0x0]: returning size 4096 00000004:00200000:1.0:1787739062.187590:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739062.187958:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad17f lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fd2 expref: 12 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.187982:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad186 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fd2 expref: 11 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.187991:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305eae600) 00010000:00010000:1.0:1787739062.187995:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739062.187998:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1ce00, rc 0) 00000100:00100000:1.0:1787739062.188061:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17592:x1874579652476160:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 695us (1703us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.188069:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23447 00000100:00100000:3.0:1787739062.198748:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739062.198938:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652476416 00000100:00100000:1.0:1787739062.198948:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652476416 00000100:00100000:1.0:1787739062.199007:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23448 00000100:00100000:1.0:1787739062.199016:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17592:x1874579652476416:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:1.0:1787739062.199151:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f1680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665670144:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739062.199248:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.199306:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.199904:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665670144 00000100:00100000:3.0:1787739062.199910:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665670144 00000100:00100000:3.0:1787739062.199954:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3128 00000100:00100000:3.0:1787739062.199962:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665670144:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739062.200067:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665670144:12345-0@lo:1000:mdt00_003.0 Request processed in 106us (818us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.200075:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3128 00000100:00100000:1.0:1787739062.200204:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f1680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665670144:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739062.200257:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f1680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665670272:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739062.200285:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.200318:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739062.200395:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665670272 00000100:00100000:0.0:1787739062.200400:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665670272 00000100:00100000:0.0:1787739062.200464:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3129 00000100:00100000:0.0:1787739062.200473:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665670272:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739062.200542:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665670272:12345-0@lo:1000:mdt00_003.0 Request processed in 71us (255us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739062.200549:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3129 00000100:00100000:1.0:1787739062.201256:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f1680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665670272:0@lo:1000:mdt00_003.0 00010000:00010000:1.0:1787739062.201314:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad18d lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.201336:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad18d lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.201348:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad18d lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.201363:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f1680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665670400:0@lo:101:mdt00_003.0 00000100:00100000:1.0:1787739062.201410:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.201616:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739062.209988:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665670400 00000100:00100000:2.0:1787739062.210002:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665670400 00000100:00100000:3.0:1787739062.212130:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3130 00000100:00100000:3.0:1787739062.212142:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665670400:12345-0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739062.212162:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739062.212185:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad194 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad18d expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.212217:0:12410:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad186 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fd2 expref: 10 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.212228:0:12410:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad194 lrc: 4/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad18d expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.212260:0:12410:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad186 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fd2 expref: 10 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.212274:0:12410:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad186 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fd2 expref: 10 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739062.212287:0:12410:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f77840 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001_UUID:12410:1874579665670528:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739062.212973:0:12410:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739062.223278:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739062.223557:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652476544 00000100:00100000:2.0:1787739062.223567:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652476544 00010000:00010000:2.0:1787739062.223606:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad186 lrc: 5/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fd2 expref: 11 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739062.223653:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1507 00000100:00100000:2.0:1787739062.223664:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2408:x1874579652476544:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739062.223687:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739062.223694:0:6510:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad186 lrc: 5/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fd2 expref: 11 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.223708:0:6510:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad186 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fd2 expref: 11 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.223731:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305eae600) 00010000:00010000:2.0:1787739062.223738:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad194 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad18d expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.223750:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad194 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad18d expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.223760:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad194 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad18d expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.223806:0:6510:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad194 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad18d expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739062.223821:0:6510:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb4f00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6510:1874579665670656:0@lo:105: 00010000:00010000:3.0:1787739062.223850:0:12410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad186 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fd2 expref: 11 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739062.223869:0:6510:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.223881:0:12410:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f77840 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001_UUID:12410:1874579665670528:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739062.223910:0:6510:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d840 going to sleep for 11 seconds 00010000:00010000:3.0:1787739062.223914:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad194 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad18d expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.223925:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305eae600) 00010000:00010000:3.0:1787739062.223928:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739062.223932:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abe00, rc 0) 00000100:00100000:2.0:1787739062.223933:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665670656 00000100:00100000:2.0:1787739062.223938:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665670656 00000100:00100000:2.0:1787739062.223971:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1152 00000100:00100000:2.0:1787739062.223979:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6510:x1874579665670656:12345-0@lo:105: 00010000:00010000:2.0:1787739062.223990:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad18d lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad194 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739062.223996:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665670400:12345-0@lo:101:mdt00_003.0 Request processed in 11853us (22582us total) trans 0 rc 0/0 00010000:00010000:2.0:1787739062.224005:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad18d lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad194 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739062.224010:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3130 00010000:00010000:2.0:1787739062.224032:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad18d lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad194 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.224041:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad18d lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad194 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.224048:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad18d lrc: 5/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad194 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.224056:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad18d lrc: 5/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ad194 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.224070:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07441a1c800) 00000100:00100000:2.0:1787739062.224109:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6510:x1874579665670656:12345-0@lo:105: Request processed in 131us (240us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.224115:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1152 00000100:00100000:3.0:1787739062.224154:0:6510:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb4f00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6510:1874579665670656:0@lo:105: 00010000:00010000:3.0:1787739062.224168:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739062.224214:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432f0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652476544:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 552us (946us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.224222:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1507 00000100:00100000:1.0:1787739062.225148:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f1680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665670400:0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739062.225167:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad18d lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad194 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739062.225188:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739062.225220:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1de00/0x7ee4453e4a1ad19b lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.225231:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1de00/0x7ee4453e4a1ad19b lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.225240:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1de00/0x7ee4453e4a1ad19b lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.225256:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f1680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665670784:0@lo:101:mdt00_003.0 00000100:00100000:1.0:1787739062.225296:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.225328:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d600 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.225456:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665670784 00000100:00100000:3.0:1787739062.225464:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665670784 00000100:00100000:3.0:1787739062.225507:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3131 00000100:00100000:3.0:1787739062.225516:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665670784:12345-0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739062.225563:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739062.225581:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad1a2 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad19b expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.225605:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad1a2 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad19b expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.225618:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad1a2 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad19b expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.225628:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xda636067 (ffffa074039f2d00) 00010000:00010000:3.0:1787739062.225631:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739062.225634:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab200, rc 0) 00000100:00100000:3.0:1787739062.225684:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665670784:12345-0@lo:101:mdt00_003.0 Request processed in 169us (387us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.225691:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3131 00000100:00100000:1.0:1787739062.225762:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f1680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665670784:0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739062.225773:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1de00/0x7ee4453e4a1ad19b lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad1a2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.225783:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1de00/0x7ee4453e4a1ad19b lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad1a2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739062.225804:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:1.0:1787739062.225836:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f1680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665670912:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739062.225862:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.225920:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739062.233293:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665670912 00000100:00100000:0.0:1787739062.233308:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665670912 00000100:00100000:0.0:1787739062.233383:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3132 00000100:00100000:0.0:1787739062.233394:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665670912:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739062.233539:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665670912:12345-0@lo:1000:mdt00_003.0 Request processed in 147us (7675us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739062.233547:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3132 00000100:00100000:3.0:1787739062.233597:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f1680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665670912:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739062.233673:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f77840 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665671040:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739062.233730:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.233773:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404180 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.233788:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665671040 00000100:00100000:1.0:1787739062.233794:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665671040 00000100:00100000:1.0:1787739062.233828:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3133 00000100:00100000:1.0:1787739062.233835:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665671040:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739062.233909:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665671040:12345-0@lo:1000:mdt00_003.0 Request processed in 74us (179us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739062.233917:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3133 00000100:00100000:3.0:1787739062.233957:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f77840 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665671040:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739062.234050:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d2940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665671168:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739062.234075:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.234101:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404180 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.234400:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665671168 00000100:00100000:1.0:1787739062.234405:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665671168 00000100:00100000:1.0:1787739062.234464:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3134 00000100:00100000:1.0:1787739062.234470:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665671168:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739062.234533:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665671168:12345-0@lo:1000:mdt00_003.0 Request processed in 62us (456us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739062.234539:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3134 00000100:00100000:1.0:1787739062.234568:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d2940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665671168:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739062.234614:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f2580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665671296:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739062.234654:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.234683:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.234695:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665671296 00000100:00100000:1.0:1787739062.234699:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665671296 00000100:00100000:1.0:1787739062.234716:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3135 00000100:00100000:1.0:1787739062.234722:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665671296:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739062.234771:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665671296:12345-0@lo:1000:mdt00_003.0 Request processed in 49us (116us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.234777:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3135 00000100:00100000:1.0:1787739062.234804:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f2580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665671296:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739062.234845:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f2580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665671424:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739062.234868:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.234893:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.234931:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665671424 00000100:00100000:3.0:1787739062.234937:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665671424 00000100:00100000:3.0:1787739062.234970:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3136 00000100:00100000:3.0:1787739062.234977:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665671424:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739062.235064:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665671424:12345-0@lo:1000:mdt00_003.0 Request processed in 87us (194us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.235072:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3136 00000100:00100000:1.0:1787739062.235209:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f2580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665671424:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739062.235245:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f2580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665671552:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739062.235268:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.235290:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739062.235341:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665671552 00000100:00100000:2.0:1787739062.235346:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665671552 00000100:00100000:2.0:1787739062.235506:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3137 00000100:00100000:2.0:1787739062.235514:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665671552:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739062.235569:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665671552:12345-0@lo:1000:mdt00_003.0 Request processed in 56us (300us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.235575:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3137 00000100:00100000:1.0:1787739062.236096:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f2580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665671552:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739062.236138:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f2580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665671680:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739062.236165:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.236220:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739062.236255:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665671680 00000100:00100000:0.0:1787739062.236260:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665671680 00000100:00100000:0.0:1787739062.236336:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3138 00000100:00100000:0.0:1787739062.236344:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665671680:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739062.236675:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665671680:12345-0@lo:1000:mdt00_003.0 Request processed in 323us (499us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739062.236684:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3138 00000100:00100000:1.0:1787739062.237039:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f2580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665671680:0@lo:1000:mdt00_003.0 00000040:00080000:1.0:1787739062.238181:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].373, 2168 off570552 00000040:00080000:1.0:1787739062.238271:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].372, 2368 off617376 00000004:00080000:1.0:1787739062.238277:0:7944:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730623d600 with version 710 00000004:00080000:2.0:1787739062.238774:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 710 rpc_version 710 00000020:00100000:2.0:1787739062.238813:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739062.238822:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07409fb5680 x1874579665671936/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739062.238844:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb5680 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665671936:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739062.238910:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.238942:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ef00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.238990:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665671936 00000100:00100000:1.0:1787739062.238997:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665671936 00000100:00100000:1.0:1787739062.239053:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3139 00000100:00100000:1.0:1787739062.239062:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665671936:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739062.239615:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+11:7740:x1874579665671936:12345-0@lo:1000:osp_up1-0.0 Request processed in 554us (704us total) trans 4294968383 rc 0/0 00000100:00100000:1.0:1787739062.239636:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3139 00000100:00100000:2.0:1787739062.239923:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb5680 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665671936:0@lo:1000:osp_up1-0.0 00010000:00010000:1.0:1787739062.240070:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad1a9 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1518:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.240085:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad1a9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.240096:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad1a9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.240115:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad1b0 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.240121:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad1b0 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.240129:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad1b0 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.240155:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad1b7 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.240162:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad1b7 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.240171:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad1b7 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.240186:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f2580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665672064:0@lo:101:mdt00_003.0 00000100:00100000:1.0:1787739062.240216:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.240283:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623df00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739062.240364:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665672064 00000100:00100000:2.0:1787739062.240592:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665672064 00000100:00100000:2.0:1787739062.240766:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3140 00000100:00100000:2.0:1787739062.240776:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+11:7944:x1874579665672064:12345-0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739062.240792:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739062.240808:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad1be lrc: 2/0,0 mode: --/PW res: [0x240000401:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad1b7 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.240828:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad1be lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad1b7 expref: 12 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.240840:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad1be lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad1b7 expref: 12 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.240849:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc6:0x0].0x0 (ffffa07303c78300) 00010000:00010000:2.0:1787739062.240852:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739062.240855:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d000, rc 0) 00000100:00100000:2.0:1787739062.240899:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+12:7944:x1874579665672064:12345-0@lo:101:mdt00_003.0 Request processed in 125us (682us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.240906:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3140 00000100:00100000:0.0:1787739062.240963:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f2580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665672064:0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739062.240977:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad1b7 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad1be expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.240986:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad1b7 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad1be expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739062.241004:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a351a324 reply state 0000000002377ef5 transno 4294983394 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad1b0 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.241016:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad1b0 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.241032:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc6:0x0].0x0 (ffffa07305eaec00) 00010000:00010000:0.0:1787739062.241034:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739062.241037:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739062.241042:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad1b7 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad1be expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.241050:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad1b7 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad1be expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739062.241061:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xc6:0x0] 00000004:00010000:0.0:1787739062.241067:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a351a324 reply state 0000000002377ef5 transno 4294983394 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad1a9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.241075:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad1a9 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.241081:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1518:0x0].0x0 (ffffa07305eae100) 00010000:00010000:0.0:1787739062.241083:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739062.241085:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787739062.241089:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1518:0x0]: returning size 4096 00000004:00080000:0.0:1787739062.241096:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739062.241105:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad18d lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad194 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.241113:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad18d lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad194 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.241120:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad18d lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad194 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.241139:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739062.241143:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739062.241147:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1de00/0x7ee4453e4a1ad19b lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad1a2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.241154:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1de00/0x7ee4453e4a1ad19b lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad1a2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739062.241221:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17592:x1874579652476416:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 42204us (42483us total) trans 4294983394 rc 0/0 00000100:00100000:0.0:1787739062.241233:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23448 00010000:00010000:1.0:1787739062.241618:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739062.241633:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad18d lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad194 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.241642:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739062.241647:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad18d lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad194 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739062.241655:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad18d lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad194 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.241796:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad18d lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad194 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.241808:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1787739062.241861:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665672192:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739062.241901:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787739062.241953:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad18d lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad194 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.241962:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad18d lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad194 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.241981:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739062.241983:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739062.241986:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739062.247992:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665672192 00000100:00100000:1.0:1787739062.248009:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665672192 00000100:00100000:1.0:1787739062.248091:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1308 00000100:00100000:1.0:1787739062.248101:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+12:3653:x1874579665672192:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739062.248118:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739062.248142:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad194 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad18d expref: 12 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.248161:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305eae600) 00010000:00010000:1.0:1787739062.248171:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739062.252876:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+11:3653:x1874579665672192:12345-0@lo:103:ldlm_bl.0 Request processed in 4770us (10968us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.252891:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1308 00000100:00100000:0.0:1787739062.260123:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665672192:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739062.269039:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739062.269153:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652476800 00000100:00100000:2.0:1787739062.269162:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652476800 00000100:00100000:2.0:1787739062.269223:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23449 00000100:00100000:2.0:1787739062.269233:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17593:x1874579652476800:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:2.0:1787739062.269429:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17593:x1874579652476800:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 179us (389us total) trans 0 rc -17/-17 00000100:00100000:2.0:1787739062.269436:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23449 00000100:00100000:0.0:1787739062.309093:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739062.309158:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652476928 00000100:00100000:2.0:1787739062.309169:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652476928 00000100:00100000:2.0:1787739062.309233:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23450 00000100:00100000:2.0:1787739062.309245:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17594:x1874579652476928:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739062.309265:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739062.309285:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1ad1c5 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094fd9 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.309359:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad1cc lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.309381:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad1cc lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.309392:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad1cc lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.309404:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad1d3 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.309445:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad1a2 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad19b expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.309454:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad1d3 lrc: 4/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.309486:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad1a2 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ad19b expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739062.309501:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d87480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665672320:0@lo:104: 00000100:00100000:2.0:1787739062.309552:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.309605:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ef00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739062.309745:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665672320 00000100:00100000:0.0:1787739062.309751:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665672320 00000100:00100000:0.0:1787739062.309792:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1153 00000100:00100000:0.0:1787739062.309800:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6524:x1874579665672320:12345-0@lo:104: 00010000:00010000:0.0:1787739062.309824:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1de00/0x7ee4453e4a1ad19b lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad1a2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.309888:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787739062.309894:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6524:x1874579665672320:12345-0@lo:104: Request processed in 97us (342us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739062.309901:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1153 00010000:00010000:0.0:1787739062.309922:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739062.309929:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1de00/0x7ee4453e4a1ad19b lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad1a2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.309939:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1de00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739062.309944:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1de00/0x7ee4453e4a1ad19b lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ad1a2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739062.309953:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1de00/0x7ee4453e4a1ad19b lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ad1a2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.309982:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1de00/0x7ee4453e4a1ad19b lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad1a2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.309989:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739062.310019:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1de00/0x7ee4453e4a1ad19b lrc: 1/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad1a2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.310037:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1de00/0x7ee4453e4a1ad19b lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad1a2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.310055:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739062.310058:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665672448:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739062.310059:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739062.310062:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739062.310092:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739062.310167:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665672448 00000100:00100000:0.0:1787739062.310171:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665672448 00010000:00010000:0.0:1787739062.310189:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad1a2 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ad19b expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739062.310217:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1508 00000100:00100000:0.0:1787739062.310224:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d852c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+11:3650:x1874579665672448:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739062.310233:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787739062.310952:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d87480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665672320:0@lo:104: 00010000:00010000:1.0:1787739062.310982:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad1d3 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787739062.320124:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968383 is committed 00000001:00080000:1.0:1787739062.320153:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968382 is committed 00000020:00080000:1.0:1787739062.320161:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 364 batchid updated 00000020:00080000:1.0:1787739062.320182:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa0730623dc00 refcount 2 committed 0 result 0 batchid 365 00000020:00080000:1.0:1787739062.320188:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2c00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730623da80 00000020:00080000:1.0:1787739062.320192:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].359 00000020:00080000:1.0:1787739062.320195:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2600 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07303c9b200 00000020:00080000:1.0:1787739062.320197:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].358 00000020:00080000:1.0:1787739062.320218:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 365 committed batchid 364 00000004:00080000:1.0:1787739062.320350:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730623d480 with version 686 00000020:00080000:1.0:1787739062.320361:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 364 cancel update log [0x200000401:0x3:0x0].358: rc = 0 00000020:00080000:1.0:1787739062.320403:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 364 cancel update log [0x240000400:0x3:0x0].357: rc = 0 00000020:00080000:1.0:1787739062.320410:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 364 00010000:00010000:1.0:1787739062.320487:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad1a2 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ad19b expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.320505:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xda636067 (ffffa074039f2d00) 00010000:00010000:1.0:1787739062.320511:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad1d3 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.320521:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad1d3 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.320529:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad1d3 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.320560:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739062.320690:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad0d0 lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320699:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad0d0 lrc: 2/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320715:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad0d0 lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320722:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739062.320726:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad0d0 lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320734:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad0d0 lrc: 4/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320745:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x82528618 (ffffa07305eaed00) 00010000:00010000:3.0:1787739062.320748:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739062.320752:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad0d0 lrc: 2/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320759:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad0d0 lrc: 0/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320770:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad0d7 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x177:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320776:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad0d7 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x177:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320781:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad0d7 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x177:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320787:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1cc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739062.320789:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad0d7 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x177:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320795:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad0d7 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x177:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320802:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x177:0x0].0x0 (ffffa07305eaef00) 00010000:00010000:3.0:1787739062.320804:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739062.320807:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad0d7 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x177:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320821:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad0d7 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x177:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320837:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad0de lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320843:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad0de lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320849:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad0de lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320854:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ca00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739062.320857:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad0de lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320863:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad0de lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320871:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc3:0x0].0x0 (ffffa07305eae200) 00010000:00010000:3.0:1787739062.320873:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739062.320877:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad0de lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320882:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad0de lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320899:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad132 lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320905:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad132 lrc: 2/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320911:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad132 lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320916:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739062.320919:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad132 lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320924:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad132 lrc: 4/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320931:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x82528618 (ffffa07305eaed00) 00010000:00010000:3.0:1787739062.320933:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739062.320936:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad132 lrc: 2/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.320941:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad132 lrc: 0/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0x82528618 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.322207:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d852c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+6:3650:x1874579665672448:12345-0@lo:103:ldlm_bl.0 Request processed in 11973us (12104us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.322222:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1508 00000004:00080000:1.0:1787739062.322276:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 686 rpc_version 686 00000020:00100000:1.0:1787739062.322309:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739062.322318:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074080f0b40 x1874579665672704/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739062.322339:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f0b40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665672704:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739062.322381:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.322437:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d000 going to sleep for 11 seconds 00010000:00010000:1.0:1787739062.322456:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad1d3 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.322464:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad1d3 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.322556:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665672832:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739062.322596:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.322616:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0efc00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.322632:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665672704 00000100:00100000:1.0:1787739062.322638:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665672704 00000100:00100000:1.0:1787739062.322682:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665672832 00000100:00100000:1.0:1787739062.322686:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665672832 00000100:00100000:1.0:1787739062.322701:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3141 00000100:00100000:1.0:1787739062.322717:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665672704:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739062.322858:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665672704:12345-0@lo:1000:osp_up0-1.0 Request processed in 141us (475us total) trans 4294983395 rc 0/0 00000100:00100000:1.0:1787739062.322864:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3141 00000100:00100000:1.0:1787739062.322879:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3142 00000100:00100000:1.0:1787739062.322885:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665672832:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739062.322932:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665672832:12345-0@lo:1000:mdt00_001.0 Request processed in 47us (336us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.322938:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3142 00000100:00100000:1.0:1787739062.322982:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f0b40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665672704:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739062.323005:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665672832:0@lo:1000:mdt00_001.0 00010000:00010000:1.0:1787739062.323061:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad1da lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1518:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.323077:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad1da lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.323088:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad1da lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739062.323111:0:6524:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad1da lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.323125:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad1cc lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.323132:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad1cc lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.323138:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad1cc lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.323144:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad1cc lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.323155:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633ce00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739062.323160:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad1cc lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.323167:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad1cc lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.323179:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07303c78f00) 00010000:00010000:1.0:1787739062.323182:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739062.323187:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad1cc lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.323194:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad1cc lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.323208:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad1d3 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.323214:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad1d3 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.323221:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad1d3 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.323226:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633de00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739062.323229:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad1d3 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.323235:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad1d3 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.323247:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xda636067 (ffffa074039f2d00) 00010000:00010000:1.0:1787739062.323249:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739062.323252:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad1d3 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.323257:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad1d3 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.323293:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1ad1c5 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fd9 expref: 8 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.323310:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad1da lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fd9 expref: 7 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.323317:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1518:0x0].0x0 (ffffa07433235b00) 00010000:00010000:1.0:1787739062.323319:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739062.323321:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c800, rc 0) 00000100:00100000:1.0:1787739062.323365:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17594:x1874579652476928:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 14123us (14279us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.323373:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23450 00000100:00100000:2.0:1787739062.324253:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665672448:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739062.331214:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739062.331765:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652477056 00000100:00100000:1.0:1787739062.331775:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652477056 00000100:00100000:1.0:1787739062.331899:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23451 00000100:00100000:1.0:1787739062.331909:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17594:x1874579652477056:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739062.331931:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739062.331951:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad1e1 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1518:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094fe0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.332017:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad1e8 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1518:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.332057:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad1e8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1518:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.332068:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad1e8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1518:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739062.332159:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1518:0x0]: returning size 4096 00000004:00200000:1.0:1787739062.332164:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739062.332203:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad1e1 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fe0 expref: 18 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.332220:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad1e8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1518:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fe0 expref: 17 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.332229:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1518:0x0].0x0 (ffffa07305eae100) 00010000:00010000:1.0:1787739062.332232:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739062.332236:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d800, rc 0) 00000100:00100000:1.0:1787739062.332348:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17594:x1874579652477056:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 431us (1136us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.332356:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23451 00000100:00100000:0.0:1787739062.345965:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739062.346060:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652477184 00000100:00100000:3.0:1787739062.346070:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652477184 00000100:00100000:3.0:1787739062.346133:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23452 00000100:00100000:3.0:1787739062.346146:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17594:x1874579652477184:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739062.346167:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739062.346188:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad1ef lrc: 2/0,0 mode: --/CR res: [0x200000400:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094fe7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.346248:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad1f6 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.346263:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad1f6 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.346274:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad1f6 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739062.346327:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xc6:0x0]: returning size 4096 00000004:00200000:3.0:1787739062.346332:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739062.346656:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad1ef lrc: 0/0,0 mode: --/CR res: [0x200000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fe7 expref: 19 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.346686:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad1f6 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fe7 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.346696:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc6:0x0].0x0 (ffffa07305eaec00) 00010000:00010000:3.0:1787739062.346699:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739062.346702:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aaa00, rc 0) 00000100:00100000:3.0:1787739062.346761:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17594:x1874579652477184:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 618us (804us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.346769:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23452 00000100:00100000:0.0:1787739062.351354:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739062.354065:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652477312 00000100:00100000:1.0:1787739062.354076:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652477312 00000100:00100000:1.0:1787739062.354142:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23453 00000100:00100000:1.0:1787739062.354152:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17594:x1874579652477312:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739062.354173:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739062.354193:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad1fd lrc: 2/0,0 mode: --/CR res: [0x240000401:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094fee expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.354662:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad204 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.354678:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad1be lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad1b7 expref: 5 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.354687:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad204 lrc: 4/1,0 mode: --/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.354825:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad1be lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ad1b7 expref: 5 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.354844:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562e940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665672960:0@lo:104: 00000100:00100000:1.0:1787739062.354908:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.354999:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739062.355154:0:6507:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:2.0:1787739062.355168:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665672960 00000100:00100000:2.0:1787739062.355172:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665672960 00000100:00100000:2.0:1787739062.356121:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1154 00000100:00100000:2.0:1787739062.356143:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f913450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6524:x1874579665672960:12345-0@lo:104: 00010000:00010000:2.0:1787739062.356165:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad1b7 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad1be expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.356381:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739062.356388:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f913450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6524:x1874579665672960:12345-0@lo:104: Request processed in 254us (1480us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.356396:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1154 00000100:00100000:1.0:1787739062.358141:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562e940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665672960:0@lo:104: 00010000:00010000:1.0:1787739062.358172:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad204 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.359639:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739062.359651:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad1b7 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad1be expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.359663:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1dc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739062.359678:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad1b7 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xc6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ad1be expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739062.359689:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad1b7 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xc6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ad1be expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.359732:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad1b7 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xc6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad1be expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.359744:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739062.359780:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad1b7 lrc: 1/0,0 mode: --/PW res: [0x240000401:0xc6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad1be expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.359789:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad1b7 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xc6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad1be expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.359810:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739062.359812:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739062.359813:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739062.359926:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665673088:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739062.359994:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.360050:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665673088 00000100:00100000:1.0:1787739062.360058:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665673088 00010000:00010000:1.0:1787739062.360088:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad1be lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ad1b7 expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.360125:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1509 00000100:00100000:1.0:1787739062.360136:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+6:3651:x1874579665673088:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739062.360150:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00080000:2.0:1787739062.372567:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 365 batchid updated 00010000:00010000:0.0:1787739062.373466:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad1be lrc: 0/0,0 mode: --/PW res: [0x240000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ad1b7 expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.373491:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc6:0x0].0x0 (ffffa07303c78300) 00010000:00010000:0.0:1787739062.373497:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad204 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.373507:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad204 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.373516:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad204 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.373552:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739062.373632:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665673088:12345-0@lo:103:ldlm_bl.0 Request processed in 13497us (13637us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739062.373642:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1509 00000100:00100000:0.0:1787739062.373841:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665673088:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739062.373925:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad204 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.373934:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad204 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739062.373993:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xc6:0x0]: returning size 4096 00000004:00200000:2.0:1787739062.373999:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739062.374056:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad1fd lrc: 0/0,0 mode: --/CR res: [0x240000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fee expref: 9 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.374074:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad204 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fee expref: 8 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.374088:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc6:0x0].0x0 (ffffa07303c78300) 00010000:00010000:2.0:1787739062.374091:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739062.374094:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1da00, rc 0) 00000100:00100000:2.0:1787739062.374150:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17594:x1874579652477312:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 20000us (22806us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.374158:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23453 00000020:00080000:1.0:1787739062.374259:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 365 00000004:00080000:1.0:1787739062.374315:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0743c0ef240 with version 687 00000020:00080000:1.0:1787739062.374327:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 365 cancel update log [0x200000401:0x3:0x0].359: rc = 0 00000020:00080000:1.0:1787739062.374394:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 365 cancel update log [0x240000400:0x3:0x0].358: rc = 0 00000020:00080000:1.0:1787739062.374402:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 365 00000004:00080000:1.0:1787739062.374426:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 687 rpc_version 687 00000020:00100000:1.0:1787739062.374454:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739062.374461:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743562e940 x1874579665673344/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739062.374482:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562e940 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665673344:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739062.374527:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.374550:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.374565:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665673344 00000100:00100000:1.0:1787739062.374571:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665673344 00000100:00100000:1.0:1787739062.374609:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3143 00000100:00100000:1.0:1787739062.374618:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665673344:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739062.374951:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665673344:12345-0@lo:1000:osp_up0-1.0 Request processed in 333us (422us total) trans 4294983396 rc 0/0 00000100:00100000:1.0:1787739062.374959:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3143 00000100:00100000:1.0:1787739062.375003:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562e940 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665673344:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739062.396534:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739062.396636:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739062.396864:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652477440 00000100:00100000:1.0:1787739062.396873:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652477440 00000100:00100000:1.0:1787739062.396942:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1309 00000100:00100000:1.0:1787739062.396953:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2408:x1874579652477440:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:1.0:1787739062.396970:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739062.396997:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad1da lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fd9 expref: 8 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.397018:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1518:0x0].0x0 (ffffa07433235b00) 00010000:00010000:1.0:1787739062.397039:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739062.397100:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652477440:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 148us (463us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.397108:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1309 00000100:00100000:2.0:1787739062.398809:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652477696 00000100:00100000:2.0:1787739062.398868:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652477696 00000100:00100000:2.0:1787739062.399119:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23454 00000100:00100000:2.0:1787739062.399129:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17594:x1874579652477696:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:2.0:1787739062.399176:0:6523:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739062.399200:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad1e8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1518:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fe0 expref: 18 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.399216:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1518:0x0].0x0 (ffffa07305eae100) 00010000:00010000:2.0:1787739062.399220:0:6523:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739062.399267:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad20b lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.399276:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad20b lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.399286:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad20b lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739062.399307:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d84780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665673472:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1787739062.399361:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.399399:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.401187:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665673472 00000100:00100000:3.0:1787739062.401200:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665673472 00000100:00100000:3.0:1787739062.401267:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3144 00000100:00100000:3.0:1787739062.401277:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d87840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6523:x1874579665673472:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739062.401296:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739062.401320:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad212 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad20b expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.401349:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad212 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad20b expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.401365:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad212 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad20b expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.401373:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074039f2300) 00010000:00010000:3.0:1787739062.401376:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739062.401380:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab000, rc 0) 00000100:00100000:3.0:1787739062.401482:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d87840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6523:x1874579665673472:12345-0@lo:101:mdt00_000.0 Request processed in 207us (2120us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.401503:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3144 00000100:00100000:3.0:1787739062.401547:0:6523:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968383 gen 2 00000004:00080000:3.0:1787739062.401553:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968378 committed transno 4294968383 00000020:00080000:3.0:1787739062.401559:0:6523:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07306241f00 refcount 2 committed 0 result 0 batchid 374 00000020:00080000:3.0:1787739062.401563:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2a00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07306241480 00000020:00080000:3.0:1787739062.401566:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].371 00000020:00080000:3.0:1787739062.401569:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2d00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304cf5400 00000020:00080000:3.0:1787739062.401572:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].372 00000004:00080000:3.0:1787739062.401597:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968380 committed transno 4294968383 00000004:00080000:3.0:1787739062.401608:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968381 committed transno 4294968383 00000004:00080000:3.0:1787739062.401618:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968383 committed transno 4294968383 00000100:00100000:3.0:1787739062.401636:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d84780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665673472:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739062.401654:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad20b lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad212 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.401665:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad20b lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad212 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739062.401681:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739062.401709:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ad219 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.401716:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ad219 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.401725:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ad219 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739062.401743:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d0b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665673600:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739062.401775:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.401798:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404a80 going to sleep for 11 seconds 00000020:00080000:3.0:1787739062.401809:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 374 committed batchid 372 00000020:00080000:3.0:1787739062.401860:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 372 00000100:00100000:3.0:1787739062.401872:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665673600 00000100:00100000:3.0:1787739062.401877:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665673600 00000100:00100000:3.0:1787739062.401901:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3145 00000100:00100000:3.0:1787739062.401907:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6523:x1874579665673600:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739062.401916:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739062.401926:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ad220 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad219 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.401941:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ad220 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad219 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.401951:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ad220 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad219 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.401957:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xda636067 (ffffa074039f2800) 00010000:00010000:3.0:1787739062.401959:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739062.401961:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab600, rc 0) 00000100:00100000:3.0:1787739062.401990:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665673600:12345-0@lo:101:mdt00_000.0 Request processed in 83us (214us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.401996:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3145 00000100:00100000:3.0:1787739062.402036:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d0b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665673600:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739062.402050:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ad219 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad220 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.402058:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ad219 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad220 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739062.402069:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:3.0:1787739062.402099:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d0b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665673728:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739062.402125:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.402135:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665673728 00000100:00100000:3.0:1787739062.402140:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739062.402140:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665673728 00000100:00100000:2.0:1787739062.402163:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3146 00000100:00100000:2.0:1787739062.402169:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665673728:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739062.402235:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665673728:12345-0@lo:1000:mdt00_000.0 Request processed in 66us (109us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.402242:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3146 00000100:00100000:3.0:1787739062.403325:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d0b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665673728:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739062.403380:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d0b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665673856:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739062.403436:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.403461:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739062.403477:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665673856 00000100:00100000:2.0:1787739062.403482:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665673856 00000100:00100000:2.0:1787739062.403521:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3147 00000100:00100000:2.0:1787739062.403528:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665673856:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739062.403630:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665673856:12345-0@lo:1000:mdt00_000.0 Request processed in 102us (193us total) trans 0 rc 1/1 00000100:00100000:2.0:1787739062.403637:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3147 00000100:00100000:3.0:1787739062.404064:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d0b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665673856:0@lo:1000:mdt00_000.0 00010000:00010000:3.0:1787739062.404120:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ad227 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.404130:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ad227 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.404139:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ad227 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739062.404151:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d0b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665673984:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739062.404179:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.404207:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739062.404814:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665673984 00000100:00100000:2.0:1787739062.404821:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665673984 00000100:00100000:2.0:1787739062.404856:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3148 00000100:00100000:2.0:1787739062.404864:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6523:x1874579665673984:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739062.404877:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739062.404894:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad22e lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad227 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.404917:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad22e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad227 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.404929:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad22e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad227 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.404943:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1518:0x0].0x0 (ffffa07303c78600) 00010000:00010000:2.0:1787739062.404946:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739062.404949:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d200, rc 0) 00000100:00100000:3.0:1787739062.405018:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d0b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665673984:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739062.405042:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ad227 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad22e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.405052:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ad227 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad22e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.405076:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad235 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.405085:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad235 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.405093:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad235 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.405109:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ad23c lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.405118:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad1f6 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fe7 expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.405126:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ad23c lrc: 4/0,1 mode: --/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.405147:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad1f6 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fe7 expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.405160:0:6523:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad1f6 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fe7 expref: 17 pid: 13793 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739062.405170:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d0b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665674112:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739062.409839:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6523:x1874579665673984:12345-0@lo:101:mdt00_000.0 Request processed in 4966us (5649us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.409853:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3148 00000100:00100000:3.0:1787739062.416906:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.419488:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d0b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665674112:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739062.419514:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ad23c lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739062.422493:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739062.422555:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652477824 00000100:00100000:3.0:1787739062.422574:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652477824 00010000:00010000:3.0:1787739062.422620:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad1f6 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fe7 expref: 18 pid: 13793 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739062.422708:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1510 00000100:00100000:0.0:1787739062.422722:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074184c1050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2409:x1874579652477824:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739062.422742:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739062.422751:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad1f6 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fe7 expref: 18 pid: 13793 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.422766:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad1f6 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fe7 expref: 18 pid: 13793 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.422799:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad1f6 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fe7 expref: 18 pid: 13793 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.422813:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc6:0x0].0x0 (ffffa07305eaec00) 00010000:00010000:0.0:1787739062.422818:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ad23c lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.422828:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ad23c lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.422835:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ad23c lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.422870:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739062.422925:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ad23c lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739062.422927:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074184c1050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652477824:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 209us (443us total) trans 0 rc 0/0 00010000:00010000:3.0:1787739062.422934:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ad23c lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739062.422935:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1510 00010000:00010000:3.0:1787739062.422975:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad243 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.422983:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad243 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.423001:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad243 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739062.423030:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d0b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665674240:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739062.423309:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.423347:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739062.423473:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665674240 00000100:00100000:2.0:1787739062.423478:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665674240 00000100:00100000:2.0:1787739062.423517:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3149 00000100:00100000:2.0:1787739062.423525:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6523:x1874579665674240:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739062.423540:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739062.423554:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad24a lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad243 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.423574:0:6529:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad204 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fee expref: 6 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.423593:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad24a lrc: 4/0,0 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad243 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.423616:0:6529:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad204 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fee expref: 6 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.423628:0:6529:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad204 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fee expref: 6 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739062.423644:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d84b40 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665674368:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739062.423697:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ed80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739062.444868:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739062.444939:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652477952 00000100:00100000:1.0:1787739062.444948:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652477952 00010000:00010000:1.0:1787739062.445003:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad204 lrc: 5/0,0 mode: PR/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fee expref: 7 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.445054:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1511 00000100:00100000:1.0:1787739062.445065:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2407:x1874579652477952:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739062.445080:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739062.445087:0:6510:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad204 lrc: 5/0,0 mode: PR/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fee expref: 7 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.445100:0:6510:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad204 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fee expref: 7 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.445121:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc6:0x0].0x0 (ffffa07303c78300) 00010000:00010000:1.0:1787739062.445128:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad24a lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad243 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.445144:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad24a lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad243 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.445154:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad24a lrc: 4/0,0 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad243 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.445184:0:6510:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad24a lrc: 4/0,0 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad243 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.445197:0:6510:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562da40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6510:1874579665674496:0@lo:105: 00000100:00100000:1.0:1787739062.445240:0:6510:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.445272:0:6510:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.445287:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665674496 00000100:00100000:1.0:1787739062.445290:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665674496 00000100:00100000:1.0:1787739062.445324:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1155 00000100:00100000:1.0:1787739062.445331:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6510:x1874579665674496:12345-0@lo:105: 00010000:00010000:1.0:1787739062.445341:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad243 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad24a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.445350:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad243 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad24a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.445359:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad243 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad24a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.445366:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad243 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad24a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.445374:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad243 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad24a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.445381:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad243 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ad24a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.445394:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069aba00) 00000100:00100000:1.0:1787739062.445457:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6510:x1874579665674496:12345-0@lo:105: Request processed in 126us (215us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739062.445476:0:6510:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562da40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6510:1874579665674496:0@lo:105: 00010000:00010000:0.0:1787739062.445492:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739062.445529:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2407:x1874579652477952:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 466us (671us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.445533:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1155 00000100:00100000:0.0:1787739062.445536:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1511 00010000:00010000:3.0:1787739062.446257:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad204 lrc: 0/0,0 mode: --/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094fee expref: 6 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739062.446280:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d84b40 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665674368:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739062.446299:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad24a lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad243 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.446308:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc6:0x0].0x0 (ffffa07303c78300) 00010000:00010000:3.0:1787739062.446310:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739062.446313:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d800, rc 0) 00000100:00100000:3.0:1787739062.446380:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:6523:x1874579665674240:12345-0@lo:101:mdt00_000.0 Request processed in 22856us (23070us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.446387:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3149 00000100:00100000:0.0:1787739062.446790:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d0b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665674240:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739062.446808:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad243 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad24a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739062.447525:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a30f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665674624:0@lo:403:mdt00_000.0 00000100:00100000:0.0:1787739062.447580:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739062.447625:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303937180 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.447664:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665674624 00000100:00100000:1.0:1787739062.447671:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665674624 00000100:00100000:1.0:1787739062.447721:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3150 00000100:00100000:1.0:1787739062.447730:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:6523:x1874579665674624:12345-0@lo:403:mdt00_000.0 00000100:00100000:1.0:1787739062.453199:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:6523:x1874579665674624:12345-0@lo:403:mdt00_000.0 Request processed in 5464us (5612us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.453210:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3150 00000100:00100000:0.0:1787739062.453274:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a30f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665674624:0@lo:403:mdt00_000.0 00080000:02000000:0.0:1787739062.454572:0:6523:0:(osd_handler.c:2218:osd_trans_start()) *** cfs_fail_loc=19a, val=0*** 00000004:00080000:0.0:1787739062.454587:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07303937180 with version 0 00000020:00080000:0.0:1787739062.454594:0:6523:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07303937cc0 refcount 3 committed 0 result -5 batchid 376 00000020:00080000:0.0:1787739062.454598:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2400 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result -5 sub_th ffffa07303937180 00000020:00080000:0.0:1787739062.454600:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2000 obd lustre-MDT0000-osd committed 1 started 0 stopped 1 result 0 sub_th ffffa073040e8e00 00000020:00080000:2.0:1787739062.454653:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 372 00010000:00010000:0.0:1787739062.454828:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ad23c lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.454835:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ad23c lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.454840:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ad23c lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.454844:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739062.454847:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ad23c lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.454852:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ad23c lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.454866:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc6:0x0].0x0 (ffffa07305eaec00) 00010000:00010000:0.0:1787739062.454868:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739062.454872:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ad23c lrc: 2/0,0 mode: --/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.454875:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ad23c lrc: 0/0,0 mode: --/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.454885:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad243 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad24a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.454890:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad243 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad24a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.454894:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad243 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad24a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.454943:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739062.454959:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xc6:0x0] 00010000:00010000:0.0:1787739062.454967:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad235 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.454973:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad235 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.454979:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad235 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.454984:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739062.454997:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad235 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.455004:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad235 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.455018:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1518:0x0].0x0 (ffffa07305eae100) 00010000:00010000:0.0:1787739062.455029:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739062.455032:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad235 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.455038:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad235 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.455055:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ad227 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad22e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.455063:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ad227 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad22e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.455071:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ad227 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad22e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.455209:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1787739062.455215:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad243 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad24a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.455219:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aba00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739062.455222:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad243 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad24a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739062.455228:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad243 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad24a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.455255:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad243 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad24a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.455259:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739062.455285:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad243 lrc: 1/0,0 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad24a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.455292:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad243 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad24a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.455306:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:1.0:1787739062.455307:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d84b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665674752:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739062.455309:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ad227 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad22e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.455319:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739062.455323:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ad227 lrc: 5/0,0 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad22e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739062.455328:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ad227 lrc: 5/0,0 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad22e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.455341:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ad227 lrc: 4/0,0 mode: --/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad22e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.455346:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739062.455352:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ad227 lrc: 3/0,0 mode: --/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad22e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.455358:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739062.455359:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739062.455362:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739062.455363:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787739062.455401:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:0.0:1787739062.455406:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa600/0x7ee4453e4a1ad227 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad22e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739062.455443:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:0.0:1787739062.455454:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665674880:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739062.455494:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787739062.455495:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739062.455602:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad20b lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad212 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.455611:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad20b lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad212 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.455618:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad20b lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad212 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.455643:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739062.455648:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739062.455655:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739062.455659:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad20b lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad212 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739062.455726:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665674752 00000100:00100000:1.0:1787739062.455726:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665674880 00000100:00100000:2.0:1787739062.455734:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665674752 00000100:00100000:1.0:1787739062.455734:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665674880 00000100:00100000:2.0:1787739062.455849:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1310 00000100:00100000:2.0:1787739062.455860:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+10:3653:x1874579665674880:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739062.455874:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739062.455892:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad22e lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad227 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.455892:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1311 00000100:00100000:1.0:1787739062.455901:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+10:3652:x1874579665674752:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739062.455906:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1518:0x0].0x0 (ffffa07303c78600) 00010000:00010000:2.0:1787739062.455915:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739062.455919:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739062.455931:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad24a lrc: 0/0,0 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad243 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.455952:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc6:0x0].0x0 (ffffa07303c78300) 00010000:00010000:1.0:1787739062.455959:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739062.455994:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+8:3652:x1874579665674752:12345-0@lo:103:ldlm_bl.0 Request processed in 94us (629us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.456002:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1311 00000100:00100000:1.0:1787739062.456054:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d84b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665674752:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739062.456094:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633cc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739062.456100:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad20b lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad212 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739062.456109:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad20b lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad212 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739062.456114:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+7:3653:x1874579665674880:12345-0@lo:103:ldlm_bl.0 Request processed in 257us (620us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.456121:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1310 00010000:00010000:0.0:1787739062.456128:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad20b lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad212 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.456134:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1787739062.456151:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665674880:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739062.456156:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad20b lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad212 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.456163:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad20b lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad212 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.456176:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739062.456178:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739062.456180:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739062.456187:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787739062.456196:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665675008:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739062.456221:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.456242:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665675008 00000100:00100000:2.0:1787739062.456246:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665675008 00000100:00100000:2.0:1787739062.456271:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1312 00000100:00100000:2.0:1787739062.456277:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+7:3651:x1874579665675008:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739062.456284:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739062.456295:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad212 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad20b expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.456304:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074039f2300) 00010000:00010000:2.0:1787739062.456309:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739062.456334:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+6:3651:x1874579665675008:12345-0@lo:103:ldlm_bl.0 Request processed in 57us (112us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.456339:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1312 00000100:00100000:0.0:1787739062.456365:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665675008:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739062.456734:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ad219 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad220 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.456743:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ad219 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad220 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.456760:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ad219 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad220 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.456782:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787739062.457266:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17594:x1874579652477696:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 58136us (60739us total) trans 0 rc -5/-5 00000100:00100000:0.0:1787739062.457275:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23454 00010000:00010000:2.0:1787739062.457302:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739062.457308:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ad219 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad220 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.457315:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739062.457319:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ad219 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad220 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739062.457325:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ad219 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad220 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.457348:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ad219 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad220 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.457354:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739062.457369:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ad219 lrc: 1/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad220 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.457376:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ad219 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad220 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.457390:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739062.457392:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739062.457393:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739062.457403:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665675136:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739062.457448:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.458255:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665675136 00000100:00100000:1.0:1787739062.458264:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665675136 00000100:00100000:1.0:1787739062.458342:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1313 00000100:00100000:1.0:1787739062.458352:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+6:3651:x1874579665675136:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739062.458362:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739062.458377:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ad220 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad219 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.458391:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xda636067 (ffffa074039f2800) 00010000:00010000:1.0:1787739062.458399:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739062.458453:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665675136:12345-0@lo:103:ldlm_bl.0 Request processed in 104us (1004us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.458460:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1313 00000100:00100000:1.0:1787739062.458730:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665675136:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739062.487933:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739062.488134:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652478080 00000100:00100000:0.0:1787739062.488144:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652478080 00000100:00100000:0.0:1787739062.488241:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23455 00000100:00100000:0.0:1787739062.488252:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17596:x1874579652478080:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739062.488271:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739062.488291:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad251 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094ff5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.488351:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad258 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.488365:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad258 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.488375:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad258 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.488389:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad25f lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.488397:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad25f lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.488404:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad25f lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.488463:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad258 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.488470:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad258 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.488476:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad258 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.488483:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad258 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.488489:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609bc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739062.488493:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad258 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.488500:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad258 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.488513:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78100) 00010000:00010000:0.0:1787739062.488515:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739062.488520:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad258 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.488526:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad258 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.488539:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad25f lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.488545:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad25f lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.488552:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad25f lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.488557:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739062.488560:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad25f lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.488566:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad25f lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.488573:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xed2251f3 (ffffa07307064c00) 00010000:00010000:0.0:1787739062.488575:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739062.488578:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad25f lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.488584:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad25f lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.488618:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad251 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ff5 expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.488627:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78100) 00010000:00010000:0.0:1787739062.488630:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739062.488633:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad251 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ff5 expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.488642:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609aa00, rc 0) 00000100:00100000:0.0:1787739062.488701:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17596:x1874579652478080:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 452us (778us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739062.488709:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23455 00000100:00100000:1.0:1787739062.493512:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739062.493687:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652478208 00000100:00100000:3.0:1787739062.493696:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652478208 00000100:00100000:3.0:1787739062.493755:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23456 00000100:00100000:3.0:1787739062.493766:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17596:x1874579652478208:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739062.493785:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739062.493820:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad266 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e094ffc expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.493880:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad26d lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.493894:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad26d lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.493917:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad26d lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739062.493963:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x5:0x0]: returning size 4096 00000004:00200000:3.0:1787739062.493968:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739062.494006:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad266 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ffc expref: 8 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.494032:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad26d lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ffc expref: 7 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.494041:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074039f2100) 00010000:00010000:3.0:1787739062.494044:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739062.494048:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aac00, rc 0) 00000100:00100000:3.0:1787739062.494110:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17596:x1874579652478208:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 346us (606us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.494118:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23456 00000100:00100000:0.0:1787739062.503640:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739062.503682:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652478464 00000100:00100000:3.0:1787739062.503692:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652478464 00000100:00100000:3.0:1787739062.503747:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23457 00000100:00100000:3.0:1787739062.503758:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17596:x1874579652478464:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:3.0:1787739062.503867:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d3480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665675264:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739062.503932:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.503980:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739062.505409:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665675264 00000100:00100000:0.0:1787739062.505516:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665675264 00000100:00100000:0.0:1787739062.505585:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3151 00000100:00100000:0.0:1787739062.505595:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665675264:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739062.505697:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665675264:12345-0@lo:1000:mdt00_000.0 Request processed in 104us (1764us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739062.505704:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3151 00000100:00100000:2.0:1787739062.506130:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d3480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665675264:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739062.506198:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d87480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665675392:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739062.506242:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.506352:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739062.507041:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665675392 00000100:00100000:0.0:1787739062.507048:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665675392 00000100:00100000:0.0:1787739062.507128:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3152 00000100:00100000:0.0:1787739062.507137:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d852c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665675392:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739062.507238:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d852c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665675392:12345-0@lo:1000:mdt00_000.0 Request processed in 101us (994us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739062.507250:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3152 00000100:00100000:2.0:1787739062.507316:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d87480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665675392:0@lo:1000:mdt00_000.0 00010000:00010000:2.0:1787739062.507376:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d600/0x7ee4453e4a1ad274 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.507387:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d600/0x7ee4453e4a1ad274 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.507397:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d600/0x7ee4453e4a1ad274 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739062.507484:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d87480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665675520:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1787739062.507517:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.507570:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739062.507603:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665675520 00000100:00100000:0.0:1787739062.507607:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665675520 00000100:00100000:0.0:1787739062.507738:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3153 00000100:00100000:0.0:1787739062.507747:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665675520:12345-0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739062.507758:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739062.507773:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad27b lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad274 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.507797:0:6528:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1acf02 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f4d expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.507808:0:6528:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad27b lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad274 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.507832:0:6528:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1acf02 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f4d expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.507857:0:6528:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1acf02 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f4d expref: 15 pid: 7944 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739062.507869:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432c3c0 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6528:1874579665675648:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739062.507931:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303937d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.520722:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432c3c0 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6528:1874579665675648:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739062.520757:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad27b lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad274 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.520770:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78100) 00010000:00010000:1.0:1787739062.520774:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739062.520778:0:6528:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad27b lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad274 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.520787:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609aa00, rc 0) 00000100:00100000:1.0:1787739062.520854:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665675520:12345-0@lo:101:mdt00_000.0 Request processed in 13107us (13335us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.520863:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3153 00000100:00100000:3.0:1787739062.520972:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739062.520993:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d87480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665675520:0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739062.521015:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d600/0x7ee4453e4a1ad274 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad27b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.521041:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d600/0x7ee4453e4a1ad274 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad27b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.521203:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652478592 00000100:00100000:1.0:1787739062.521210:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652478592 00010000:00010000:1.0:1787739062.521237:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1acf02 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f4d expref: 16 pid: 7944 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.521272:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1512 00000100:00100000:1.0:1787739062.521282:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:2406:x1874579652478592:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739062.521298:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739062.521304:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1acf02 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f4d expref: 16 pid: 7944 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.521316:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1acf02 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f4d expref: 16 pid: 7944 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.521340:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1acf02 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094f4d expref: 16 pid: 7944 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.521360:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78100) 00010000:00010000:1.0:1787739062.521365:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad27b lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad274 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.521374:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad27b lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad274 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.521384:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad27b lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad274 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.521560:0:6508:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad27b lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad274 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.521585:0:6508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562cf00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665675776:0@lo:105: 00000100:00100000:1.0:1787739062.521627:0:6508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.521668:0:6508:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d000 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.521861:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665675776 00000100:00100000:3.0:1787739062.521867:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665675776 00000100:00100000:3.0:1787739062.521914:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1156 00000100:00100000:3.0:1787739062.521923:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562fc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665675776:12345-0@lo:105: 00010000:00010000:3.0:1787739062.521935:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d600/0x7ee4453e4a1ad274 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad27b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.521945:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d600/0x7ee4453e4a1ad274 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad27b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.521954:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d600/0x7ee4453e4a1ad274 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad27b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.521961:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d600/0x7ee4453e4a1ad274 lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad27b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.521967:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d600/0x7ee4453e4a1ad274 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad27b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.521974:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d600/0x7ee4453e4a1ad274 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ad27b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.522002:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633d600) 00000100:00100000:3.0:1787739062.522062:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562fc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665675776:12345-0@lo:105: Request processed in 141us (434us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.522070:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1156 00000100:00100000:1.0:1787739062.522719:0:6508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562cf00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665675776:0@lo:105: 00010000:00010000:1.0:1787739062.522739:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739062.522797:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2406:x1874579652478592:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 1514us (1825us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.522806:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1512 00010000:00010000:1.0:1787739062.522841:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d600/0x7ee4453e4a1ad274 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad27b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.522850:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d600/0x7ee4453e4a1ad274 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad27b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739062.522874:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739062.522990:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad282 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.522998:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad282 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.523008:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad282 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.523034:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562cf00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665675904:0@lo:101:mdt00_000.0 00000100:00100000:1.0:1787739062.523071:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.523104:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739062.523702:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665675904 00000100:00100000:2.0:1787739062.523709:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665675904 00000100:00100000:2.0:1787739062.523743:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3154 00000100:00100000:2.0:1787739062.523752:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665675904:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739062.523770:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739062.523788:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ad289 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad282 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.523992:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ad289 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad282 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.524008:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ad289 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad282 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.524016:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xed2251f3 (ffffa07303c78b00) 00010000:00010000:2.0:1787739062.524029:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739062.524033:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633da00, rc 0) 00000100:00100000:2.0:1787739062.524091:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665675904:12345-0@lo:101:mdt00_000.0 Request processed in 340us (1020us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.524099:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3154 00000100:00100000:1.0:1787739062.524523:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562cf00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665675904:0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739062.524541:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad282 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad289 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.524551:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad282 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad289 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739062.524570:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:1.0:1787739062.524682:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562cf00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665676032:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739062.524718:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.524750:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d540 going to sleep for 11 seconds 00000100:00100000:2.0:1787739062.524922:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665676032 00000100:00100000:2.0:1787739062.524928:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665676032 00000100:00100000:2.0:1787739062.524970:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3155 00000100:00100000:2.0:1787739062.524978:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665676032:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739062.525061:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665676032:12345-0@lo:1000:mdt00_000.0 Request processed in 84us (343us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.525068:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3155 00000100:00100000:1.0:1787739062.525559:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562cf00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665676032:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739062.525698:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562cf00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665676160:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739062.525734:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.525763:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d540 going to sleep for 11 seconds 00000100:00100000:2.0:1787739062.525809:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665676160 00000100:00100000:2.0:1787739062.525814:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665676160 00000100:00100000:2.0:1787739062.525884:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3156 00000100:00100000:2.0:1787739062.525893:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665676160:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739062.525995:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665676160:12345-0@lo:1000:mdt00_000.0 Request processed in 104us (260us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739062.526002:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3156 00000100:00100000:1.0:1787739062.526061:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562cf00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665676160:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739062.526132:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562cf00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665676288:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739062.526164:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.529124:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665676288 00000100:00100000:3.0:1787739062.529141:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665676288 00000100:00100000:1.0:1787739062.529165:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.529217:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3157 00000100:00100000:3.0:1787739062.529227:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665676288:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739062.529557:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665676288:12345-0@lo:1000:mdt00_000.0 Request processed in 138us (3199us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739062.529570:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3157 00000100:00100000:1.0:1787739062.531095:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562cf00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665676288:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739062.531194:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665676416:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739062.531255:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.531281:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.532734:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665676416 00000100:00100000:3.0:1787739062.532747:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665676416 00000100:00100000:3.0:1787739062.532804:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3158 00000100:00100000:3.0:1787739062.532816:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665676416:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739062.532944:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665676416:12345-0@lo:1000:mdt00_000.0 Request processed in 130us (1688us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.532951:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3158 00000100:00100000:1.0:1787739062.533998:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665676416:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739062.534102:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665676544:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739062.534148:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.534181:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.534458:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665676544 00000100:00100000:3.0:1787739062.534464:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665676544 00000100:00100000:3.0:1787739062.534566:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3159 00000100:00100000:3.0:1787739062.534585:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665676544:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739062.534690:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665676544:12345-0@lo:1000:mdt00_000.0 Request processed in 115us (539us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.534699:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3159 00000100:00100000:2.0:1787739062.534773:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665676544:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739062.534821:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d87480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665676672:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739062.534853:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.537363:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.537407:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665676672 00000100:00100000:1.0:1787739062.537427:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665676672 00000100:00100000:1.0:1787739062.537491:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3160 00000100:00100000:1.0:1787739062.537501:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665676672:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739062.537673:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665676672:12345-0@lo:1000:mdt00_000.0 Request processed in 172us (2816us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.537681:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3160 00000100:00100000:2.0:1787739062.537978:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d87480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665676672:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739062.538055:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb4b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665676800:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739062.538093:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.538197:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.538229:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665676800 00000100:00100000:1.0:1787739062.538234:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665676800 00000100:00100000:1.0:1787739062.538268:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3161 00000100:00100000:1.0:1787739062.538276:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665676800:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739062.538401:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665676800:12345-0@lo:1000:mdt00_000.0 Request processed in 125us (308us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.538407:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3161 00000100:00100000:2.0:1787739062.538657:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb4b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665676800:0@lo:1000:mdt00_000.0 00000040:00080000:2.0:1787739062.539840:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].360, 2168 off546568 00000040:00080000:2.0:1787739062.540003:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].361, 2368 off598416 00000004:00080000:2.0:1787739062.540009:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730610e0c0 with version 688 00000004:00080000:1.0:1787739062.540107:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 688 rpc_version 688 00000020:00100000:1.0:1787739062.540143:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739062.540152:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073060d0780 x1874579665677056/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739062.540173:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665677056:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739062.540234:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.540296:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d900 going to sleep for 11 seconds 00000100:00100000:2.0:1787739062.540339:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665677056 00000100:00100000:2.0:1787739062.540345:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665677056 00000100:00100000:2.0:1787739062.540457:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3162 00000100:00100000:2.0:1787739062.540468:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665677056:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739062.541368:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:7467:x1874579665677056:12345-0@lo:1000:osp_up0-1.0 Request processed in 901us (1131us total) trans 4294983397 rc 0/0 00000100:00100000:2.0:1787739062.541377:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3162 00000100:00100000:1.0:1787739062.543082:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665677056:0@lo:1000:osp_up0-1.0 00010000:00010000:2.0:1787739062.543322:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ad290 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x17a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.543337:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ad290 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.543347:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ad290 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.543367:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad297 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xc4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.543374:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad297 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.543380:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad297 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.544923:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad29e lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.544939:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad29e lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.544961:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad29e lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739062.545535:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb4b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665677184:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1787739062.545727:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.545825:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ed80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.545879:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665677184 00000100:00100000:1.0:1787739062.545886:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665677184 00000100:00100000:1.0:1787739062.546051:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3163 00000100:00100000:1.0:1787739062.546062:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665677184:12345-0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739062.546084:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739062.546108:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad2a5 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xc4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad29e expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.546135:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad2a5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad29e expref: 12 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.546148:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad2a5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad29e expref: 12 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.546156:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc4:0x0].0x0 (ffffa07433235e00) 00010000:00010000:1.0:1787739062.546159:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739062.546162:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c200, rc 0) 00000100:00100000:1.0:1787739062.546221:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:6523:x1874579665677184:12345-0@lo:101:mdt00_000.0 Request processed in 161us (494us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.546229:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3163 00000100:00100000:2.0:1787739062.546298:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb4b40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665677184:0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739062.546315:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad29e lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad2a5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.546325:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad29e lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad2a5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739062.546351:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000acadc293 reply state 0000000099fccba6 transno 4294968384 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad297 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.546363:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad297 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.546370:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc4:0x0].0x0 (ffffa07303c78700) 00010000:00010000:2.0:1787739062.546373:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739062.546375:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787739062.546381:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad29e lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad2a5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.546389:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad29e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad2a5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739062.546402:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xc4:0x0] 00000004:00010000:2.0:1787739062.546408:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000acadc293 reply state 0000000099fccba6 transno 4294968384 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ad290 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.546435:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ad290 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x17a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.546441:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17a:0x0].0x0 (ffffa07303c78900) 00010000:00010000:2.0:1787739062.546443:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739062.546444:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:2.0:1787739062.546450:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17a:0x0]: returning size 4096 00000004:00080000:2.0:1787739062.546459:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739062.546468:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d600/0x7ee4453e4a1ad274 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad27b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.546477:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d600/0x7ee4453e4a1ad274 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad27b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.546484:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d600/0x7ee4453e4a1ad274 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad27b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.546504:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739062.546508:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739062.546515:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739062.546520:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d600/0x7ee4453e4a1ad274 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad27b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.546528:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739062.546533:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d600/0x7ee4453e4a1ad274 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad27b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739062.546542:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d600/0x7ee4453e4a1ad274 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad27b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.546569:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d600/0x7ee4453e4a1ad274 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad27b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.546588:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739062.546610:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d600/0x7ee4453e4a1ad274 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad27b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.546618:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d600/0x7ee4453e4a1ad274 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad27b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.546633:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739062.546635:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739062.546636:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739062.546643:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787739062.546654:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665677312:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739062.546686:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.546715:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665677312 00000100:00100000:2.0:1787739062.546720:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665677312 00000100:00100000:2.0:1787739062.547168:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1314 00000100:00100000:2.0:1787739062.547178:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074184c3450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+12:3653:x1874579665677312:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739062.547188:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739062.547383:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad27b lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad274 expref: 12 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.547402:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07303c78100) 00010000:00010000:2.0:1787739062.547410:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739062.547472:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074184c3450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+11:3653:x1874579665677312:12345-0@lo:103:ldlm_bl.0 Request processed in 296us (784us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.547478:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1314 00000100:00100000:2.0:1787739062.547564:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665677312:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739062.547657:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad282 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad289 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.547667:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad282 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad289 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739062.547862:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17596:x1874579652478464:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 44104us (44225us total) trans 4294968384 rc 0/0 00000100:00100000:2.0:1787739062.547870:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23457 00000100:00100000:3.0:1787739062.564635:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739062.565102:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652478848 00000100:00100000:1.0:1787739062.565114:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652478848 00000100:00100000:1.0:1787739062.565171:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23458 00000100:00100000:1.0:1787739062.565183:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17605:x1874579652478848:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:1.0:1787739062.565353:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17605:x1874579652478848:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 172us (735us total) trans 0 rc -17/-17 00000100:00100000:1.0:1787739062.565361:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23458 00000100:00100000:2.0:1787739062.590108:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739062.591846:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652478976 00000100:00100000:1.0:1787739062.591860:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652478976 00000100:00100000:1.0:1787739062.591921:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23459 00000100:00100000:1.0:1787739062.591931:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17606:x1874579652478976:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739062.591949:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739062.591972:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad2ac lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095003 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.592043:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad2b3 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.592058:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad2b3 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.592069:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad2b3 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.592085:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad2ba lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.592097:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ad289 lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad282 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.592105:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad2ba lrc: 4/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.592136:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ad289 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ad282 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.592149:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d2d00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665677440:0@lo:104: 00000100:00100000:1.0:1787739062.592210:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.592250:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dc00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.592290:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665677440 00000100:00100000:1.0:1787739062.592294:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665677440 00000100:00100000:1.0:1787739062.592321:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1157 00000100:00100000:1.0:1787739062.592328:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665677440:12345-0@lo:104: 00010000:00010000:1.0:1787739062.592340:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad282 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad289 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.592402:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787739062.592408:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665677440:12345-0@lo:104: Request processed in 80us (197us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.592439:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1157 00010000:00010000:1.0:1787739062.592462:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739062.592468:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad282 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad289 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.592477:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739062.592482:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad282 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ad289 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739062.592491:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad282 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ad289 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.592515:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad282 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad289 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.592522:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739062.592540:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad282 lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad289 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.592548:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad282 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad289 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.592564:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739062.592567:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739062.592568:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739062.592575:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787739062.592585:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665677568:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739062.592614:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.592641:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665677568 00000100:00100000:1.0:1787739062.592645:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665677568 00010000:00010000:1.0:1787739062.592660:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ad289 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ad282 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.592685:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1513 00000100:00100000:1.0:1787739062.592692:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d2940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+11:3652:x1874579665677568:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739062.592701:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787739062.592844:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d2d00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665677440:0@lo:104: 00010000:00010000:1.0:1787739062.592863:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad2ba lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787739062.596813:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983397 is committed 00000020:00080000:3.0:1787739062.596826:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 374 batchid updated 00000001:00080000:3.0:1787739062.596843:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983394 is committed 00000020:00080000:3.0:1787739062.596848:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa0730623dcc0 refcount 2 committed 0 result 0 batchid 375 00000020:00080000:3.0:1787739062.596853:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2000 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730623d600 00000020:00080000:3.0:1787739062.596857:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].372 00000020:00080000:3.0:1787739062.596859:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2700 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07303c9b800 00000020:00080000:3.0:1787739062.596862:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].373 00000020:00080000:3.0:1787739062.596879:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 373 batchid updated 00010000:00010000:3.0:1787739062.596907:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad1a9 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.596919:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad1a9 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.596927:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad1a9 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.596934:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739062.596939:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad1a9 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.596947:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad1a9 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.596956:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739062.596963:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1518:0x0].0x0 (ffffa07305eae100) 00010000:00010000:3.0:1787739062.596967:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739062.596971:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad1a9 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.596978:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad1a9 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.596994:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad1b0 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.597001:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad1b0 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.597006:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad1b0 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.597012:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ce00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739062.597016:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad1b0 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.597033:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad1b0 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.597041:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc6:0x0].0x0 (ffffa07305eaec00) 00010000:00010000:3.0:1787739062.597044:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739062.597047:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad1b0 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.597053:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad1b0 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1787739062.597074:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 376 committed batchid 374 00000004:00080000:3.0:1787739062.597168:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa074104049c0 with version 711 00000020:00080000:3.0:1787739062.597178:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 373 cancel update log [0x240000401:0x3:0x0].370: rc = 0 00000020:00080000:3.0:1787739062.597219:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 373 cancel update log [0x200000400:0x3:0x0].371: rc = 0 00000004:00080000:3.0:1787739062.597256:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07410404f00 with version 712 00000020:00080000:3.0:1787739062.597260:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 374 cancel update log [0x240000401:0x3:0x0].371: rc = 0 00000020:00080000:3.0:1787739062.597281:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 374 cancel update log [0x200000400:0x3:0x0].372: rc = 0 00000020:00080000:3.0:1787739062.597286:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 374 00000004:00080000:3.0:1787739062.597295:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 711 rpc_version 711 00000020:00100000:3.0:1787739062.597324:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739062.597335:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074080d1680 x1874579665677824/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739062.597357:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d1680 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665677824:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739062.597442:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.597467:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665677824 00000100:00100000:2.0:1787739062.597474:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665677824 00000100:00100000:3.0:1787739062.597484:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404a80 going to sleep for 11 seconds 00010000:00010000:3.0:1787739062.597793:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ad289 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ad282 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.597806:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xed2251f3 (ffffa07303c78b00) 00010000:00010000:3.0:1787739062.597810:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad2ba lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.597818:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad2ba lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.597825:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad2ba lrc: 4/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.599489:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739062.599512:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3164 00000100:00100000:2.0:1787739062.599528:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:7740:x1874579665677824:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739062.599727:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d2940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+6:3652:x1874579665677568:12345-0@lo:103:ldlm_bl.0 Request processed in 7035us (7111us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.599737:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1513 00000100:00100000:2.0:1787739062.599774:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665677824:12345-0@lo:1000:osp_up1-0.0 Request processed in 250us (2330us total) trans 4294968385 rc 0/0 00000100:00100000:2.0:1787739062.599783:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3164 00000100:00100000:3.0:1787739062.599802:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d1680 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665677824:0@lo:1000:osp_up1-0.0 00000004:00080000:3.0:1787739062.599816:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 712 rpc_version 712 00000020:00100000:3.0:1787739062.599836:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739062.599843:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074080d2d00 x1874579665678080/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739062.599869:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d2d00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665678080:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739062.599902:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.599925:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.599939:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665678080 00000100:00100000:3.0:1787739062.599944:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665678080 00000100:00100000:3.0:1787739062.600187:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3165 00000100:00100000:3.0:1787739062.600195:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665678080:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739062.600293:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665678080:12345-0@lo:1000:osp_up1-0.0 Request processed in 98us (389us total) trans 4294968386 rc 0/0 00000100:00100000:3.0:1787739062.600304:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3165 00000100:00100000:1.0:1787739062.600311:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665677568:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739062.600331:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d2d00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665678080:0@lo:1000:osp_up1-0.0 00010000:00010000:3.0:1787739062.600347:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad2ba lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.600357:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad2ba lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739062.600711:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d12c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665678208:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739062.600746:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.600765:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404300 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.600864:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665678208 00000100:00100000:3.0:1787739062.600869:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665678208 00000100:00100000:3.0:1787739062.601310:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3166 00000100:00100000:3.0:1787739062.601321:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665678208:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739062.601385:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6524:x1874579665678208:12345-0@lo:1000:mdt00_001.0 Request processed in 67us (639us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.605801:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3166 00000100:00100000:3.0:1787739062.605877:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d12c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665678208:0@lo:1000:mdt00_001.0 00010000:00010000:3.0:1787739062.605942:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad2c1 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x17a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.606097:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad2c1 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.606115:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad2c1 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739062.606139:0:6524:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad2c1 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.606155:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad2b3 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.606162:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad2b3 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.606168:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad2b3 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.606175:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad2b3 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.606181:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739062.606186:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad2b3 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.606193:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad2b3 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.606204:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235700) 00010000:00010000:3.0:1787739062.606207:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739062.606211:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad2b3 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.606218:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad2b3 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.606232:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad2ba lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.606238:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad2ba lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.606244:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad2ba lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.606249:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ca00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739062.606252:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad2ba lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.606257:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad2ba lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.606264:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xed2251f3 (ffffa07303c78b00) 00010000:00010000:3.0:1787739062.606267:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739062.606270:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad2ba lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.606275:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad2ba lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.606310:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad2ac lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095003 expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.606329:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad2c1 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x17a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095003 expref: 14 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.606340:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17a:0x0].0x0 (ffffa074039f2c00) 00010000:00010000:3.0:1787739062.606342:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739062.606344:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa200, rc 0) 00000100:00100000:3.0:1787739062.606491:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17606:x1874579652478976:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 14561us (16391us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.606499:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23459 00000100:00100000:2.0:1787739062.617735:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739062.617901:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652479104 00000100:00100000:3.0:1787739062.617910:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652479104 00000100:00100000:3.0:1787739062.617962:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23460 00000100:00100000:3.0:1787739062.617971:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17606:x1874579652479104:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739062.617992:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739062.618010:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad2c8 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x17a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09500a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.618084:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad2cf lrc: 3/1,0 mode: --/PR res: [0x240000402:0x17a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.618097:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad2cf lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.618110:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad2cf lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739062.618201:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17a:0x0]: returning size 4096 00000004:00200000:3.0:1787739062.618206:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739062.618240:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad2c8 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x17a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09500a expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.618272:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad2cf lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x17a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09500a expref: 10 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.618281:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17a:0x0].0x0 (ffffa07303c78900) 00010000:00010000:3.0:1787739062.618283:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739062.618286:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa400, rc 0) 00000100:00100000:3.0:1787739062.618339:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17606:x1874579652479104:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 369us (612us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.618346:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23460 00000100:00100000:0.0:1787739062.627061:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739062.627694:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652479232 00000100:00100000:1.0:1787739062.627706:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652479232 00000100:00100000:1.0:1787739062.627772:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23461 00000100:00100000:1.0:1787739062.627783:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17606:x1874579652479232:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739062.627805:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739062.627829:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ad2d6 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xc4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095011 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.627895:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad2dd lrc: 3/1,0 mode: --/PR res: [0x240000400:0xc4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.627910:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad2dd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.627922:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad2dd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739062.627965:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xc4:0x0]: returning size 4096 00000004:00200000:1.0:1787739062.627970:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739062.628003:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ad2d6 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095011 expref: 12 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.628028:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad2dd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095011 expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.628038:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc4:0x0].0x0 (ffffa07303c78700) 00010000:00010000:1.0:1787739062.628041:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739062.628044:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1cc00, rc 0) 00000100:00100000:1.0:1787739062.628111:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17606:x1874579652479232:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 330us (1057us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.628119:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23461 00000100:00100000:3.0:1787739062.633244:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739062.633322:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652479360 00000100:00100000:1.0:1787739062.633331:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652479360 00000100:00100000:1.0:1787739062.633391:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23462 00000100:00100000:1.0:1787739062.633402:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17606:x1874579652479360:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739062.633448:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739062.633471:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad2e4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xc4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095018 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.633553:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad2eb lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.633568:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad2a5 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad29e expref: 5 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.633578:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad2eb lrc: 4/1,0 mode: --/PR res: [0x200000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.633611:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad2a5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ad29e expref: 5 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.633627:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665678336:0@lo:104: 00000100:00100000:1.0:1787739062.633678:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.633725:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dc00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739062.633884:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665678336 00000100:00100000:2.0:1787739062.633889:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665678336 00000100:00100000:2.0:1787739062.633927:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1158 00000100:00100000:2.0:1787739062.633936:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d2d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665678336:12345-0@lo:104: 00010000:00010000:2.0:1787739062.633949:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad29e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad2a5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.634010:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739062.634016:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d2d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6524:x1874579665678336:12345-0@lo:104: Request processed in 82us (338us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.634033:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1158 00010000:00010000:2.0:1787739062.634055:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739062.634062:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad29e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad2a5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.634070:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739062.634076:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad29e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xc4:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ad2a5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739062.634085:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad29e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xc4:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ad2a5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.634111:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad29e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xc4:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad2a5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.634117:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739062.634134:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad29e lrc: 1/0,0 mode: --/PW res: [0x200000401:0xc4:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad2a5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.634141:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad29e lrc: 0/0,0 mode: --/PW res: [0x200000401:0xc4:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad2a5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.634160:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739062.634162:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739062.634164:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739062.634219:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665678336:0@lo:104: 00010000:00010000:3.0:1787739062.634241:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad2eb lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.634302:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a30f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665678464:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739062.634333:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.634408:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665678464 00000100:00100000:2.0:1787739062.634521:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665678464 00010000:00010000:2.0:1787739062.634541:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad2a5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ad29e expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739062.634599:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1514 00000100:00100000:2.0:1787739062.634607:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d1a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+6:3650:x1874579665678464:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739062.634618:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00080000:1.0:1787739062.641154:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 376 batchid updated 00000020:00080000:3.0:1787739062.641221:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 376 00000004:00080000:3.0:1787739062.641280:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07303c6ec00 with version 713 00000020:00080000:3.0:1787739062.641297:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 375 cancel update log [0x240000401:0x3:0x0].372: rc = 0 00000020:00080000:3.0:1787739062.641483:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 375 cancel update log [0x200000400:0x3:0x0].373: rc = 0 00000020:00080000:3.0:1787739062.641491:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 376 00010000:00010000:2.0:1787739062.641597:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad2a5 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ad29e expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.641619:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc4:0x0].0x0 (ffffa07433235e00) 00010000:00010000:2.0:1787739062.641624:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad2eb lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.641634:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad2eb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.641642:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad2eb lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.641679:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739062.641822:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d1a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+5:3650:x1874579665678464:12345-0@lo:103:ldlm_bl.0 Request processed in 7214us (7485us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.641832:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1514 00010000:00010000:3.0:1787739062.642956:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad2eb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.642969:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad2eb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739062.643038:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xc4:0x0]: returning size 4096 00000004:00200000:3.0:1787739062.643044:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739062.643094:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad2e4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095018 expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.643113:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad2eb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095018 expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.643122:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc4:0x0].0x0 (ffffa07433235e00) 00010000:00010000:3.0:1787739062.643125:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739062.643128:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1de00, rc 0) 00000004:00080000:1.0:1787739062.643626:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 713 rpc_version 713 00000020:00100000:1.0:1787739062.643656:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739062.643674:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073060d30c0 x1874579665678720/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739062.643698:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d30c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665678720:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739062.643872:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.643904:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d900 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.643928:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665678720 00000100:00100000:1.0:1787739062.643934:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665678720 00000100:00100000:1.0:1787739062.643977:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3167 00000100:00100000:1.0:1787739062.643985:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665678720:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739062.644135:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665678720:12345-0@lo:1000:osp_up1-0.0 Request processed in 149us (262us total) trans 4294968387 rc 0/0 00000100:00100000:1.0:1787739062.644143:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3167 00000100:00100000:1.0:1787739062.644188:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a30f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665678464:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739062.644227:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d30c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665678720:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739062.644303:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17606:x1874579652479360:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 10903us (11068us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.644311:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23462 00000100:00100000:3.0:1787739062.650223:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739062.650271:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739062.651841:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652479488 00000100:00100000:2.0:1787739062.651854:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652479488 00000100:00100000:2.0:1787739062.651927:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1315 00000100:00100000:2.0:1787739062.651938:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2409:x1874579652479488:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:2.0:1787739062.651956:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739062.651982:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad2c1 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x17a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095003 expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.652001:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17a:0x0].0x0 (ffffa074039f2c00) 00010000:00010000:2.0:1787739062.652011:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739062.652075:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652479488:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 139us (1861us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.652083:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1315 00000100:00100000:1.0:1787739062.653576:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652479744 00000100:00100000:1.0:1787739062.653589:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652479744 00000100:00100000:1.0:1787739062.653687:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23463 00000100:00100000:1.0:1787739062.653697:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f76580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17606:x1874579652479744:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:1.0:1787739062.653760:0:13793:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739062.653784:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad2cf lrc: 0/0,0 mode: --/PR res: [0x240000402:0x17a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09500a expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.653803:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17a:0x0].0x0 (ffffa07303c78900) 00010000:00010000:1.0:1787739062.653808:0:13793:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739062.653854:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad2f2 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.653863:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad2f2 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.653873:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad2f2 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.653892:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d3840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665678848:0@lo:101:mdt00_004.0 00000100:00100000:1.0:1787739062.653948:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.653984:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.654242:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665678848 00000100:00100000:3.0:1787739062.654248:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665678848 00000100:00100000:3.0:1787739062.654287:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3168 00000100:00100000:3.0:1787739062.654294:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+5:13793:x1874579665678848:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739062.654304:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739062.654320:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad2f9 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad2f2 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.654342:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad2f9 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad2f2 expref: 6 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.654355:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad2f9 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad2f2 expref: 6 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.654380:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074039f2b00) 00010000:00010000:3.0:1787739062.654383:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739062.654386:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aae00, rc 0) 00000100:00100000:3.0:1787739062.654469:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:13793:x1874579665678848:12345-0@lo:101:mdt00_004.0 Request processed in 174us (520us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.654476:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3168 00000100:00100000:3.0:1787739062.654509:0:13793:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983397 gen 2 00000004:00080000:3.0:1787739062.654515:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983393 committed transno 4294983397 00000020:00080000:3.0:1787739062.654521:0:13793:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07303c6e240 refcount 2 committed 0 result 0 batchid 366 00000020:00080000:3.0:1787739062.654525:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2900 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07303c6ecc0 00000020:00080000:3.0:1787739062.654529:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].360 00000020:00080000:3.0:1787739062.654531:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2b00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304cf5400 00000020:00080000:3.0:1787739062.654534:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].359 00000004:00080000:3.0:1787739062.654560:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983395 committed transno 4294983397 00000004:00080000:3.0:1787739062.654570:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983396 committed transno 4294983397 00000004:00080000:3.0:1787739062.654583:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983397 committed transno 4294983397 00000100:00100000:3.0:1787739062.654606:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d3840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665678848:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739062.654622:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad2f2 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad2f9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.654647:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad2f2 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad2f9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739062.654663:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739062.654686:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad300 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.654694:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad300 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.654713:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad300 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739062.654726:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f75a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665678976:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787739062.654755:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.654775:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6ecc0 going to sleep for 11 seconds 00000020:00080000:3.0:1787739062.654789:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 366 committed batchid 365 00000020:00080000:3.0:1787739062.654851:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 365 00000100:00100000:1.0:1787739062.655303:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665678976 00000100:00100000:1.0:1787739062.655310:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665678976 00000100:00100000:1.0:1787739062.655351:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3169 00000100:00100000:1.0:1787739062.655359:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f77840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:13793:x1874579665678976:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739062.655379:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739062.655395:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad307 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad300 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.655432:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad307 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad300 expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.655445:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad307 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad300 expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.655452:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xed2251f3 (ffffa07433235200) 00010000:00010000:1.0:1787739062.655454:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739062.655457:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c200, rc 0) 00000100:00100000:1.0:1787739062.655492:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f77840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665678976:12345-0@lo:101:mdt00_004.0 Request processed in 135us (737us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.655498:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3169 00000100:00100000:3.0:1787739062.656005:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f75a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665678976:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739062.656034:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad300 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad307 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.656044:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad300 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad307 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739062.656061:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:3.0:1787739062.656095:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f75a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665679104:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739062.656124:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.656149:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.656165:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665679104 00000100:00100000:3.0:1787739062.656169:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665679104 00000100:00100000:3.0:1787739062.656191:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3170 00000100:00100000:3.0:1787739062.656196:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665679104:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739062.656248:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665679104:12345-0@lo:1000:mdt00_004.0 Request processed in 51us (123us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.656255:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3170 00000100:00100000:3.0:1787739062.656276:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f75a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665679104:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739062.656306:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f75a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665679232:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739062.656324:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.656337:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.656346:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665679232 00000100:00100000:3.0:1787739062.656349:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665679232 00000100:00100000:3.0:1787739062.656362:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3171 00000100:00100000:3.0:1787739062.656367:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665679232:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739062.656444:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665679232:12345-0@lo:1000:mdt00_004.0 Request processed in 76us (118us total) trans 0 rc 1/1 00000100:00100000:3.0:1787739062.656450:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3171 00000100:00100000:3.0:1787739062.656468:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f75a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665679232:0@lo:1000:mdt00_004.0 00010000:00010000:3.0:1787739062.656504:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ad30e lrc: 3/0,1 mode: --/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.656512:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ad30e lrc: 3/0,1 mode: --/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.656520:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ad30e lrc: 3/0,1 mode: --/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739062.656529:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f75a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665679360:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787739062.656563:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.656578:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6ecc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.656587:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665679360 00000100:00100000:3.0:1787739062.656590:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665679360 00000100:00100000:3.0:1787739062.656604:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3172 00000100:00100000:3.0:1787739062.656608:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306390b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665679360:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739062.656615:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739062.656626:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad315 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad30e expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.656641:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad315 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad30e expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.656653:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad315 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad30e expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.656660:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17a:0x0].0x0 (ffffa074039f2e00) 00010000:00010000:3.0:1787739062.656663:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739062.656679:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739062.656943:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306390b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665679360:12345-0@lo:101:mdt00_004.0 Request processed in 333us (378us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.656949:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3172 00000100:00100000:3.0:1787739062.656974:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f75a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665679360:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739062.656984:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ad30e lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad315 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.656992:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ad30e lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad315 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.657013:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ad31c lrc: 3/0,1 mode: --/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.657073:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ad31c lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.657081:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ad31c lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.657099:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad323 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.657110:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad2dd lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095011 expref: 10 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.657118:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad323 lrc: 4/0,1 mode: --/EX res: [0x240000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.657140:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad2dd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095011 expref: 10 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.657153:0:13793:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad2dd lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095011 expref: 10 pid: 6525 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739062.657163:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306393840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665679488:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739062.657256:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6ecc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.662008:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739062.662103:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306393840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665679488:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739062.662132:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad323 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.662222:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652479872 00000100:00100000:1.0:1787739062.662229:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652479872 00010000:00010000:1.0:1787739062.662265:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad2dd lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095011 expref: 11 pid: 6525 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.662302:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1515 00000100:00100000:1.0:1787739062.662312:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306390000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2408:x1874579652479872:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739062.662330:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739062.662338:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad2dd lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095011 expref: 11 pid: 6525 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.662356:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad2dd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095011 expref: 11 pid: 6525 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.662377:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad2dd lrc: 0/0,0 mode: --/PR res: [0x240000400:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095011 expref: 11 pid: 6525 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.662391:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc4:0x0].0x0 (ffffa07303c78700) 00010000:00010000:1.0:1787739062.662396:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad323 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.662404:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad323 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.662452:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad323 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.662474:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739062.662528:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306390000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652479872:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 216us (528us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.662537:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1515 00010000:00010000:3.0:1787739062.662584:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad323 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.662593:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad323 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.662707:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ad32a lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.662715:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ad32a lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.662724:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ad32a lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739062.662743:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073063903c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665679616:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787739062.662787:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.662831:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.662964:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665679616 00000100:00100000:1.0:1787739062.662969:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665679616 00000100:00100000:1.0:1787739062.663004:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3173 00000100:00100000:1.0:1787739062.663012:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306390f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665679616:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739062.663042:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739062.663058:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad331 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad32a expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.663086:0:12410:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad2eb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095018 expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.663095:0:12410:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad331 lrc: 4/0,0 mode: --/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad32a expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.663120:0:12410:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad2eb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095018 expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.663132:0:12410:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad2eb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095018 expref: 13 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.663143:0:12410:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d3c00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:12410:1874579665679744:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739062.663199:0:12410:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.667327:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739062.667555:0:12410:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d3c00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:12410:1874579665679744:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739062.667587:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad331 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad32a expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.667600:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc4:0x0].0x0 (ffffa07433235e00) 00010000:00010000:1.0:1787739062.667603:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739062.667607:0:12410:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad331 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad32a expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.667616:0:12410:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad331 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad32a expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.667628:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1dc00, rc 0) 00000100:00100000:1.0:1787739062.667692:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306390f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665679616:12345-0@lo:101:mdt00_004.0 Request processed in 4681us (4904us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.667804:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652480000 00000100:00100000:2.0:1787739062.667810:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652480000 00010000:00010000:2.0:1787739062.667844:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad2eb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095018 expref: 14 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739062.667878:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1516 00000100:00100000:2.0:1787739062.667887:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306391a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652480000:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739062.667901:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739062.667907:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad2eb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095018 expref: 14 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.667919:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad2eb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095018 expref: 14 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.667946:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad2eb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc4:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095018 expref: 14 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.667960:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc4:0x0].0x0 (ffffa07433235e00) 00010000:00010000:2.0:1787739062.667965:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad331 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad32a expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.667975:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad331 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad32a expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.667983:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad331 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad32a expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.668012:0:6508:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad331 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad32a expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.668317:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3173 00000100:00100000:3.0:1787739062.668856:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073063903c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665679616:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739062.668876:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ad32a lrc: 4/0,1 mode: --/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad331 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.668886:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ad32a lrc: 4/0,1 mode: --/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad331 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739062.669001:0:6508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a30f00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665679872:0@lo:105: 00000100:00100000:2.0:1787739062.669932:0:6508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.669974:0:6508:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ef00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.670230:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665679872 00000100:00100000:1.0:1787739062.670239:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665679872 00000100:00100000:1.0:1787739062.670288:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1159 00000100:00100000:1.0:1787739062.670296:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665679872:12345-0@lo:105: 00010000:00010000:1.0:1787739062.670310:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ad32a lrc: 5/0,1 mode: --/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad331 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.670320:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ad32a lrc: 5/0,1 mode: --/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad331 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.670330:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ad32a lrc: 5/0,1 mode: --/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad331 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.670338:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ad32a lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad331 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.670346:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ad32a lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad331 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.670355:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ad32a lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ad331 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.670376:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069ab600) 00000100:00100000:1.0:1787739062.670439:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665679872:12345-0@lo:105: Request processed in 144us (510us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.670446:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1159 00010000:00010000:3.0:1787739062.670603:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ad32a lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad331 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.670613:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ad32a lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad331 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739062.671204:0:6508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a30f00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665679872:0@lo:105: 00010000:00010000:2.0:1787739062.671220:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739062.671278:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306391a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:2409:x1874579652480000:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 3389us (3959us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.671288:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1516 00000100:00100000:2.0:1787739062.673605:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d87480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665680000:0@lo:403:mdt00_004.0 00000100:00100000:2.0:1787739062.673697:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.674398:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ef00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.674897:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665680000 00000100:00100000:3.0:1787739062.674907:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665680000 00000100:00100000:3.0:1787739062.674972:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3174 00000100:00100000:3.0:1787739062.674980:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665680000:12345-0@lo:403:mdt00_004.0 00000100:00100000:3.0:1787739062.683406:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665680000:12345-0@lo:403:mdt00_004.0 Request processed in 8418us (9700us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.683444:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3174 00000100:00100000:3.0:1787739062.683511:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d87480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665680000:0@lo:403:mdt00_004.0 00000040:00080000:3.0:1787739062.684473:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].361, 664 off547232 00000040:00080000:3.0:1787739062.684651:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].362, 744 off599160 00000004:00080000:3.0:1787739062.684657:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07303c6e9c0 with version 689 00000004:00080000:3.0:1787739062.684682:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 689 rpc_version 689 00000020:00100000:3.0:1787739062.684710:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739062.684719:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073063903c0 x1874579665680256/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1808/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739062.684740:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073063903c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665680256:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739062.684789:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.691255:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e900 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.691369:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665680256 00000100:00100000:1.0:1787739062.691379:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665680256 00000100:00100000:1.0:1787739062.691460:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3175 00000100:00100000:1.0:1787739062.691471:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073063921c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665680256:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739062.693233:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073063921c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665680256:12345-0@lo:1000:osp_up0-1.0 Request processed in 1759us (8437us total) trans 4294983398 rc 0/0 00000100:00100000:1.0:1787739062.693246:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3175 00000100:00100000:3.0:1787739062.695128:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073063903c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665680256:0@lo:1000:osp_up0-1.0 00000004:00200000:2.0:1787739062.695239:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17a:0x0]: returning size 4096 00010000:00010000:2.0:1787739062.695262:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad323 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.695272:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad323 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.695280:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad323 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.695286:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739062.695290:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad323 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.695297:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad323 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.695311:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc4:0x0].0x0 (ffffa07303c78700) 00010000:00010000:2.0:1787739062.695313:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739062.695317:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad323 lrc: 2/0,0 mode: --/EX res: [0x240000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.695323:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad323 lrc: 0/0,0 mode: --/EX res: [0x240000400:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.695335:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ad32a lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad331 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.695342:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ad32a lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad331 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.695356:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ad32a lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad331 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.695381:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739062.695389:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xc4:0x0] 00010000:00010000:2.0:1787739062.695396:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ad31c lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.695402:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ad31c lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.695409:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ad31c lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.695501:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739062.695505:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ad31c lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.695510:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ad31c lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.695519:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17a:0x0].0x0 (ffffa07303c78900) 00010000:00010000:2.0:1787739062.695521:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739062.695524:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ad31c lrc: 2/0,0 mode: --/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.695530:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ad31c lrc: 0/0,0 mode: --/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.695540:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ad30e lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad315 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.695547:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ad30e lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad315 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.695553:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ad30e lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad315 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.695575:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:2.0:1787739062.695584:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739062.695588:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad2f2 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad2f9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.695594:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad2f2 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad2f9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.695600:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad2f2 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad2f9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.695609:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:2.0:1787739062.695612:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739062.695616:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad300 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad307 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.695622:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad300 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad307 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.695684:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1787739062.695691:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ad32a lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad331 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.695701:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739062.695706:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ad32a lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad331 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739062.695714:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ad32a lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad331 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.695849:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787739062.695855:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ad30e lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad315 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.695863:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739062.695867:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ad30e lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad315 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739062.695875:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ad30e lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad315 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.695901:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ad30e lrc: 2/0,0 mode: --/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad315 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.695907:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739062.695925:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ad30e lrc: 1/0,0 mode: --/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad315 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.695932:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab000/0x7ee4453e4a1ad30e lrc: 0/0,0 mode: --/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad315 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.695946:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739062.695950:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad2f2 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad2f9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.695955:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739062.695959:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad2f2 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad2f9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739062.695966:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad2f2 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad2f9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.695978:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad2f2 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad2f9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.695983:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739062.695994:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad2f2 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad2f9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.696006:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad2f2 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad2f9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.696016:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ad32a lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad331 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.696026:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739062.696030:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739062.696033:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739062.696037:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739062.696046:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ad32a lrc: 1/0,0 mode: --/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad331 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739062.696049:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306393c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665680384:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739062.696061:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ad32a lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad331 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.696075:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739062.696076:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739062.696078:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739062.696085:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739062.696129:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f76580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17606:x1874579652479744:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 42431us (45854us total) trans 4294968388 rc 0/0 00000100:00100000:2.0:1787739062.696141:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23463 00000100:00100000:2.0:1787739062.696175:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665680640:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739062.696818:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306392940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665680512:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739062.699203:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.699281:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665680384 00000100:00100000:3.0:1787739062.699288:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665680384 00000100:00100000:3.0:1787739062.699332:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1316 00000100:00100000:3.0:1787739062.699340:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306392580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+10:3651:x1874579665680384:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739062.699355:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739062.699378:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad315 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x17a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad30e expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.699396:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17a:0x0].0x0 (ffffa074039f2e00) 00010000:00010000:3.0:1787739062.699405:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739062.699453:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306392580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+9:3651:x1874579665680384:12345-0@lo:103:ldlm_bl.0 Request processed in 113us (258us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.699460:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1316 00000100:00100000:3.0:1787739062.699497:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306393c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665680384:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739062.703582:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.706006:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665680640 00000100:00100000:3.0:1787739062.706029:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665680640 00000100:00100000:3.0:1787739062.706117:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1317 00000100:00100000:3.0:1787739062.706127:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074184c0c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3653:x1874579665680640:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739062.706146:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739062.706172:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad331 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad32a expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.706192:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc4:0x0].0x0 (ffffa07433235e00) 00010000:00010000:3.0:1787739062.706201:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739062.706279:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074184c0c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+8:3653:x1874579665680640:12345-0@lo:103:ldlm_bl.0 Request processed in 153us (2704us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.706287:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1317 00000100:00100000:2.0:1787739062.706434:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665680640:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739062.707992:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.708268:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665680512 00000100:00100000:2.0:1787739062.708277:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665680512 00000100:00100000:2.0:1787739062.708333:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1318 00000100:00100000:2.0:1787739062.708343:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d3c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3652:x1874579665680512:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739062.708357:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739062.708378:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad2f9 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad2f2 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.708394:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074039f2b00) 00010000:00010000:2.0:1787739062.708403:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739062.708601:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d3c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1874579665680512:12345-0@lo:103:ldlm_bl.0 Request processed in 259us (614us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.708609:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1318 00000100:00100000:1.0:1787739062.708813:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306392940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665680512:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739062.734002:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739062.734134:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652480128 00000100:00100000:1.0:1787739062.734143:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652480128 00000100:00100000:1.0:1787739062.734201:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23464 00000100:00100000:1.0:1787739062.734211:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306390000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17607:x1874579652480128:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739062.734236:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739062.734257:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad338 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09501f expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.734324:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad33f lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.734342:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad33f lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.734354:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad33f lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.734368:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad346 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.734374:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad346 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.734380:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad346 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.734460:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad33f lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.734468:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad33f lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.734474:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad33f lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.734481:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad33f lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.734487:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ce00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739062.734492:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad33f lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.734500:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad33f lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.734512:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074039f2100) 00010000:00010000:1.0:1787739062.734515:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739062.734520:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad33f lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.734526:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad33f lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.734539:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad346 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.734545:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad346 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.734551:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad346 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.734556:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739062.734559:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad346 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.734565:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad346 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.734573:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x63ac40ed (ffffa07433235b00) 00010000:00010000:1.0:1787739062.734575:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739062.734578:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad346 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.734584:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad346 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.734621:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad338 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09501f expref: 11 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.734632:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074039f2100) 00010000:00010000:1.0:1787739062.734635:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739062.734638:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad338 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09501f expref: 11 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.734647:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1cc00, rc 0) 00000100:00100000:1.0:1787739062.734980:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306390000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17607:x1874579652480128:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 768us (984us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739062.734988:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23464 00000100:00100000:0.0:1787739062.749875:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739062.749937:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652480256 00000100:00100000:3.0:1787739062.749948:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652480256 00000100:00100000:3.0:1787739062.750008:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23465 00000100:00100000:3.0:1787739062.750031:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17607:x1874579652480256:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739062.750057:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739062.750077:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad34d lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095026 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.750131:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad354 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.750144:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad354 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.750156:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad354 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739062.750201:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:3.0:1787739062.750206:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739062.750539:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad34d lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095026 expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.750562:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad354 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095026 expref: 14 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.750572:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074039f2e00) 00010000:00010000:3.0:1787739062.750575:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739062.750578:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab000, rc 0) 00000100:00100000:3.0:1787739062.750885:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17607:x1874579652480256:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 866us (1019us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.750893:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23465 00000100:00100000:3.0:1787739062.767528:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739062.767635:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652480512 00000100:00100000:0.0:1787739062.767655:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652480512 00000100:00100000:0.0:1787739062.767716:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23466 00000100:00100000:0.0:1787739062.767726:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306391a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17607:x1874579652480512:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:0.0:1787739062.767838:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383c780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665680768:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739062.767887:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739062.767947:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303937c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.768095:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665680768 00000100:00100000:1.0:1787739062.768100:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665680768 00000100:00100000:1.0:1787739062.768142:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3176 00000100:00100000:1.0:1787739062.768149:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665680768:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739062.768244:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665680768:12345-0@lo:1000:mdt00_004.0 Request processed in 94us (354us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.768251:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3176 00000100:00100000:3.0:1787739062.768342:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383c780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665680768:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739062.768388:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665680896:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739062.768441:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.768474:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6ecc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.768658:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665680896 00000100:00100000:1.0:1787739062.768663:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665680896 00000100:00100000:1.0:1787739062.768695:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3177 00000100:00100000:1.0:1787739062.768702:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306393c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665680896:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739062.768776:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306393c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665680896:12345-0@lo:1000:mdt00_004.0 Request processed in 74us (334us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739062.768783:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3177 00000100:00100000:2.0:1787739062.769303:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665680896:0@lo:1000:mdt00_004.0 00010000:00010000:2.0:1787739062.769383:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ad35b lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.769393:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ad35b lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.769404:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ad35b lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739062.769458:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41fc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665681024:0@lo:101:mdt00_004.0 00000100:00100000:2.0:1787739062.769507:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.769547:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ecc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.769560:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665681024 00000100:00100000:1.0:1787739062.769566:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665681024 00000100:00100000:1.0:1787739062.769640:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3178 00000100:00100000:1.0:1787739062.769648:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:13793:x1874579665681024:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739062.769659:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739062.769674:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad362 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad35b expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.769698:0:6528:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad26d lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ffc expref: 9 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.769708:0:6528:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad362 lrc: 4/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad35b expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.769730:0:6528:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad26d lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ffc expref: 9 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.769742:0:6528:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad26d lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ffc expref: 9 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.769751:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d2940 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001_UUID:6528:1874579665681152:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739062.769809:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.773878:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739062.773953:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652480640 00000100:00100000:0.0:1787739062.773963:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652480640 00010000:00010000:0.0:1787739062.774000:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad26d lrc: 5/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ffc expref: 10 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739062.774056:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1517 00000100:00100000:0.0:1787739062.774066:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306391e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652480640:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739062.774081:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739062.774088:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad26d lrc: 5/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ffc expref: 10 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.774101:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad26d lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ffc expref: 10 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.774123:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074039f2100) 00010000:00010000:0.0:1787739062.774130:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad362 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad35b expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.774141:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad362 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad35b expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.774149:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad362 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad35b expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.774180:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad362 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad35b expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739062.774197:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383c780 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665681280:0@lo:105: 00000100:00100000:0.0:1787739062.774240:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739062.774273:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303937600 going to sleep for 11 seconds 00000100:00100000:0.0:1787739062.774289:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665681280 00000100:00100000:0.0:1787739062.774292:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665681280 00000100:00100000:0.0:1787739062.774316:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1160 00000100:00100000:0.0:1787739062.774322:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665681280:12345-0@lo:105: 00010000:00010000:0.0:1787739062.774333:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ad35b lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad362 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.774342:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ad35b lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad362 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.774350:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ad35b lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad362 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.774357:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ad35b lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad362 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.774364:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ad35b lrc: 5/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad362 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.774372:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ad35b lrc: 5/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ad362 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.774383:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633de00) 00000100:00100000:0.0:1787739062.774436:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665681280:12345-0@lo:105: Request processed in 89us (170us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739062.774443:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1160 00000100:00100000:0.0:1787739062.774480:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383c780 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665681280:0@lo:105: 00010000:00010000:0.0:1787739062.774494:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739062.774526:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306391e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652480640:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 462us (657us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739062.774533:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1517 00010000:00010000:1.0:1787739062.775037:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad26d lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e094ffc expref: 9 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.775056:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d2940 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001_UUID:6528:1874579665681152:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739062.775075:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad362 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad35b expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.775084:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074039f2100) 00010000:00010000:1.0:1787739062.775087:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739062.775089:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1cc00, rc 0) 00000100:00100000:1.0:1787739062.775139:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:13793:x1874579665681024:12345-0@lo:101:mdt00_004.0 Request processed in 5492us (5633us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.775146:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3178 00000100:00100000:3.0:1787739062.775256:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41fc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665681024:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739062.775272:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ad35b lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad362 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739062.775292:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739062.775327:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad369 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.775333:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad369 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.775341:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad369 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739062.775356:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073063930c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665681408:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787739062.775384:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.775408:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739062.779797:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665681408 00000100:00100000:0.0:1787739062.779811:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665681408 00000100:00100000:0.0:1787739062.779878:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3179 00000100:00100000:0.0:1787739062.779888:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306390780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:13793:x1874579665681408:12345-0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739062.779907:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739062.779929:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad370 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad369 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.779958:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad370 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad369 expref: 10 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.779972:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad370 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad369 expref: 10 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.779981:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x63ac40ed (ffffa07307064c00) 00010000:00010000:0.0:1787739062.779985:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739062.779988:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a600, rc 0) 00000100:00100000:0.0:1787739062.780081:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306390780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665681408:12345-0@lo:101:mdt00_004.0 Request processed in 195us (4696us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739062.780092:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3179 00000100:00100000:1.0:1787739062.780263:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073063930c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665681408:0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739062.780282:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad369 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad370 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.780293:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad369 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad370 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739062.780315:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:1.0:1787739062.780361:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d2940 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665681536:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739062.780400:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.780443:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dc00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.780493:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665681536 00000100:00100000:3.0:1787739062.780498:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665681536 00000100:00100000:3.0:1787739062.780524:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3180 00000100:00100000:3.0:1787739062.780530:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665681536:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739062.780598:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665681536:12345-0@lo:1000:mdt00_004.0 Request processed in 68us (198us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.780605:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3180 00000100:00100000:1.0:1787739062.780659:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d2940 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665681536:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739062.780704:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d2940 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665681664:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739062.780725:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.780745:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dc00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.780838:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665681664 00000100:00100000:3.0:1787739062.780842:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665681664 00000100:00100000:3.0:1787739062.780870:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3181 00000100:00100000:3.0:1787739062.780877:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665681664:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739062.780945:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665681664:12345-0@lo:1000:mdt00_004.0 Request processed in 69us (218us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739062.780952:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3181 00000100:00100000:3.0:1787739062.780974:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d2940 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665681664:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739062.781050:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306393480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665681792:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739062.781074:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.781092:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404180 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.781104:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665681792 00000100:00100000:3.0:1787739062.781108:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665681792 00000100:00100000:3.0:1787739062.781123:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3182 00000100:00100000:3.0:1787739062.781129:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306392d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665681792:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739062.781175:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306392d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665681792:12345-0@lo:1000:mdt00_004.0 Request processed in 45us (99us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739062.781180:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3182 00000100:00100000:3.0:1787739062.781202:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306393480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665681792:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739062.781242:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306393480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665681920:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739062.781263:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.781278:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404180 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.781288:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665681920 00000100:00100000:3.0:1787739062.781291:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665681920 00000100:00100000:3.0:1787739062.781308:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3183 00000100:00100000:3.0:1787739062.781313:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306391680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665681920:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739062.781356:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306391680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665681920:12345-0@lo:1000:mdt00_004.0 Request processed in 43us (91us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.781362:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3183 00000100:00100000:3.0:1787739062.781382:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306393480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665681920:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739062.781441:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306393480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665682048:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739062.781462:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.781476:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404180 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.781486:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665682048 00000100:00100000:3.0:1787739062.781489:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665682048 00000100:00100000:3.0:1787739062.781503:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3184 00000100:00100000:3.0:1787739062.781509:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073063912c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665682048:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739062.781576:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073063912c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665682048:12345-0@lo:1000:mdt00_004.0 Request processed in 66us (113us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.781583:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3184 00000100:00100000:3.0:1787739062.781604:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306393480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665682048:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739062.781635:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306393480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665682176:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739062.781656:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.781681:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404180 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.781692:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665682176 00000100:00100000:3.0:1787739062.781695:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665682176 00000100:00100000:3.0:1787739062.781711:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3185 00000100:00100000:3.0:1787739062.781716:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073063930c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665682176:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739062.781756:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073063930c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665682176:12345-0@lo:1000:mdt00_004.0 Request processed in 40us (99us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.781762:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3185 00000100:00100000:3.0:1787739062.781783:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306393480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665682176:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739062.781810:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306393480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665682304:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739062.781831:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.781846:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404180 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.781856:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665682304 00000100:00100000:3.0:1787739062.781859:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665682304 00000100:00100000:3.0:1787739062.781871:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3186 00000100:00100000:3.0:1787739062.781876:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306392580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665682304:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739062.781915:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306392580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:13793:x1874579665682304:12345-0@lo:1000:mdt00_004.0 Request processed in 38us (83us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.781921:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3186 00000100:00100000:3.0:1787739062.781943:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306393480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665682304:0@lo:1000:mdt00_004.0 00000040:00080000:3.0:1787739062.782692:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].374, 2168 off572720 00000040:00080000:3.0:1787739062.782766:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].373, 2368 off619744 00000004:00080000:3.0:1787739062.782771:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07410404300 with version 714 00000004:00080000:1.0:1787739062.783806:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 714 rpc_version 714 00000020:00100000:1.0:1787739062.783851:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739062.783860:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073060d0780 x1874579665682560/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739062.783887:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665682560:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739062.783947:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.783975:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d000 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.784000:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665682560 00000100:00100000:1.0:1787739062.784005:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665682560 00000100:00100000:1.0:1787739062.784056:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3187 00000100:00100000:1.0:1787739062.784065:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665682560:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739062.789219:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+11:7740:x1874579665682560:12345-0@lo:1000:osp_up1-0.0 Request processed in 5145us (5261us total) trans 4294968389 rc 0/0 00000100:00100000:1.0:1787739062.789235:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3187 00000100:00100000:1.0:1787739062.790197:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665682560:0@lo:1000:osp_up1-0.0 00010000:00010000:3.0:1787739062.790355:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad377 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x151a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.790369:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad377 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x151a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.790379:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad377 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x151a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.790399:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad37e lrc: 3/0,1 mode: --/PW res: [0x200000400:0xc7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.790406:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad37e lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.790437:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad37e lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.790469:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ad385 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.790476:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ad385 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.790485:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ad385 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739062.790502:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306393480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665682688:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787739062.790548:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.790587:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4db40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.790652:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665682688 00000100:00100000:1.0:1787739062.790659:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665682688 00000100:00100000:1.0:1787739062.790716:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3188 00000100:00100000:1.0:1787739062.790724:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306392940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+11:13793:x1874579665682688:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739062.790740:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739062.790754:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad38c lrc: 2/0,0 mode: --/PW res: [0x240000401:0xc7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad385 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.790773:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad38c lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad385 expref: 12 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.790785:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad38c lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad385 expref: 12 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.790794:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc7:0x0].0x0 (ffffa07433235b00) 00010000:00010000:1.0:1787739062.790797:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739062.790800:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d000, rc 0) 00000100:00100000:1.0:1787739062.790834:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306392940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+12:13793:x1874579665682688:12345-0@lo:101:mdt00_004.0 Request processed in 112us (287us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.790840:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3188 00000100:00100000:1.0:1787739062.790868:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306393480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665682688:0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739062.790880:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ad385 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad38c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.790890:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ad385 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad38c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739062.790908:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000bdb41dc4 reply state 00000000ab32f098 transno 4294983399 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad37e lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.790920:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad37e lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.790926:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc7:0x0].0x0 (ffffa074039f2b00) 00010000:00010000:1.0:1787739062.790929:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739062.790931:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787739062.790937:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ad385 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad38c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.790944:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ad385 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad38c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739062.790953:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xc7:0x0] 00000004:00010000:1.0:1787739062.790958:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000bdb41dc4 reply state 00000000ab32f098 transno 4294983399 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad377 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x151a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.790970:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad377 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x151a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.790975:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x151a:0x0].0x0 (ffffa074039f2700) 00010000:00010000:1.0:1787739062.790977:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739062.790978:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:1.0:1787739062.790983:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x151a:0x0]: returning size 4096 00000004:00080000:1.0:1787739062.790991:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739062.790999:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ad35b lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad362 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.791006:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ad35b lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad362 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.791013:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ad35b lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad362 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.791045:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739062.791049:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739062.791053:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad369 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad370 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.791060:0:13793:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad369 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad370 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.791122:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306391a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17607:x1874579652480512:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 23396us (23602us total) trans 4294983399 rc 0/0 00000100:00100000:1.0:1787739062.791128:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23466 00010000:00010000:0.0:1787739062.796700:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739062.796717:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ad35b lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad362 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.796731:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633de00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739062.796738:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ad35b lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad362 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739062.796748:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ad35b lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad362 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.796912:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ad35b lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad362 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.796921:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1787739062.796976:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665682816:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739062.797057:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787739062.797116:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ad35b lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad362 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.797125:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633de00/0x7ee4453e4a1ad35b lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad362 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.797146:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739062.797147:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739062.797149:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739062.800089:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665682816 00000100:00100000:3.0:1787739062.800103:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665682816 00000100:00100000:3.0:1787739062.800169:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1319 00000100:00100000:3.0:1787739062.800181:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562e940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+12:3650:x1874579665682816:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739062.800198:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739062.800221:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad362 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad35b expref: 12 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.800239:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074039f2100) 00010000:00010000:3.0:1787739062.800248:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739062.801138:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562e940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+11:3650:x1874579665682816:12345-0@lo:103:ldlm_bl.0 Request processed in 956us (4077us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.801149:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1319 00000100:00100000:1.0:1787739062.801216:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383c780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665682816:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739062.818368:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739062.818534:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652480896 00000100:00100000:1.0:1787739062.818543:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652480896 00000100:00100000:1.0:1787739062.818601:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23467 00000100:00100000:1.0:1787739062.818610:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306393840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17608:x1874579652480896:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:1.0:1787739062.818766:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306393840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17608:x1874579652480896:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 157us (407us total) trans 0 rc -17/-17 00000100:00100000:1.0:1787739062.818773:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23467 00000100:00100000:0.0:1787739062.848266:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739062.848392:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652481024 00000100:00100000:3.0:1787739062.848401:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652481024 00000100:00100000:3.0:1787739062.848492:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23468 00000100:00100000:3.0:1787739062.848504:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17609:x1874579652481024:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739062.848525:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739062.848549:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ad393 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09502d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.848615:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad39a lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.848629:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad39a lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.848649:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad39a lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.848661:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad3a1 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.848672:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad370 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad369 expref: 10 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.848680:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad3a1 lrc: 4/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.848711:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad370 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ad369 expref: 10 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739062.848726:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306393480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665682944:0@lo:104: 00000100:00100000:3.0:1787739062.848998:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.849095:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739062.849574:0:6506:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:2.0:1787739062.849591:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665682944 00000100:00100000:2.0:1787739062.849598:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665682944 00000100:00100000:2.0:1787739062.849646:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1161 00000100:00100000:2.0:1787739062.849655:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f913050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6524:x1874579665682944:12345-0@lo:104: 00010000:00010000:2.0:1787739062.849670:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad369 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad370 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.849738:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739062.849744:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f913050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6524:x1874579665682944:12345-0@lo:104: Request processed in 91us (747us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.849750:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1161 00010000:00010000:2.0:1787739062.849813:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739062.849819:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad369 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad370 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.849828:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aba00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739062.849834:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad369 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ad370 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739062.849843:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad369 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ad370 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.849872:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad369 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad370 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.849880:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1787739062.849916:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665683072:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739062.849951:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787739062.850030:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad369 lrc: 1/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad370 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.850040:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad369 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad370 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.850055:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739062.850057:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739062.850058:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739062.850107:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306393480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665682944:0@lo:104: 00010000:00010000:3.0:1787739062.850135:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad3a1 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739062.851201:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665683072 00000100:00100000:2.0:1787739062.851211:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665683072 00010000:00010000:2.0:1787739062.851237:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad370 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ad369 expref: 11 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739062.851271:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1518 00000100:00100000:2.0:1787739062.851280:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+11:3651:x1874579665683072:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739062.851293:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739062.859702:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad300 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad307 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739062.859728:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad300 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad307 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.859781:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad300 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad307 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.859788:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739062.860075:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad300 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad307 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787739062.860095:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968389 is committed 00000100:00100000:1.0:1787739062.860154:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060061c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665683200:0@lo:103:jbd2/dm-1-8.0 00000001:00080000:0.0:1787739062.860175:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968388 is committed 00000020:00080000:0.0:1787739062.860186:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 366 batchid updated 00000020:00080000:0.0:1787739062.860203:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa0730610e300 refcount 2 committed 0 result 0 batchid 367 00000020:00080000:0.0:1787739062.860207:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88800 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730610e0c0 00000020:00080000:0.0:1787739062.860211:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].361 00000020:00080000:0.0:1787739062.860213:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07304e88900 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07303c9b400 00000020:00080000:0.0:1787739062.860216:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].360 00000100:00100000:1.0:1787739062.860229:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787739062.860244:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ad290 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x17a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.860254:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ad290 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x17a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.860261:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ad290 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x17a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.860267:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739062.860273:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ad290 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x17a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.860280:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ad290 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x17a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.860292:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17a:0x0].0x0 (ffffa07303c78900) 00010000:00010000:0.0:1787739062.860363:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739062.860368:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ad290 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x17a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.860375:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ad290 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x17a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.860393:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad297 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.860399:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad297 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.860403:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad297 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.860408:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739062.860410:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad297 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.860440:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad297 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.860449:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc4:0x0].0x0 (ffffa07303c78700) 00010000:00010000:0.0:1787739062.860452:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739062.860455:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad297 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.860460:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad297 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xc4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:0.0:1787739062.860492:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 367 committed batchid 366 00000004:00080000:0.0:1787739062.860605:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07303937600 with version 690 00000020:00080000:0.0:1787739062.860624:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 366 cancel update log [0x200000401:0x3:0x0].360: rc = 0 00000004:00080000:1.0:1787739062.861311:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 690 rpc_version 690 00000020:00100000:1.0:1787739062.861351:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739062.861359:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743562f480 x1874579665683456/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739062.861379:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562f480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665683456:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739062.861481:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.861517:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665683456 00000100:00100000:1.0:1787739062.861523:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665683456 00000100:00100000:1.0:1787739062.861578:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3189 00000100:00100000:1.0:1787739062.861587:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665683456:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739062.861714:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665683456:12345-0@lo:1000:osp_up0-1.0 Request processed in 128us (232us total) trans 4294983400 rc 0/0 00000100:00100000:1.0:1787739062.861722:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3189 00000100:00100000:1.0:1787739062.861748:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d000 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.861778:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562f480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665683456:0@lo:1000:osp_up0-1.0 00000020:00080000:0.0:1787739062.862606:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 366 cancel update log [0x240000400:0x3:0x0].359: rc = 0 00000020:00080000:0.0:1787739062.862621:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 366 00010000:00010000:0.0:1787739062.862690:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad370 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ad369 expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.862708:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x63ac40ed (ffffa07307064c00) 00010000:00010000:0.0:1787739062.862714:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad3a1 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.862723:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad3a1 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.862730:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad3a1 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.862755:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739062.862823:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+6:3651:x1874579665683072:12345-0@lo:103:ldlm_bl.0 Request processed in 11545us (12871us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739062.862832:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1518 00000100:00100000:0.0:1787739062.862854:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665683200 00000100:00100000:0.0:1787739062.862860:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665683200 00000100:00100000:1.0:1787739062.862875:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665683072:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739062.863474:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad3a1 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.863485:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad3a1 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.863675:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304245a40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665683584:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739062.863725:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.863753:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dc00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739062.864272:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1320 00000100:00100000:0.0:1787739062.864346:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562cf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3652:x1874579665683200:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:0.0:1787739062.864364:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739062.864386:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad307 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xed2251f3 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad300 expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.864403:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xed2251f3 (ffffa07433235200) 00010000:00010000:0.0:1787739062.864427:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739062.864474:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562cf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+7:3652:x1874579665683200:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 190us (4246us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739062.864482:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1320 00000100:00100000:1.0:1787739062.864600:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060061c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665683200:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:3.0:1787739062.868525:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665683584 00000100:00100000:3.0:1787739062.868539:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665683584 00000100:00100000:3.0:1787739062.870072:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3190 00000100:00100000:3.0:1787739062.870091:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042461c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665683584:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739062.870243:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042461c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665683584:12345-0@lo:1000:mdt00_001.0 Request processed in 159us (6518us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.870258:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3190 00000100:00100000:1.0:1787739062.870345:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304245a40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665683584:0@lo:1000:mdt00_001.0 00010000:00010000:1.0:1787739062.870404:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad3a8 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x151a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.870455:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad3a8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x151a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.870466:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad3a8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x151a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739062.870501:0:6524:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad3a8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x151a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.870516:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad39a lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.870523:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad39a lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.870528:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad39a lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.870535:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad39a lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.870540:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739062.870544:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad39a lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.870551:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad39a lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.870563:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074039f2100) 00010000:00010000:1.0:1787739062.870566:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739062.870571:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad39a lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.870576:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad39a lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.870588:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad3a1 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.870593:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad3a1 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.870599:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad3a1 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.870604:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739062.870606:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad3a1 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.870612:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad3a1 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.870618:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x63ac40ed (ffffa07307064c00) 00010000:00010000:1.0:1787739062.870620:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739062.870623:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad3a1 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.870627:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad3a1 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.870679:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ad393 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09502d expref: 8 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.870697:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad3a8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x151a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09502d expref: 7 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.870704:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x151a:0x0].0x0 (ffffa07433235000) 00010000:00010000:1.0:1787739062.870706:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739062.870709:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1da00, rc 0) 00000100:00100000:1.0:1787739062.870755:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17609:x1874579652481024:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 22254us (22499us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.870762:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23468 00000100:00100000:0.0:1787739062.892662:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739062.904539:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652481152 00000100:00100000:1.0:1787739062.904554:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652481152 00000100:00100000:1.0:1787739062.904613:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23469 00000100:00100000:1.0:1787739062.904622:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17609:x1874579652481152:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739062.904645:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739062.906960:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad3af lrc: 2/0,0 mode: --/CR res: [0x200000402:0x151a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095034 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.907067:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad3b6 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x151a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.907082:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad3b6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x151a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.907094:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad3b6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x151a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739062.907202:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x151a:0x0]: returning size 4096 00000004:00200000:1.0:1787739062.907208:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739062.907249:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad3af lrc: 0/0,0 mode: --/CR res: [0x200000402:0x151a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095034 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.907265:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad3b6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x151a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095034 expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.907275:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x151a:0x0].0x0 (ffffa074039f2700) 00010000:00010000:1.0:1787739062.907278:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739062.907281:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d800, rc 0) 00000100:00100000:1.0:1787739062.907392:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17609:x1874579652481152:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 2769us (14738us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.907405:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23469 00000100:00100000:3.0:1787739062.918188:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739062.918781:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652481280 00000100:00100000:0.0:1787739062.918799:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652481280 00000100:00100000:0.0:1787739062.918865:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23470 00000100:00100000:0.0:1787739062.918876:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306393480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17609:x1874579652481280:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739062.918897:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739062.918919:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad3bd lrc: 2/0,0 mode: --/CR res: [0x200000400:0xc7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09503b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.918978:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad3c4 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xc7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.918992:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad3c4 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.919003:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad3c4 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739062.919060:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xc7:0x0]: returning size 4096 00000004:00200000:0.0:1787739062.919065:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739062.919100:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad3bd lrc: 0/0,0 mode: --/CR res: [0x200000400:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09503b expref: 19 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.919126:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad3c4 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09503b expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.919134:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc7:0x0].0x0 (ffffa074039f2b00) 00010000:00010000:0.0:1787739062.919137:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739062.919140:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a200, rc 0) 00000100:00100000:0.0:1787739062.919193:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306393480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17609:x1874579652481280:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 319us (1097us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739062.919200:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23470 00000100:00100000:1.0:1787739062.933730:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739062.933816:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652481408 00000100:00100000:3.0:1787739062.933826:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652481408 00000100:00100000:3.0:1787739062.933887:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23471 00000100:00100000:3.0:1787739062.933898:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304245a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17609:x1874579652481408:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739062.933918:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739062.933940:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad3cb lrc: 2/0,0 mode: --/CR res: [0x240000401:0xc7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095042 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.934000:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad3d2 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.934014:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad38c lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad385 expref: 5 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.934034:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad3d2 lrc: 4/1,0 mode: --/PR res: [0x240000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.934067:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad38c lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ad385 expref: 5 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739062.934082:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d84780 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665683712:0@lo:104: 00000100:00100000:3.0:1787739062.934140:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.934190:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665683712 00000100:00100000:3.0:1787739062.934194:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665683712 00000100:00100000:3.0:1787739062.934229:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1162 00000100:00100000:3.0:1787739062.934236:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d87480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6524:x1874579665683712:12345-0@lo:104: 00010000:00010000:3.0:1787739062.934248:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ad385 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad38c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.934292:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739062.934298:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d87480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6524:x1874579665683712:12345-0@lo:104: Request processed in 63us (158us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.934304:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1162 00010000:00010000:3.0:1787739062.934323:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739062.934329:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ad385 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad38c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.934337:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739062.934342:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ad385 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xc7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ad38c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739062.934350:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ad385 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xc7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ad38c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.934374:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ad385 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xc7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad38c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.934381:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739062.934399:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ad385 lrc: 1/0,0 mode: --/PW res: [0x240000401:0xc7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad38c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.934407:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ad385 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xc7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad38c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.934457:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739062.934459:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739062.934461:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739062.934472:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665683840:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739062.934501:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739062.934554:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665683840 00000100:00100000:3.0:1787739062.934558:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665683840 00010000:00010000:3.0:1787739062.934573:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad38c lrc: 5/0,0 mode: PW/PW res: [0x240000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ad385 expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739062.934592:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1519 00000100:00100000:3.0:1787739062.934598:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f75a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+6:3653:x1874579665683840:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739062.934606:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787739062.935108:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.935134:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d84780 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665683712:0@lo:104: 00010000:00010000:3.0:1787739062.935152:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad3d2 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1787739062.945629:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 367 batchid updated 00000020:00080000:3.0:1787739062.945657:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 367 00000004:00080000:3.0:1787739062.945730:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07410b4d0c0 with version 691 00000020:00080000:3.0:1787739062.945741:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 367 cancel update log [0x200000401:0x3:0x0].361: rc = 0 00000020:00080000:3.0:1787739062.945820:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 367 cancel update log [0x240000400:0x3:0x0].360: rc = 0 00000020:00080000:3.0:1787739062.945832:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 367 00010000:00010000:3.0:1787739062.945895:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad38c lrc: 0/0,0 mode: --/PW res: [0x240000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ad385 expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.945914:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc7:0x0].0x0 (ffffa07433235b00) 00010000:00010000:3.0:1787739062.945920:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad3d2 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.945930:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad3d2 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.945938:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad3d2 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.945963:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739062.946064:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f75a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+5:3653:x1874579665683840:12345-0@lo:103:ldlm_bl.0 Request processed in 11464us (11560us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.946073:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1519 00000100:00100000:3.0:1787739062.946130:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665683840:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739062.946157:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad3d2 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.946164:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad3d2 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739062.946459:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xc7:0x0]: returning size 4096 00000004:00200000:3.0:1787739062.946465:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739062.946509:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad3cb lrc: 0/0,0 mode: --/CR res: [0x240000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095042 expref: 9 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.946527:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad3d2 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095042 expref: 8 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.946534:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc7:0x0].0x0 (ffffa07433235b00) 00010000:00010000:3.0:1787739062.946537:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739062.946539:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aac00, rc 0) 00000100:00100000:3.0:1787739062.946586:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304245a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17609:x1874579652481408:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 12690us (12867us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.946593:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23471 00000004:00080000:1.0:1787739062.951104:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 691 rpc_version 691 00000020:00100000:1.0:1787739062.951167:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739062.951179:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304247c00 x1874579665684096/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739062.951209:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304247c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665684096:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739062.951283:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.951323:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623da80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.951786:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665684096 00000100:00100000:3.0:1787739062.951793:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665684096 00000100:00100000:3.0:1787739062.951850:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3191 00000100:00100000:3.0:1787739062.951860:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304246940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665684096:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739062.954089:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304246940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665684096:12345-0@lo:1000:osp_up0-1.0 Request processed in 2223us (2797us total) trans 4294983401 rc 0/0 00000100:00100000:3.0:1787739062.954103:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3191 00000100:00100000:1.0:1787739062.954277:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304247c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665684096:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739062.959579:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739062.959651:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652481536 00000100:00100000:2.0:1787739062.959659:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652481536 00000100:00100000:2.0:1787739062.959724:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1321 00000100:00100000:2.0:1787739062.959734:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074184c2c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2408:x1874579652481536:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:2.0:1787739062.959749:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739062.959774:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad3a8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x151a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09502d expref: 8 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.959804:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x151a:0x0].0x0 (ffffa07433235000) 00010000:00010000:2.0:1787739062.959815:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739062.959881:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074184c2c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652481536:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 149us (312us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.959889:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1321 00000100:00100000:3.0:1787739062.963359:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739062.963471:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652481792 00000100:00100000:2.0:1787739062.963481:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652481792 00000100:00100000:2.0:1787739062.963543:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23472 00000100:00100000:2.0:1787739062.963553:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17609:x1874579652481792:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:2.0:1787739062.963607:0:7944:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739062.963633:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ad3b6 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x151a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095034 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.963654:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x151a:0x0].0x0 (ffffa074039f2700) 00010000:00010000:2.0:1787739062.963658:0:7944:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739062.963727:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ad3d9 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.963736:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ad3d9 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.963746:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ad3d9 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739062.963768:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41fc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665684224:0@lo:101:mdt00_003.0 00000100:00100000:2.0:1787739062.963818:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.963849:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.963994:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665684224 00000100:00100000:3.0:1787739062.963999:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665684224 00000100:00100000:3.0:1787739062.964054:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3192 00000100:00100000:3.0:1787739062.964061:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306391e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:7944:x1874579665684224:12345-0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739062.964071:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739062.964086:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ad3e0 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad3d9 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.964111:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ad3e0 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad3d9 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.964124:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ad3e0 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad3d9 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.964131:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608bf00) 00010000:00010000:3.0:1787739062.964133:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739062.964137:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa600, rc 0) 00000100:00100000:3.0:1787739062.964212:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306391e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:7944:x1874579665684224:12345-0@lo:101:mdt00_003.0 Request processed in 141us (382us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.964219:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3192 00000100:00100000:0.0:1787739062.964477:0:7944:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968389 gen 2 00000004:00080000:0.0:1787739062.964484:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968385 committed transno 4294968389 00000004:00080000:0.0:1787739062.964501:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968386 committed transno 4294968389 00000004:00080000:0.0:1787739062.964509:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968387 committed transno 4294968389 00000004:00080000:0.0:1787739062.964519:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968389 committed transno 4294968389 00000100:00100000:0.0:1787739062.964538:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41fc00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665684224:0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739062.964555:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ad3d9 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad3e0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.964567:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ad3d9 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad3e0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739062.964586:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739062.964612:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad3e7 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.964620:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad3e7 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739062.964628:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad3e7 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739062.964642:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060061c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665684352:0@lo:101:mdt00_003.0 00000100:00100000:0.0:1787739062.964675:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739062.964702:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303937180 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.964739:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665684352 00000100:00100000:3.0:1787739062.964744:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665684352 00000100:00100000:3.0:1787739062.964780:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3193 00000100:00100000:3.0:1787739062.964787:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410b08450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:7944:x1874579665684352:12345-0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739062.964796:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739062.964809:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ad3ee lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad3e7 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.964824:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ad3ee lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad3e7 expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.964836:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ad3ee lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad3e7 expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739062.964843:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x63ac40ed (ffffa0730608ba00) 00010000:00010000:3.0:1787739062.964845:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739062.964848:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab600, rc 0) 00000100:00100000:3.0:1787739062.964927:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410b08450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665684352:12345-0@lo:101:mdt00_003.0 Request processed in 140us (251us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.964934:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3193 00000100:00100000:2.0:1787739062.972238:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060061c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665684352:0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739062.972273:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad3e7 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad3ee expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.972288:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad3e7 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad3ee expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739062.972317:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:2.0:1787739062.972373:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306390b40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665684480:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739062.972467:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.972493:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ef00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.972621:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665684480 00000100:00100000:3.0:1787739062.972629:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665684480 00000100:00100000:3.0:1787739062.972689:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3194 00000100:00100000:3.0:1787739062.972698:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306390f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665684480:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739062.972797:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306390f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665684480:12345-0@lo:1000:mdt00_003.0 Request processed in 100us (330us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739062.972805:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3194 00000100:00100000:2.0:1787739062.973112:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306390b40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665684480:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739062.973154:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306390b40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665684608:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739062.973180:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739062.973815:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ef00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.974131:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665684608 00000100:00100000:3.0:1787739062.974139:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665684608 00000100:00100000:3.0:1787739062.974196:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3195 00000100:00100000:3.0:1787739062.974215:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073063921c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665684608:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739062.975145:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073063921c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665684608:12345-0@lo:1000:mdt00_003.0 Request processed in 935us (1958us total) trans 0 rc 1/1 00000100:00100000:3.0:1787739062.975155:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3195 00000100:00100000:1.0:1787739062.978451:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306390b40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665684608:0@lo:1000:mdt00_003.0 00010000:00010000:1.0:1787739062.978544:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad3f5 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.978560:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad3f5 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.978575:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad3f5 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.978600:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304246580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665684736:0@lo:101:mdt00_003.0 00000100:00100000:1.0:1787739062.978676:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.978714:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623da80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739062.978858:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665684736 00000100:00100000:2.0:1787739062.978864:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665684736 00000100:00100000:2.0:1787739062.978950:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3196 00000100:00100000:2.0:1787739062.978961:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304245e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665684736:12345-0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739062.978977:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739062.978995:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad3fc lrc: 2/0,0 mode: --/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad3f5 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.979031:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad3fc lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad3f5 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.979046:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad3fc lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad3f5 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739062.979054:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x151a:0x0].0x0 (ffffa074419d5300) 00010000:00010000:2.0:1787739062.979058:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739062.979062:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d400, rc 0) 00000100:00100000:2.0:1787739062.979107:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304245e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665684736:12345-0@lo:101:mdt00_003.0 Request processed in 147us (444us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739062.979113:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3196 00000100:00100000:1.0:1787739062.980710:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304246580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665684736:0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739062.980892:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad3f5 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad3fc expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.980912:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad3f5 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad3fc expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.980954:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad403 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.980964:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad403 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.980972:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad403 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.980991:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad40a lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.981004:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad3c4 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09503b expref: 17 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.981012:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad40a lrc: 4/0,1 mode: --/EX res: [0x200000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.981050:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad3c4 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09503b expref: 17 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.981064:0:7944:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad3c4 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09503b expref: 17 pid: 6525 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.981077:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304246580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665684864:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739062.981156:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623da80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739062.988891:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304246580 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665684864:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739062.988920:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad40a lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739062.990230:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739062.990343:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652481920 00000100:00100000:1.0:1787739062.990352:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652481920 00010000:00010000:1.0:1787739062.990387:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad3c4 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09503b expref: 18 pid: 6525 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.990434:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1520 00000100:00100000:1.0:1787739062.990443:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f77840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2409:x1874579652481920:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739062.990457:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739062.990464:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad3c4 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09503b expref: 18 pid: 6525 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.990477:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad3c4 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09503b expref: 18 pid: 6525 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.990499:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad3c4 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09503b expref: 18 pid: 6525 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.990512:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc7:0x0].0x0 (ffffa074039f2b00) 00010000:00010000:1.0:1787739062.990517:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad40a lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.990526:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad40a lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.990534:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad40a lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.990557:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739062.990606:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f77840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652481920:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 163us (381us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739062.990614:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1520 00010000:00010000:1.0:1787739062.990712:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad40a lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.990720:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad40a lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.990761:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1ad411 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.990769:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1ad411 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.990779:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1ad411 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.990796:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042470c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665684992:0@lo:101:mdt00_003.0 00000100:00100000:1.0:1787739062.990834:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739062.990858:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d000 going to sleep for 11 seconds 00000100:00100000:1.0:1787739062.990873:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665684992 00000100:00100000:1.0:1787739062.990877:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665684992 00000100:00100000:1.0:1787739062.990907:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3197 00000100:00100000:1.0:1787739062.990914:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304244780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665684992:12345-0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739062.990928:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739062.990940:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad418 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad411 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.990960:0:12410:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad3d2 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095042 expref: 6 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.990973:0:12410:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad418 lrc: 4/0,0 mode: --/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad411 expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.990995:0:12410:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad3d2 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095042 expref: 6 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739062.991008:0:12410:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad3d2 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095042 expref: 6 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739062.991029:0:12410:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304247480 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001_UUID:12410:1874579665685120:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739062.991072:0:12410:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dc00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739063.005206:0:12410:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304247480 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001_UUID:12410:1874579665685120:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739063.005243:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad418 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad411 expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.005255:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc7:0x0].0x0 (ffffa07433235b00) 00010000:00010000:2.0:1787739063.005259:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739063.005263:0:12410:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad418 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad411 expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.005273:0:12410:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad418 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad411 expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.005280:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1dc00, rc 0) 00000100:00100000:2.0:1787739063.005357:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304244780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665684992:12345-0@lo:101:mdt00_003.0 Request processed in 14442us (14520us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.005366:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3197 00000100:00100000:1.0:1787739063.005481:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042470c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665684992:0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739063.005500:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1ad411 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad418 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.005510:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1ad411 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad418 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739063.006483:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739063.008309:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652482048 00000100:00100000:3.0:1787739063.008321:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652482048 00010000:00010000:3.0:1787739063.008368:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad3d2 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095042 expref: 7 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739063.008720:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1521 00000100:00100000:3.0:1787739063.008734:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304246d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2409:x1874579652482048:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739063.008751:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739063.008759:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad3d2 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095042 expref: 7 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.009104:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad3d2 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095042 expref: 7 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.009132:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad3d2 lrc: 0/0,0 mode: --/PR res: [0x240000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095042 expref: 7 pid: 6524 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.009145:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc7:0x0].0x0 (ffffa07433235b00) 00010000:00010000:3.0:1787739063.009152:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad418 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad411 expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.009162:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad418 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad411 expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.009172:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad418 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad411 expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.009208:0:6509:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad418 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad411 expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739063.009224:0:6509:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f75a40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001_UUID:6509:1874579665685248:0@lo:105: 00000100:00080000:1.0:1787739063.009408:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787739063.010426:0:6509:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.010488:0:6509:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.015491:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665685248 00000100:00100000:0.0:1787739063.015509:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665685248 00000100:00100000:0.0:1787739063.015576:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1163 00000100:00100000:0.0:1787739063.015588:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6509:x1874579665685248:12345-0@lo:105: 00010000:00010000:0.0:1787739063.015605:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1ad411 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad418 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.015618:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1ad411 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad418 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.015627:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1ad411 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad418 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.015636:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1ad411 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad418 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.015649:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1ad411 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad418 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.015658:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1ad411 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ad418 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.015690:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07441a1ce00) 00000100:00100000:0.0:1787739063.015779:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6509:x1874579665685248:12345-0@lo:105: Request processed in 193us (6487us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.015787:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1163 00000100:00100000:0.0:1787739063.015848:0:6509:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f75a40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001_UUID:6509:1874579665685248:0@lo:105: 00010000:00010000:0.0:1787739063.015886:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739063.015935:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304246d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2409:x1874579652482048:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 7204us (9457us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.015944:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1521 00010000:00010000:0.0:1787739063.015981:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1ad411 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad418 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.015990:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1ad411 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad418 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.018654:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060061c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665685376:0@lo:403:mdt00_003.0 00000100:00100000:0.0:1787739063.018727:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.018766:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303937000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739063.018819:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665685376 00000100:00100000:2.0:1787739063.018827:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665685376 00000100:00100000:2.0:1787739063.018873:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3198 00000100:00100000:2.0:1787739063.018881:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f75a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665685376:12345-0@lo:403:mdt00_003.0 00000100:00100000:2.0:1787739063.022157:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f75a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665685376:12345-0@lo:403:mdt00_003.0 Request processed in 3266us (3419us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.022175:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3198 00000100:00100000:0.0:1787739063.026149:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060061c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665685376:0@lo:403:mdt00_003.0 00000040:00080000:0.0:1787739063.027239:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].375, 664 off573384 00000040:00080000:0.0:1787739063.027499:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].374, 744 off620488 00000004:00080000:0.0:1787739063.027506:0:7944:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07303937000 with version 715 00000004:00080000:1.0:1787739063.030891:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 715 rpc_version 715 00000020:00100000:1.0:1787739063.030938:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739063.030950:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304245680 x1874579665685632/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2064/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739063.030975:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304245680 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665685632:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739063.031053:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739063.031088:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d000 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.031126:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665685632 00000100:00100000:0.0:1787739063.031134:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665685632 00000100:00100000:0.0:1787739063.031188:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3199 00000100:00100000:0.0:1787739063.031197:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042452c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665685632:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739063.031640:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042452c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665685632:12345-0@lo:1000:osp_up1-0.0 Request processed in 443us (585us total) trans 4294968390 rc 0/0 00000100:00100000:0.0:1787739063.031649:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3199 00000100:00100000:1.0:1787739063.031855:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304245680 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665685632:0@lo:1000:osp_up1-0.0 00000004:00200000:0.0:1787739063.032204:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x151a:0x0]: returning size 4096 00010000:00010000:0.0:1787739063.032227:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad40a lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032237:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad40a lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032245:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad40a lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032251:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739063.032255:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad40a lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032262:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad40a lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032277:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc7:0x0].0x0 (ffffa074039f2b00) 00010000:00010000:0.0:1787739063.032280:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739063.032284:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad40a lrc: 2/0,0 mode: --/EX res: [0x200000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032290:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad40a lrc: 0/0,0 mode: --/EX res: [0x200000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032303:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1ad411 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad418 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032311:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1ad411 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad418 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032317:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1ad411 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad418 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032341:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739063.032350:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xc7:0x0] 00010000:00010000:0.0:1787739063.032357:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad403 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032363:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad403 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032368:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad403 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032372:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739063.032375:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad403 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032380:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad403 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032391:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x151a:0x0].0x0 (ffffa074039f2700) 00010000:00010000:0.0:1787739063.032393:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739063.032396:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad403 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032401:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad403 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032410:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad3f5 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad3fc expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032437:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad3f5 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad3fc expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032443:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad3f5 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad3fc expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032457:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:0.0:1787739063.032466:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739063.032474:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787739063.032479:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1ad411 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad418 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032485:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ce00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739063.032488:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1ad411 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad418 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739063.032495:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1ad411 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad418 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032523:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1ad411 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad418 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032529:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739063.032548:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1ad411 lrc: 1/0,0 mode: --/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad418 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032555:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1ce00/0x7ee4453e4a1ad411 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad418 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032570:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739063.032573:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad3f5 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad3fc expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032579:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739063.032583:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad3f5 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad3fc expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739063.032589:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad3f5 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad3fc expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032606:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad3f5 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad3fc expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032611:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739063.032617:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad3f5 lrc: 1/0,0 mode: --/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad3fc expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032628:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad3f5 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad3fc expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.032639:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739063.032641:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739063.032643:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739063.032656:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060061c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665685760:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739063.032699:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.032730:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787739063.032736:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665685888:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739063.032757:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.032787:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665685760 00000100:00100000:0.0:1787739063.032792:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665685760 00000100:00100000:0.0:1787739063.032829:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665685888 00000100:00100000:0.0:1787739063.032832:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665685888 00010000:00010000:1.0:1787739063.032995:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739063.034052:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1322 00000100:00100000:0.0:1787739063.034072:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41fc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+11:3650:x1874579665685760:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739063.034089:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787739063.034158:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1323 00000100:00100000:1.0:1787739063.034170:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+11:3650:x1874579665685888:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739063.034180:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739063.034195:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad3fc lrc: 0/0,0 mode: --/EX res: [0x200000402:0x151a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad3f5 expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.034215:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x151a:0x0].0x0 (ffffa074419d5300) 00010000:00010000:1.0:1787739063.034224:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739063.034284:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+10:3650:x1874579665685888:12345-0@lo:103:ldlm_bl.0 Request processed in 118us (1526us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739063.034292:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1323 00010000:00010000:1.0:1787739063.034632:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ad3d9 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad3e0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.034643:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ad3d9 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad3e0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.034651:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ad3d9 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad3e0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.034668:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739063.034677:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739063.034682:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad3e7 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad3ee expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.034690:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad3e7 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad3ee expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739063.034755:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17609:x1874579652481792:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 71202us (71403us total) trans 4294983402 rc 0/0 00000100:00100000:1.0:1787739063.034762:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23472 00010000:00010000:2.0:1787739063.034883:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739063.034890:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ad3d9 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad3e0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.034899:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633ce00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739063.034904:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ad3d9 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad3e0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739063.034913:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ad3d9 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad3e0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.034943:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ad3d9 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad3e0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.034950:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739063.034975:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ad3d9 lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad3e0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.034982:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633ce00/0x7ee4453e4a1ad3d9 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad3e0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.034998:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739063.035000:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739063.035002:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739063.035010:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787739063.035032:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306393c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665686016:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739063.035069:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739063.035125:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665685888:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739063.035470:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665686016 00000100:00100000:2.0:1787739063.035477:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665686016 00000100:00100000:2.0:1787739063.035520:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1324 00000100:00100000:2.0:1787739063.035528:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306390780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+10:3653:x1874579665686016:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739063.035538:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739063.035555:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ad3e0 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad3d9 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.035569:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608bf00) 00010000:00010000:0.0:1787739063.035573:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad418 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad411 expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.035578:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739063.035592:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc7:0x0].0x0 (ffffa07433235b00) 00010000:00010000:0.0:1787739063.035602:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739063.035619:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306390780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+8:3653:x1874579665686016:12345-0@lo:103:ldlm_bl.0 Request processed in 91us (548us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.035626:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1324 00000100:00100000:0.0:1787739063.035650:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41fc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+7:3650:x1874579665685760:12345-0@lo:103:ldlm_bl.0 Request processed in 1586us (2949us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.035659:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306393c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665686016:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739063.036073:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1322 00000100:00100000:0.0:1787739063.036272:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060061c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665685760:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739063.058502:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739063.058629:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652482176 00000100:00100000:3.0:1787739063.058639:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652482176 00000100:00100000:3.0:1787739063.058744:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23473 00000100:00100000:3.0:1787739063.058765:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17610:x1874579652482176:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739063.058791:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739063.058812:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad41f lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095049 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.058895:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad426 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.058910:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad426 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.058921:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad426 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.058935:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad42d lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.058954:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad42d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.058961:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad42d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.059004:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad426 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.059012:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad426 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.059032:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad426 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.059039:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad426 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.059045:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aba00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739063.059050:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad426 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.059057:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad426 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.059069:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074039f2e00) 00010000:00010000:3.0:1787739063.059072:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739063.059076:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad426 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.059082:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad426 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.059094:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad42d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.059100:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad42d lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.059105:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad42d lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.059110:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aae00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739063.059113:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad42d lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.059118:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad42d lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.059125:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd1c5f96a (ffffa0730608b700) 00010000:00010000:3.0:1787739063.059128:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739063.059131:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad42d lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.059136:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad42d lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.059164:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad41f lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095049 expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.059173:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074039f2e00) 00010000:00010000:3.0:1787739063.059175:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739063.059178:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad41f lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095049 expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.059186:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aac00, rc 0) 00000100:00100000:3.0:1787739063.059247:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17610:x1874579652482176:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 494us (752us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739063.059255:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23473 00000100:00080000:1.0:1787739063.059813:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787739063.072714:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739063.072846:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652482304 00000100:00100000:0.0:1787739063.072856:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652482304 00000100:00100000:0.0:1787739063.072926:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23474 00000100:00100000:0.0:1787739063.072936:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17610:x1874579652482304:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739063.072966:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739063.072988:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad434 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095050 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.073066:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad43b lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.073080:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad43b lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.073092:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad43b lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739063.073142:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x5:0x0]: returning size 4096 00000004:00200000:0.0:1787739063.073148:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739063.073183:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad434 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095050 expref: 8 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.073201:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad43b lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095050 expref: 7 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.073209:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064a00) 00010000:00010000:0.0:1787739063.073212:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739063.073216:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a200, rc 0) 00000100:00100000:0.0:1787739063.073292:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17610:x1874579652482304:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 349us (579us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.073299:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23474 00000100:00100000:3.0:1787739063.095875:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739063.095953:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652482560 00000100:00100000:1.0:1787739063.095964:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652482560 00000100:00100000:1.0:1787739063.096032:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23475 00000100:00100000:1.0:1787739063.096042:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17610:x1874579652482560:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:1.0:1787739063.096156:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304244b40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665686144:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739063.096205:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739063.096253:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739063.096359:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665686144 00000100:00100000:2.0:1787739063.096363:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665686144 00000100:00100000:2.0:1787739063.096401:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3200 00000100:00100000:2.0:1787739063.096408:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304247840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665686144:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739063.096498:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304247840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665686144:12345-0@lo:1000:mdt00_003.0 Request processed in 90us (292us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.096505:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3200 00000100:00100000:1.0:1787739063.096654:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304244b40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665686144:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739063.096698:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304244b40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665686272:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739063.096722:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739063.096788:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739063.099748:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665686272 00000100:00100000:2.0:1787739063.100188:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665686272 00000100:00100000:2.0:1787739063.100317:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3201 00000100:00100000:2.0:1787739063.100336:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304244000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665686272:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739063.100490:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304244000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665686272:12345-0@lo:1000:mdt00_003.0 Request processed in 165us (3766us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739063.100498:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3201 00000100:00100000:2.0:1787739063.101116:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304244b40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665686272:0@lo:1000:mdt00_003.0 00010000:00010000:2.0:1787739063.101784:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1ad442 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.101797:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1ad442 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.101808:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1ad442 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739063.101830:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306393c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665686400:0@lo:101:mdt00_003.0 00000100:00100000:2.0:1787739063.101895:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739063.101940:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739063.103078:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665686400 00000100:00100000:1.0:1787739063.103088:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665686400 00000100:00100000:1.0:1787739063.103185:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3202 00000100:00100000:1.0:1787739063.103195:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306392d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665686400:12345-0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739063.103214:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739063.103235:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad449 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad442 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.103266:0:12410:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad354 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095026 expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.103277:0:12410:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad449 lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad442 expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.103321:0:12410:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad354 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095026 expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.103335:0:12410:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad354 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095026 expref: 16 pid: 6524 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739063.103347:0:12410:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042443c0 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:12410:1874579665686528:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739063.103440:0:12410:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623da80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739063.108751:0:12410:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042443c0 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:12410:1874579665686528:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739063.108800:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad449 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad442 expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.108814:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074039f2e00) 00010000:00010000:1.0:1787739063.108816:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739063.108820:0:12410:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad449 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad442 expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.108830:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c000, rc 0) 00000100:00100000:1.0:1787739063.108914:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306392d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665686400:12345-0@lo:101:mdt00_003.0 Request processed in 5719us (7017us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739063.108925:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3202 00000100:00100000:2.0:1787739063.109092:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306393c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665686400:0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739063.109110:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1ad442 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad449 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.109121:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1ad442 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad449 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739063.110269:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739063.110454:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652482688 00000100:00100000:1.0:1787739063.110464:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652482688 00010000:00010000:1.0:1787739063.110495:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad354 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095026 expref: 17 pid: 6524 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739063.110537:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1522 00000100:00100000:1.0:1787739063.110548:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f3840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2408:x1874579652482688:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739063.110563:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739063.110569:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad354 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095026 expref: 17 pid: 6524 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.110581:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad354 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095026 expref: 17 pid: 6524 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.110601:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab000/0x7ee4453e4a1ad354 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095026 expref: 17 pid: 6524 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.110615:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074039f2e00) 00010000:00010000:1.0:1787739063.110621:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad449 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad442 expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.110630:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad449 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad442 expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.110638:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad449 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad442 expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.110674:0:6508:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad449 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad442 expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739063.110689:0:6508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042443c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665686656:0@lo:105: 00000100:00100000:1.0:1787739063.110734:0:6508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739063.110774:0:6508:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623da80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739063.110779:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665686656 00000100:00100000:2.0:1787739063.110794:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665686656 00000100:00100000:2.0:1787739063.110825:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1164 00000100:00100000:2.0:1787739063.110832:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304244f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665686656:12345-0@lo:105: 00010000:00010000:2.0:1787739063.110843:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1ad442 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad449 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.110856:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1ad442 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad449 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.110864:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1ad442 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad449 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.110871:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1ad442 lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad449 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.110877:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1ad442 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad449 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.110886:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1ad442 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ad449 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.110921:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633ce00) 00000100:00100000:2.0:1787739063.110972:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304244f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6508:x1874579665686656:12345-0@lo:105: Request processed in 140us (237us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.110979:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1164 00000100:00100000:3.0:1787739063.111129:0:6508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042443c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:6508:1874579665686656:0@lo:105: 00010000:00010000:3.0:1787739063.111144:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739063.111191:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f3840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:2408:x1874579652482688:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 645us (927us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.111198:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1522 00010000:00010000:1.0:1787739063.111321:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1ad442 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad449 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.111331:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1ad442 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad449 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739063.111359:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739063.111394:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad450 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.111401:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad450 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.111409:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad450 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739063.111456:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042443c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665686784:0@lo:101:mdt00_003.0 00000100:00100000:1.0:1787739063.111483:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739063.111507:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739063.111692:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665686784 00000100:00100000:2.0:1787739063.111697:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665686784 00000100:00100000:2.0:1787739063.111734:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3203 00000100:00100000:2.0:1787739063.111741:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304244b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:7944:x1874579665686784:12345-0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739063.111755:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739063.111771:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad457 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad450 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.111808:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad457 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad450 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.111820:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad457 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad450 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.111827:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd1c5f96a (ffffa074419d5e00) 00010000:00010000:2.0:1787739063.111830:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739063.111833:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633de00, rc 0) 00000100:00100000:2.0:1787739063.111868:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304244b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665686784:12345-0@lo:101:mdt00_003.0 Request processed in 128us (384us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.111874:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3203 00000100:00100000:1.0:1787739063.111876:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042443c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665686784:0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739063.111888:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad450 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad457 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.111898:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad450 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad457 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739063.111911:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:1.0:1787739063.111954:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042443c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665686912:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739063.111980:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739063.112060:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.115309:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665686912 00000100:00100000:0.0:1787739063.115325:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665686912 00000100:00100000:0.0:1787739063.115466:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3204 00000100:00100000:0.0:1787739063.115479:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042470c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665686912:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739063.115607:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042470c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665686912:12345-0@lo:1000:mdt00_003.0 Request processed in 130us (3625us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.115613:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3204 00000100:00100000:2.0:1787739063.115699:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042443c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665686912:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739063.115772:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306391680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665687040:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739063.115814:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739063.119650:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e840 going to sleep for 11 seconds 00000100:00100000:1.0:1787739063.123010:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665687040 00000100:00100000:1.0:1787739063.123204:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665687040 00000100:00100000:1.0:1787739063.123271:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3205 00000100:00100000:1.0:1787739063.123281:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073063912c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665687040:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739063.123968:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073063912c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665687040:12345-0@lo:1000:mdt00_003.0 Request processed in 685us (8151us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739063.123978:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3205 00000100:00100000:2.0:1787739063.127154:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306391680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665687040:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739063.127289:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306391680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665687168:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739063.127346:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739063.127373:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e840 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.127498:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665687168 00000100:00100000:3.0:1787739063.127506:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665687168 00000100:00100000:3.0:1787739063.127556:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3206 00000100:00100000:3.0:1787739063.127566:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073063930c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665687168:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739063.127665:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073063930c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665687168:12345-0@lo:1000:mdt00_003.0 Request processed in 101us (318us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739063.127672:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3206 00000100:00100000:0.0:1787739063.127771:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306391680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665687168:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739063.127820:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665687296:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739063.127851:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.127875:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739063.128108:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665687296 00000100:00100000:1.0:1787739063.128114:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665687296 00000100:00100000:1.0:1787739063.128157:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3207 00000100:00100000:1.0:1787739063.128165:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665687296:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739063.128232:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665687296:12345-0@lo:1000:mdt00_003.0 Request processed in 68us (379us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739063.128238:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3207 00000100:00100000:0.0:1787739063.128469:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665687296:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739063.128514:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665687424:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739063.128540:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.128563:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404c00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.128935:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665687424 00000100:00100000:3.0:1787739063.128953:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665687424 00000100:00100000:3.0:1787739063.128988:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3208 00000100:00100000:3.0:1787739063.128995:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665687424:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739063.129086:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665687424:12345-0@lo:1000:mdt00_003.0 Request processed in 92us (545us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.129094:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3208 00000100:00100000:0.0:1787739063.129378:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665687424:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739063.129461:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665687552:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739063.129492:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.129513:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739063.130377:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665687552 00000100:00100000:1.0:1787739063.130386:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665687552 00000100:00100000:1.0:1787739063.130462:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3209 00000100:00100000:1.0:1787739063.130472:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665687552:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739063.130576:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665687552:12345-0@lo:1000:mdt00_003.0 Request processed in 106us (1082us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739063.130584:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3209 00000100:00100000:0.0:1787739063.130614:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665687552:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739063.130660:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665687680:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739063.130688:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.130703:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739063.130747:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665687680 00000100:00100000:2.0:1787739063.130752:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665687680 00000100:00100000:2.0:1787739063.130787:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3210 00000100:00100000:2.0:1787739063.130794:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665687680:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739063.130853:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:7944:x1874579665687680:12345-0@lo:1000:mdt00_003.0 Request processed in 60us (163us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.130858:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3210 00000100:00100000:0.0:1787739063.130997:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665687680:0@lo:1000:mdt00_003.0 00000040:00080000:0.0:1787739063.131797:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].362, 2168 off549400 00000040:00080000:0.0:1787739063.131890:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].363, 2368 off601528 00000004:00080000:0.0:1787739063.131897:0:7944:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07410404f00 with version 692 00000004:00080000:1.0:1787739063.132007:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 692 rpc_version 692 00000020:00100000:1.0:1787739063.132109:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739063.132118:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304247480 x1874579665687936/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739063.132134:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304247480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665687936:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739063.132182:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739063.132209:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739063.132303:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665687936 00000100:00100000:2.0:1787739063.132310:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665687936 00000100:00100000:2.0:1787739063.132355:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3211 00000100:00100000:2.0:1787739063.132363:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304246580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665687936:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739063.132878:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304246580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:7467:x1874579665687936:12345-0@lo:1000:osp_up0-1.0 Request processed in 514us (693us total) trans 4294983403 rc 0/0 00000100:00100000:2.0:1787739063.132886:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3211 00000100:00100000:1.0:1787739063.133297:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304247480 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665687936:0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1787739063.133551:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad45e lrc: 3/0,1 mode: --/PW res: [0x240000402:0x17c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.133568:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad45e lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.133577:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad45e lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.133594:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad465 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.133600:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad465 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.133605:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad465 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.133631:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ad46c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.133638:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ad46c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.133646:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ad46c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.133665:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665688064:0@lo:101:mdt00_003.0 00000100:00100000:0.0:1787739063.133701:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.133729:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074051da600 going to sleep for 11 seconds 00000100:00100000:1.0:1787739063.134025:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665688064 00000100:00100000:1.0:1787739063.134031:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665688064 00000100:00100000:1.0:1787739063.134076:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3212 00000100:00100000:1.0:1787739063.134085:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:7944:x1874579665688064:12345-0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739063.134101:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739063.134118:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad473 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad46c expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.134138:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad473 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad46c expref: 12 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.134149:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad473 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad46c expref: 12 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.134157:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc5:0x0].0x0 (ffffa07433235600) 00010000:00010000:1.0:1787739063.134160:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739063.134163:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1dc00, rc 0) 00000100:00100000:1.0:1787739063.134285:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:7944:x1874579665688064:12345-0@lo:101:mdt00_003.0 Request processed in 200us (582us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739063.134293:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3212 00000100:00100000:0.0:1787739063.135062:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665688064:0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739063.135083:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ad46c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad473 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.135095:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ad46c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad473 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739063.135120:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000004f8ecc75 reply state 00000000b2c3e81c transno 4294968391 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad465 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.135132:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad465 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.135139:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc5:0x0].0x0 (ffffa07307064700) 00010000:00010000:0.0:1787739063.135142:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739063.135146:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739063.135153:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ad46c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad473 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.135161:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ad46c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad473 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739063.135172:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xc5:0x0] 00000004:00010000:0.0:1787739063.135179:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000004f8ecc75 reply state 00000000b2c3e81c transno 4294968391 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad45e lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.135192:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad45e lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.135198:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17c:0x0].0x0 (ffffa07307064e00) 00010000:00010000:0.0:1787739063.135200:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739063.135202:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787739063.135207:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17c:0x0]: returning size 4096 00000004:00080000:0.0:1787739063.135215:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739063.135223:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1ad442 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad449 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.135230:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1ad442 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad449 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.135236:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1ad442 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad449 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.135263:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739063.135268:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739063.135273:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad450 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad457 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.135280:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad450 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad457 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.135301:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739063.135309:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1ad442 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad449 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.135316:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633ce00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739063.135321:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1ad442 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad449 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739063.135328:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1ad442 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad449 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.135356:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1ad442 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad449 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.135362:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1787739063.135371:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17610:x1874579652482560:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 39329us (39504us total) trans 4294968391 rc 0/0 00000100:00100000:0.0:1787739063.135379:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23475 00010000:00010000:3.0:1787739063.135386:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1ad442 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad449 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.135397:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633ce00/0x7ee4453e4a1ad442 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad449 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.135465:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739063.135468:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739063.135469:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739063.135516:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f1e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665688192:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739063.135562:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.135611:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665688192 00000100:00100000:0.0:1787739063.135617:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665688192 00000100:00100000:0.0:1787739063.135662:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1325 00000100:00100000:0.0:1787739063.135670:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba4050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+12:3653:x1874579665688192:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739063.135681:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739063.135697:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad449 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad442 expref: 12 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.135710:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074039f2e00) 00010000:00010000:0.0:1787739063.135717:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739063.135763:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba4050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+11:3653:x1874579665688192:12345-0@lo:103:ldlm_bl.0 Request processed in 94us (200us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.135770:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1325 00000100:00100000:3.0:1787739063.135784:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f1e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665688192:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739063.163240:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739063.163353:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652482944 00000100:00100000:1.0:1787739063.163363:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652482944 00000100:00100000:1.0:1787739063.163443:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23476 00000100:00100000:1.0:1787739063.163454:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17611:x1874579652482944:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:1.0:1787739063.163605:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17611:x1874579652482944:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 152us (374us total) trans 0 rc -17/-17 00000100:00100000:1.0:1787739063.163611:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23476 00000100:00100000:3.0:1787739063.182596:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739063.183149:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652483072 00000100:00100000:2.0:1787739063.183162:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652483072 00000100:00100000:2.0:1787739063.183238:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23477 00000100:00100000:2.0:1787739063.183250:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17612:x1874579652483072:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739063.183274:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739063.183300:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ad47a lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095057 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.183376:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad481 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.183391:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad481 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.183404:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad481 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.183438:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad488 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.183450:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad457 lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad450 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.183459:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad488 lrc: 4/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.183497:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad457 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ad450 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739063.183515:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6525:1874579665688320:0@lo:104: 00000100:00100000:2.0:1787739063.183580:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739063.183634:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739063.183650:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665688320 00000100:00100000:2.0:1787739063.183654:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665688320 00000100:00100000:2.0:1787739063.183683:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1165 00000100:00100000:2.0:1787739063.183689:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306392940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6525:x1874579665688320:12345-0@lo:104: 00010000:00010000:2.0:1787739063.183702:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad450 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad457 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.183748:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739063.183755:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306392940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6525:x1874579665688320:12345-0@lo:104: Request processed in 66us (175us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.183761:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1165 00010000:00010000:2.0:1787739063.183785:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739063.183791:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad450 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad457 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.183801:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739063.183807:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad450 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ad457 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739063.183817:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad450 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ad457 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.183845:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad450 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad457 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.183852:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739063.183872:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad450 lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad457 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.183880:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad450 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad457 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.183897:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739063.183899:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739063.183901:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739063.183912:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306391e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665688448:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739063.183954:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739063.183994:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665688448 00000100:00100000:2.0:1787739063.183998:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665688448 00010000:00010000:2.0:1787739063.184013:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad457 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ad450 expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739063.184049:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1523 00000100:00100000:2.0:1787739063.184056:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306390780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3650:x1874579665688448:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739063.184066:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1787739063.184786:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6525:1874579665688320:0@lo:104: 00010000:00010000:2.0:1787739063.184805:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad488 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.195120:0:6502:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad3e7 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad3ee expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739063.195141:0:6502:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad3e7 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad3ee expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.195186:0:6502:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad3e7 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad3ee expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.195193:0:6502:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739063.195517:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad3e7 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad3ee expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787739063.195538:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983403 is committed 00000001:00080000:2.0:1787739063.195556:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983402 is committed 00000020:00080000:2.0:1787739063.195568:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07410404180 refcount 2 committed 0 result 0 batchid 377 00000020:00080000:2.0:1787739063.195572:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07404f86200 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07410404300 00000020:00080000:2.0:1787739063.195575:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].373 00000020:00080000:2.0:1787739063.195577:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07404f86000 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07303c9a200 00000020:00080000:2.0:1787739063.195580:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].374 00010000:00010000:2.0:1787739063.195610:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad377 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x151a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.195619:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad377 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x151a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.195625:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad377 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x151a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.195631:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aaa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739063.195636:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad377 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x151a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.195642:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad377 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x151a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.195655:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x151a:0x0].0x0 (ffffa074039f2700) 00010000:00010000:2.0:1787739063.195658:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739063.195751:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665688576:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:0.0:1787739063.195827:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787739063.197359:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad377 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x151a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.197376:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad377 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x151a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.197401:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad37e lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.197408:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad37e lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.197430:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad37e lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.197437:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739063.197441:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad37e lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.197447:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad37e lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.197461:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc7:0x0].0x0 (ffffa074039f2b00) 00010000:00010000:2.0:1787739063.197464:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739063.197467:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad37e lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.197482:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad37e lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739063.197515:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 377 committed batchid 376 00000020:00080000:2.0:1787739063.197583:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 376 00010000:00010000:2.0:1787739063.197647:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad457 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ad450 expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.197659:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd1c5f96a (ffffa074419d5e00) 00010000:00010000:2.0:1787739063.197663:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad488 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.197673:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad488 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.197680:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad488 lrc: 4/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.197716:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739063.197796:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306390780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+6:3650:x1874579665688448:12345-0@lo:103:ldlm_bl.0 Request processed in 13739us (13854us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.197807:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1523 00000100:00100000:2.0:1787739063.197830:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665688576 00000100:00100000:2.0:1787739063.197837:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665688576 00000100:00100000:2.0:1787739063.197878:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1326 00000100:00100000:2.0:1787739063.197886:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+7:3651:x1874579665688576:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:2.0:1787739063.197900:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739063.197914:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ad3ee lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x63ac40ed bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad3e7 expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.197924:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x63ac40ed (ffffa0730608ba00) 00010000:00010000:2.0:1787739063.197930:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739063.197960:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+6:3651:x1874579665688576:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 75us (2134us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.197967:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1326 00000100:00100000:2.0:1787739063.198010:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306391e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665688448:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739063.198051:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665688576:0@lo:103:jbd2/dm-0-8.0 00010000:00010000:0.0:1787739063.199564:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad488 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.199578:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad488 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.199713:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665688704:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739063.199777:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.199808:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068483c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.199832:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665688704 00000100:00100000:0.0:1787739063.199837:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665688704 00000100:00100000:0.0:1787739063.199880:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3213 00000100:00100000:0.0:1787739063.199887:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6525:x1874579665688704:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739063.199991:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6525:x1874579665688704:12345-0@lo:1000:mdt00_002.0 Request processed in 103us (213us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.199997:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3213 00000100:00100000:0.0:1787739063.200044:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665688704:0@lo:1000:mdt00_002.0 00010000:00010000:0.0:1787739063.200089:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad48f lrc: 3/1,0 mode: --/PR res: [0x240000402:0x17c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.200102:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad48f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.200113:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad48f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739063.200133:0:6525:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad48f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.200146:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad481 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.200153:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad481 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.200159:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad481 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.200165:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad481 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.200171:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739063.200175:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad481 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.200185:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad481 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.200196:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5700) 00010000:00010000:0.0:1787739063.200199:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739063.200203:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad481 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.200368:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad481 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.200382:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad488 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.200388:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad488 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.200395:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad488 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.200400:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739063.200403:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad488 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.200410:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad488 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.200433:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd1c5f96a (ffffa074419d5e00) 00010000:00010000:0.0:1787739063.200436:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739063.200439:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad488 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.200449:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad488 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.200485:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ad47a lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095057 expref: 15 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.200501:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad48f lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095057 expref: 14 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.200508:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17c:0x0].0x0 (ffffa07307064200) 00010000:00010000:0.0:1787739063.200511:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739063.200513:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a000, rc 0) 00000100:00100000:0.0:1787739063.200557:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17612:x1874579652483072:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 17310us (17969us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.200564:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23477 00000100:00100000:1.0:1787739063.220505:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739063.221469:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652483200 00000100:00100000:0.0:1787739063.221481:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652483200 00000100:00100000:0.0:1787739063.221549:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23478 00000100:00100000:0.0:1787739063.221559:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042461c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17612:x1874579652483200:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739063.221584:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739063.221605:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad496 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x17c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09505e expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.221665:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad49d lrc: 3/1,0 mode: --/PR res: [0x240000402:0x17c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.221679:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad49d lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.221691:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad49d lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739063.221786:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17c:0x0]: returning size 4096 00000004:00200000:0.0:1787739063.221792:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739063.221830:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad496 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09505e expref: 11 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.221848:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad49d lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x17c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09505e expref: 10 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.221857:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17c:0x0].0x0 (ffffa07307064e00) 00010000:00010000:0.0:1787739063.221860:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739063.221864:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609bc00, rc 0) 00000100:00100000:0.0:1787739063.221924:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042461c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17612:x1874579652483200:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 366us (1428us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.221932:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23478 00000100:00100000:0.0:1787739063.232821:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739063.233199:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652483328 00000100:00100000:3.0:1787739063.233209:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652483328 00000100:00100000:3.0:1787739063.233274:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23479 00000100:00100000:3.0:1787739063.233285:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17612:x1874579652483328:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739063.233314:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739063.233336:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad4a4 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095065 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.233397:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad4ab lrc: 3/1,0 mode: --/PR res: [0x240000400:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.233429:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad4ab lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.233442:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad4ab lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739063.233494:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xc5:0x0]: returning size 4096 00000004:00200000:3.0:1787739063.233500:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739063.233547:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad4a4 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095065 expref: 12 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.233566:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad4ab lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095065 expref: 11 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.233574:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc5:0x0].0x0 (ffffa07307064700) 00010000:00010000:3.0:1787739063.233577:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739063.233581:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aac00, rc 0) 00000100:00100000:3.0:1787739063.233637:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17612:x1874579652483328:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 353us (823us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.233644:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23479 00000100:00100000:0.0:1787739063.246352:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739063.258557:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652483456 00000100:00100000:1.0:1787739063.258573:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652483456 00000100:00100000:0.0:1787739063.258712:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23480 00000100:00100000:0.0:1787739063.258724:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17612:x1874579652483456:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739063.258745:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739063.258766:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad4b2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09506c expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.258823:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad4b9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.258839:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad473 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad46c expref: 5 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.258849:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad4b9 lrc: 4/1,0 mode: --/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.258883:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad473 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ad46c expref: 5 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.258898:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d2940 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6525:1874579665688832:0@lo:104: 00000100:00100000:0.0:1787739063.258972:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.259032:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068489c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.259062:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665688832 00000100:00100000:3.0:1787739063.259068:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665688832 00000100:00100000:3.0:1787739063.259110:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1166 00000100:00100000:3.0:1787739063.259118:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d2d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6525:x1874579665688832:12345-0@lo:104: 00010000:00010000:3.0:1787739063.259131:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ad46c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad473 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.259200:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739063.259205:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d2d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6525:x1874579665688832:12345-0@lo:104: Request processed in 89us (234us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.259212:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1166 00010000:00010000:1.0:1787739063.259243:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739063.259255:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ad46c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad473 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.259264:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ac00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739063.259270:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ad46c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xc5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ad473 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739063.259279:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ad46c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xc5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ad473 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.259305:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ad46c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xc5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad473 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.259316:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739063.259346:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ad46c lrc: 1/0,0 mode: --/PW res: [0x200000401:0xc5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad473 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.259353:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609ac00/0x7ee4453e4a1ad46c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xc5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad473 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.259372:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739063.259374:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739063.259376:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739063.259387:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304246940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665688960:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739063.259403:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d2940 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6525:1874579665688832:0@lo:104: 00010000:00010000:1.0:1787739063.259456:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad4b9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.259466:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.259506:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665688960 00000100:00100000:0.0:1787739063.259521:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665688960 00010000:00010000:0.0:1787739063.259539:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad473 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ad46c expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.259569:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1524 00000100:00100000:0.0:1787739063.259584:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+6:3652:x1874579665688960:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739063.259593:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00080000:1.0:1787739063.261539:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 377 batchid updated 00000020:00080000:3.0:1787739063.262138:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 377 00000004:00080000:3.0:1787739063.262201:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07410b4d300 with version 716 00000020:00080000:3.0:1787739063.262296:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 377 cancel update log [0x240000401:0x3:0x0].373: rc = 0 00000020:00080000:3.0:1787739063.262366:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 377 cancel update log [0x200000400:0x3:0x0].374: rc = 0 00000020:00080000:3.0:1787739063.262375:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 377 00000004:00080000:2.0:1787739063.262660:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 716 rpc_version 716 00000020:00100000:2.0:1787739063.262692:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739063.262703:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306392580 x1874579665689216/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739063.262725:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665689216:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739063.262788:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739063.262825:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ea80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.262875:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665689216 00000100:00100000:3.0:1787739063.262882:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665689216 00000100:00100000:3.0:1787739063.262933:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3214 00000100:00100000:3.0:1787739063.262954:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306391e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665689216:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739063.263094:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306391e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665689216:12345-0@lo:1000:osp_up1-0.0 Request processed in 142us (306us total) trans 4294968392 rc 0/0 00000100:00100000:3.0:1787739063.263101:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3214 00000100:00100000:2.0:1787739063.263326:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665689216:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787739063.264856:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad473 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ad46c expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.264877:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc5:0x0].0x0 (ffffa07433235600) 00010000:00010000:0.0:1787739063.264883:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad4b9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.264892:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad4b9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.264900:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad4b9 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.264944:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739063.264978:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad4b9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.264989:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad4b9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.265032:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+5:3652:x1874579665688960:12345-0@lo:103:ldlm_bl.0 Request processed in 5447us (5567us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.265042:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1524 00000004:00200000:2.0:1787739063.265058:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xc5:0x0]: returning size 4096 00000004:00200000:2.0:1787739063.265065:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739063.265112:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad4b2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09506c expref: 16 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.265130:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad4b9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09506c expref: 15 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.265137:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc5:0x0].0x0 (ffffa07433235600) 00010000:00010000:2.0:1787739063.265139:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739063.265141:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a400, rc 0) 00000100:00100000:2.0:1787739063.265194:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17612:x1874579652483456:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 6473us (18850us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.265202:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23480 00000100:00100000:3.0:1787739063.265326:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304246940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665688960:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739063.290566:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739063.292261:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652483584 00000100:00100000:0.0:1787739063.292275:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652483584 00000100:00100000:0.0:1787739063.292348:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1327 00000100:00100000:0.0:1787739063.292359:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2408:x1874579652483584:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:0.0:1787739063.292375:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739063.292400:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad48f lrc: 0/0,0 mode: --/PR res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095057 expref: 15 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.292445:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17c:0x0].0x0 (ffffa07307064200) 00010000:00010000:0.0:1787739063.292457:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739063.292517:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2408:x1874579652483584:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 160us (1959us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.292525:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1327 00000100:00100000:0.0:1787739063.299386:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739063.302558:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652483840 00000100:00100000:2.0:1787739063.302573:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652483840 00000100:00100000:2.0:1787739063.302637:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23481 00000100:00100000:2.0:1787739063.302647:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17612:x1874579652483840:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:2.0:1787739063.302872:0:13793:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739063.302900:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad49d lrc: 0/0,0 mode: --/PR res: [0x240000402:0x17c:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09505e expref: 11 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.302919:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17c:0x0].0x0 (ffffa07307064e00) 00010000:00010000:2.0:1787739063.302924:0:13793:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739063.302973:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1ad4c0 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.302982:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1ad4c0 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.302992:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1ad4c0 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739063.304056:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306391680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665689344:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787739063.304335:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.304387:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ea80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739063.304564:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665689344 00000100:00100000:2.0:1787739063.304570:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665689344 00000100:00100000:2.0:1787739063.304618:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3215 00000100:00100000:2.0:1787739063.304627:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+5:13793:x1874579665689344:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739063.304642:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739063.304671:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad4c7 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad4c0 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.304699:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad4c7 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad4c0 expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.304714:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad4c7 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad4c0 expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.304723:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403a6fd00) 00010000:00010000:2.0:1787739063.304726:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739063.304730:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633cc00, rc 0) 00000100:00100000:2.0:1787739063.304777:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:13793:x1874579665689344:12345-0@lo:101:mdt00_004.0 Request processed in 151us (443us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.304784:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3215 00000100:00100000:0.0:1787739063.305030:0:13793:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983403 gen 2 00000004:00080000:0.0:1787739063.305036:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983398 committed transno 4294983403 00000020:00080000:0.0:1787739063.305042:0:13793:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07303c6ed80 refcount 2 committed 0 result 0 batchid 368 00000020:00080000:0.0:1787739063.305046:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07404f86400 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07303c6e9c0 00000020:00080000:0.0:1787739063.305049:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].362 00000020:00080000:0.0:1787739063.305052:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07404f86f00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07303c9b000 00000020:00080000:0.0:1787739063.305055:0:13793:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].361 00000004:00080000:0.0:1787739063.305083:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983400 committed transno 4294983403 00000004:00080000:0.0:1787739063.305093:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983401 committed transno 4294983403 00000004:00080000:0.0:1787739063.305101:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983403 committed transno 4294983403 00000100:00100000:0.0:1787739063.305119:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306391680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665689344:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739063.305136:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1ad4c0 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad4c7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739063.305148:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 368 committed batchid 367 00010000:00010000:0.0:1787739063.305148:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1ad4c0 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad4c7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739063.305168:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739063.305202:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ad4ce lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739063.305208:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 367 00010000:00010000:0.0:1787739063.305288:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ad4ce lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.305297:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ad4ce lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.305316:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f77840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665689472:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739063.305352:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.305389:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068483c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739063.305555:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665689472 00000100:00100000:2.0:1787739063.305560:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665689472 00000100:00100000:2.0:1787739063.305591:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3216 00000100:00100000:2.0:1787739063.305598:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060061c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:13793:x1874579665689472:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739063.305607:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739063.305619:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad4d5 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad4ce expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.305649:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad4d5 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad4ce expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.306211:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad4d5 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad4ce expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.306225:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd1c5f96a (ffffa07403a6f300) 00010000:00010000:2.0:1787739063.306228:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739063.306231:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d600, rc 0) 00000100:00100000:2.0:1787739063.306395:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060061c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665689472:12345-0@lo:101:mdt00_004.0 Request processed in 797us (1041us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.306403:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3216 00000100:00100000:3.0:1787739063.315117:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f77840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665689472:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739063.315146:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ad4ce lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad4d5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.315160:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ad4ce lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad4d5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739063.315182:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:3.0:1787739063.315325:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f0780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665689600:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739063.315385:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.315626:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.324164:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665689600 00000100:00100000:0.0:1787739063.324179:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665689600 00000100:00100000:0.0:1787739063.326107:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3217 00000100:00100000:0.0:1787739063.326129:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665689600:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739063.326250:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665689600:12345-0@lo:1000:mdt00_004.0 Request processed in 131us (10864us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.326258:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3217 00000100:00100000:3.0:1787739063.326362:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f0780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665689600:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739063.326450:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f0780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665689728:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739063.326486:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.335074:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739063.335339:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665689728 00000100:00100000:1.0:1787739063.335353:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665689728 00000100:00100000:1.0:1787739063.335394:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3218 00000100:00100000:1.0:1787739063.335402:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665689728:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739063.335531:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665689728:12345-0@lo:1000:mdt00_004.0 Request processed in 129us (9043us total) trans 0 rc 1/1 00000100:00100000:1.0:1787739063.335542:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3218 00000100:00100000:3.0:1787739063.335984:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f0780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665689728:0@lo:1000:mdt00_004.0 00010000:00010000:3.0:1787739063.336092:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ad4dc lrc: 3/0,1 mode: --/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.336102:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ad4dc lrc: 3/0,1 mode: --/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.336111:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ad4dc lrc: 3/0,1 mode: --/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739063.336127:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f0780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665689856:0@lo:101:mdt00_004.0 00000100:00100000:3.0:1787739063.336161:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.336201:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.336250:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665689856 00000100:00100000:0.0:1787739063.336255:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665689856 00000100:00100000:0.0:1787739063.336294:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3219 00000100:00100000:0.0:1787739063.336301:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665689856:12345-0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739063.336311:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739063.336326:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad4e3 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad4dc expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.336348:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad4e3 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad4dc expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.336361:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad4e3 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad4dc expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.336369:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17c:0x0].0x0 (ffffa07307064200) 00010000:00010000:0.0:1787739063.336373:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739063.336377:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a000, rc 0) 00000100:00100000:0.0:1787739063.336488:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665689856:12345-0@lo:101:mdt00_004.0 Request processed in 187us (326us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.336494:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3219 00000100:00100000:3.0:1787739063.336538:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f0780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665689856:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739063.336551:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ad4dc lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad4e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.336561:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ad4dc lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad4e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.336583:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad4ea lrc: 3/0,1 mode: --/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.336591:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad4ea lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.336599:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad4ea lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.336614:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad4f1 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.336625:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad4ab lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095065 expref: 10 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.336634:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad4f1 lrc: 4/0,1 mode: --/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.336662:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad4ab lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095065 expref: 10 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.336676:0:13793:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad4ab lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095065 expref: 10 pid: 6524 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739063.336861:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f0780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665689984:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739063.336928:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.346671:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739063.346758:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652483968 00000100:00100000:1.0:1787739063.346767:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652483968 00010000:00010000:1.0:1787739063.346804:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad4ab lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095065 expref: 11 pid: 6524 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739063.346838:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1525 00000100:00100000:1.0:1787739063.346849:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f3840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2409:x1874579652483968:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739063.346864:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739063.346870:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad4ab lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095065 expref: 11 pid: 6524 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.346884:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad4ab lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095065 expref: 11 pid: 6524 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.346905:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc5:0x0].0x0 (ffffa07307064700) 00010000:00010000:1.0:1787739063.346911:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad4f1 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.346919:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad4f1 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.346922:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad4ab lrc: 0/0,0 mode: --/PR res: [0x240000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095065 expref: 11 pid: 6524 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.346927:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad4f1 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.346943:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f0780 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665689984:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739063.346964:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad4f1 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.347024:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739063.347074:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f3840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2409:x1874579652483968:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 225us (411us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739063.347081:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1525 00010000:00010000:0.0:1787739063.347184:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad4f1 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.347193:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad4f1 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.347235:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad4f8 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.347243:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad4f8 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.347254:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad4f8 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.347273:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561cf00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665690112:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739063.347312:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.347349:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306848d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.347393:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665690112 00000100:00100000:3.0:1787739063.347398:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665690112 00000100:00100000:3.0:1787739063.347509:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3220 00000100:00100000:3.0:1787739063.347519:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665690112:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739063.347535:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739063.347550:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad4ff lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad4f8 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.347570:0:6528:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad4b9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09506c expref: 13 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.347579:0:6528:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad4ff lrc: 4/0,0 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad4f8 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.347599:0:6528:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad4b9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09506c expref: 13 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.347613:0:6528:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad4b9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09506c expref: 13 pid: 6525 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739063.347625:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f0780 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6528:1874579665690240:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739063.347680:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.357315:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f0780 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6528:1874579665690240:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739063.357351:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad4ff lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad4f8 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.357364:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc5:0x0].0x0 (ffffa07433235600) 00010000:00010000:0.0:1787739063.357368:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739063.357372:0:6528:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad4ff lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad4f8 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.357382:0:6528:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad4ff lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad4f8 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.357389:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab400, rc 0) 00000100:00100000:0.0:1787739063.359985:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665690112:12345-0@lo:101:mdt00_004.0 Request processed in 12459us (12663us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.359999:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3220 00000100:00100000:3.0:1787739063.363869:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561cf00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665690112:0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739063.363897:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad4f8 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad4ff expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.363910:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad4f8 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad4ff expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.364592:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739063.365199:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652484096 00000100:00100000:3.0:1787739063.365208:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652484096 00010000:00010000:3.0:1787739063.365247:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad4b9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09506c expref: 14 pid: 6525 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739063.365285:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1526 00000100:00100000:3.0:1787739063.365295:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156efc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652484096:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739063.365312:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739063.365319:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad4b9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09506c expref: 14 pid: 6525 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.365337:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad4b9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09506c expref: 14 pid: 6525 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.365361:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad4b9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09506c expref: 14 pid: 6525 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.365374:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc5:0x0].0x0 (ffffa07433235600) 00010000:00010000:3.0:1787739063.365380:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad4ff lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad4f8 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.365391:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad4ff lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad4f8 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.365400:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad4ff lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad4f8 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.365461:0:6509:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad4ff lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad4f8 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739063.365475:0:6509:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f0780 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665690368:0@lo:105: 00000100:00100000:3.0:1787739063.365528:0:6509:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.365573:0:6509:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062413c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.365593:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665690368 00000100:00100000:3.0:1787739063.365597:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665690368 00000100:00100000:3.0:1787739063.365621:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1167 00000100:00100000:3.0:1787739063.365628:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f77840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665690368:12345-0@lo:105: 00010000:00010000:3.0:1787739063.365640:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad4f8 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad4ff expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.365650:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad4f8 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad4ff expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.365672:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad4f8 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad4ff expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.365680:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad4f8 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad4ff expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.365687:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad4f8 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad4ff expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.366922:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad4f8 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ad4ff expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.366949:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609be00) 00000100:00100000:3.0:1787739063.367136:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f77840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665690368:12345-0@lo:105: Request processed in 1384us (1482us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.367147:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1167 00000100:00100000:3.0:1787739063.367200:0:6509:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f0780 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665690368:0@lo:105: 00010000:00010000:3.0:1787739063.367215:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739063.367254:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156efc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:2409:x1874579652484096:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 1960us (2666us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.367262:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1526 00010000:00010000:3.0:1787739063.367997:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad4f8 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad4ff expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.368010:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad4f8 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad4ff expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739063.370446:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561cf00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665690496:0@lo:403:mdt00_004.0 00000100:00100000:3.0:1787739063.370525:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.370575:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062413c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739063.371066:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665690496 00000100:00100000:1.0:1787739063.371074:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665690496 00000100:00100000:1.0:1787739063.371131:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3221 00000100:00100000:1.0:1787739063.371140:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665690496:12345-0@lo:403:mdt00_004.0 00000100:00100000:1.0:1787739063.372830:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665690496:12345-0@lo:403:mdt00_004.0 Request processed in 1687us (2301us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739063.372847:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3221 00000100:00100000:3.0:1787739063.372973:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561cf00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665690496:0@lo:403:mdt00_004.0 00080000:02000000:0.0:1787739063.374250:0:13793:0:(osd_handler.c:2218:osd_trans_start()) *** cfs_fail_loc=19a, val=0*** 00000004:00080000:0.0:1787739063.374268:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07306848180 with version 0 00000020:00080000:0.0:1787739063.374274:0:13793:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07306848600 refcount 3 committed 0 result -5 batchid 370 00000020:00080000:0.0:1787739063.374278:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2400 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result -5 sub_th ffffa07306848180 00000020:00080000:0.0:1787739063.374281:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2000 obd lustre-MDT0001-osd committed 1 started 0 stopped 1 result 0 sub_th ffffa073040e9000 00010000:00010000:0.0:1787739063.374316:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad4f1 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.374336:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad4f1 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.374344:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad4f1 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.374350:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aba00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739063.374355:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad4f1 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.374363:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad4f1 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.374378:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc5:0x0].0x0 (ffffa07307064700) 00010000:00010000:0.0:1787739063.374381:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739063.374386:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad4f1 lrc: 2/0,0 mode: --/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.374391:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ad4f1 lrc: 0/0,0 mode: --/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.374405:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad4f8 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad4ff expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739063.374453:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 367 00010000:00010000:0.0:1787739063.374475:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad4f8 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad4ff expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.374482:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad4f8 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad4ff expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.374619:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739063.374627:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xc5:0x0] 00010000:00010000:0.0:1787739063.374636:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad4ea lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.374643:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad4ea lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.374648:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad4ea lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.374654:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aae00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739063.374657:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad4ea lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.374787:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad4ea lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.374797:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17c:0x0].0x0 (ffffa07307064e00) 00010000:00010000:0.0:1787739063.374800:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739063.374803:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad4ea lrc: 2/0,0 mode: --/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.374809:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad4ea lrc: 0/0,0 mode: --/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.374820:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ad4dc lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad4e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.374828:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ad4dc lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad4e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.374835:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ad4dc lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad4e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.379167:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:0.0:1787739063.379196:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739063.379206:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787739063.379215:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad4f8 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad4ff expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.379228:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609be00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739063.379234:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad4f8 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad4ff expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739063.379249:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad4f8 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad4ff expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.379289:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad4f8 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad4ff expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.379297:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739063.379318:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad4f8 lrc: 1/0,0 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad4ff expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.379325:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad4f8 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad4ff expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.379342:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739063.379346:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ad4dc lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad4e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.379352:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739063.379356:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ad4dc lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad4e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739063.379362:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ad4dc lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad4e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.379378:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ad4dc lrc: 2/0,0 mode: --/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad4e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.379384:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739063.379399:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ad4dc lrc: 1/0,0 mode: --/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad4e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.379405:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069abe00/0x7ee4453e4a1ad4dc lrc: 0/0,0 mode: --/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad4e3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.379437:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739063.379439:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739063.379441:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739063.379458:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ec3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665690624:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739063.379526:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.379573:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665690752:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739063.379596:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.379629:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665690624 00000100:00100000:0.0:1787739063.379635:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665690624 00000100:00100000:0.0:1787739063.379678:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665690752 00000100:00100000:0.0:1787739063.379681:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665690752 00000100:00100000:0.0:1787739063.379701:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1328 00000100:00100000:0.0:1787739063.379710:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba5c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+10:3653:x1874579665690624:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739063.379723:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739063.379740:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad4ff lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad4f8 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.379754:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc5:0x0].0x0 (ffffa07433235600) 00010000:00010000:0.0:1787739063.379763:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739063.379795:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba5c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+9:3653:x1874579665690624:12345-0@lo:103:ldlm_bl.0 Request processed in 87us (269us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.379803:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1328 00000100:00100000:0.0:1787739063.379843:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1329 00000100:00100000:0.0:1787739063.379850:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415700b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+8:3650:x1874579665690752:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739063.379857:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739063.379867:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad4e3 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad4dc expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.379877:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17c:0x0].0x0 (ffffa07307064200) 00010000:00010000:0.0:1787739063.379883:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739063.379907:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415700b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+7:3650:x1874579665690752:12345-0@lo:103:ldlm_bl.0 Request processed in 58us (310us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.379913:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1329 00000100:00100000:0.0:1787739063.379949:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665690752:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739063.379970:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ec3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665690624:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739063.379984:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1ad4c0 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad4c7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.379993:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1ad4c0 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad4c7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.380001:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1ad4c0 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad4c7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.380031:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739063.380038:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739063.380045:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739063.380050:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1ad4c0 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad4c7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.380062:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633de00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739063.380066:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1ad4c0 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad4c7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739063.380073:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1ad4c0 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad4c7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.380092:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1ad4c0 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad4c7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.380099:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739063.380111:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1ad4c0 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad4c7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.380118:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633de00/0x7ee4453e4a1ad4c0 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad4c7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.380129:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739063.380131:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739063.380132:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739063.380141:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665690880:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739063.380165:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787739063.380192:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ad4ce lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad4d5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.380200:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ad4ce lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad4d5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.380212:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ad4ce lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad4d5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.380225:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1787739063.380287:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17612:x1874579652483840:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 77639us (80905us total) trans 0 rc -5/-5 00000100:00100000:0.0:1787739063.380295:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23481 00000100:00100000:0.0:1787739063.380494:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665690880 00000100:00100000:0.0:1787739063.380499:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665690880 00000100:00100000:0.0:1787739063.380528:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1330 00000100:00100000:0.0:1787739063.380534:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+7:3651:x1874579665690880:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739063.380542:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739063.380553:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad4c7 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad4c0 expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.380564:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403a6fd00) 00010000:00010000:0.0:1787739063.380571:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739063.380599:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+6:3651:x1874579665690880:12345-0@lo:103:ldlm_bl.0 Request processed in 65us (433us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.380605:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1330 00000100:00100000:0.0:1787739063.380630:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665690880:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739063.381116:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739063.381124:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ad4ce lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad4d5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.381133:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609aa00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739063.381137:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ad4ce lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad4d5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739063.381145:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ad4ce lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad4d5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.381170:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ad4ce lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad4d5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.381176:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739063.381193:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ad4ce lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad4d5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.381200:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609aa00/0x7ee4453e4a1ad4ce lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad4d5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.381215:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739063.381216:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739063.381218:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739063.381228:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304247480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665691008:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739063.381259:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787739063.381535:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739063.381816:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665691008 00000100:00100000:3.0:1787739063.381821:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665691008 00000100:00100000:3.0:1787739063.381870:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1331 00000100:00100000:3.0:1787739063.381878:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304247c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+6:3652:x1874579665691008:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739063.381889:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739063.381904:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad4d5 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad4ce expref: 6 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.381918:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd1c5f96a (ffffa07403a6f300) 00010000:00010000:3.0:1787739063.381926:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739063.381977:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304247c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+5:3652:x1874579665691008:12345-0@lo:103:ldlm_bl.0 Request processed in 100us (717us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.381984:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1331 00000100:00100000:1.0:1787739063.382840:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304247480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665691008:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739063.410446:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739063.410476:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652484224 00000100:00100000:3.0:1787739063.410486:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652484224 00000100:00100000:3.0:1787739063.410544:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23482 00000100:00100000:3.0:1787739063.410555:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304247480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17616:x1874579652484224:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739063.410582:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739063.410605:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad506 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095073 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.410670:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad50d lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.410684:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad50d lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.410695:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad50d lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.410708:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad514 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.410715:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad514 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.410721:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad514 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.410760:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad50d lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.410768:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad50d lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.410774:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad50d lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.410780:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad50d lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.410787:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739063.410791:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad50d lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.410798:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad50d lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.410810:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730608bb00) 00010000:00010000:3.0:1787739063.410813:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739063.410817:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad50d lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.410823:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad50d lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.410834:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad514 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.410840:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad514 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.410845:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad514 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.410849:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739063.410852:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad514 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.410857:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad514 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.410869:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x49ac0f22 (ffffa0730608b300) 00010000:00010000:3.0:1787739063.410872:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739063.410875:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad514 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.410880:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad514 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.410908:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad506 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095073 expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.410918:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730608bb00) 00010000:00010000:3.0:1787739063.410921:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739063.410924:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad506 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095073 expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.410935:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa200, rc 0) 00000100:00100000:3.0:1787739063.411061:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304247480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17616:x1874579652484224:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 509us (649us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739063.411068:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23482 00000100:00100000:3.0:1787739063.417783:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739063.417866:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652484352 00000100:00100000:2.0:1787739063.417876:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652484352 00000100:00100000:2.0:1787739063.417934:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23483 00000100:00100000:2.0:1787739063.417942:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17616:x1874579652484352:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739063.417961:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739063.417997:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ad51b lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09507a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.418072:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad522 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.418086:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad522 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.418097:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad522 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.418110:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad529 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.418117:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad529 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.418124:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad529 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.418164:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad522 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.418171:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad522 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.418176:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad522 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.418182:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad522 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.418188:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739063.418196:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad522 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.418202:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad522 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.418213:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:2.0:1787739063.418216:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739063.418220:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad522 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.418225:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad522 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.418237:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad529 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.418243:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad529 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.418248:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad529 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.418252:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739063.418255:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad529 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.418261:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad529 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.418267:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x49ac0f22 (ffffa07403a6f000) 00010000:00010000:2.0:1787739063.418270:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739063.418272:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad529 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.418277:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad529 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.418305:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ad51b lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09507a expref: 14 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.418315:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:2.0:1787739063.418317:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739063.418320:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ad51b lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09507a expref: 14 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.418330:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633da00, rc 0) 00000100:00100000:2.0:1787739063.419040:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17616:x1874579652484352:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 1093us (1260us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739063.419049:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23483 00000100:00100000:1.0:1787739063.441547:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739063.442961:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652484480 00000100:00100000:3.0:1787739063.442971:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652484480 00000100:00100000:3.0:1787739063.443037:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23484 00000100:00100000:3.0:1787739063.443048:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304246940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17616:x1874579652484480:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739063.443068:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739063.443089:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad530 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095081 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.443145:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad537 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.443158:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad537 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.443168:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad537 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739063.443209:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:3.0:1787739063.443215:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739063.443248:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad530 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095081 expref: 15 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.443263:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad537 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095081 expref: 14 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.443272:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730608bb00) 00010000:00010000:3.0:1787739063.443275:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739063.443278:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab400, rc 0) 00000100:00100000:3.0:1787739063.443698:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304246940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17616:x1874579652484480:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 651us (2158us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.443707:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23484 00000100:00100000:1.0:1787739063.461125:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739063.461198:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652484736 00000100:00100000:3.0:1787739063.461208:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652484736 00000100:00100000:3.0:1787739063.461271:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23485 00000100:00100000:3.0:1787739063.461282:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304246d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17616:x1874579652484736:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:3.0:1787739063.461340:0:6523:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739063.461365:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad537 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095081 expref: 14 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.461384:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730608bb00) 00010000:00010000:3.0:1787739063.461393:0:6523:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739063.461472:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad53e lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.461486:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad53e lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.461496:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad53e lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.461508:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad545 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.461515:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad545 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.461521:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad545 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1787739063.462575:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].376, 2168 off575552 00000040:00080000:3.0:1787739063.462660:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].376, 2368 off624960 00000004:00080000:3.0:1787739063.462666:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa073062413c0 with version 717 00000004:00080000:2.0:1787739063.464055:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 717 rpc_version 717 00000020:00100000:2.0:1787739063.464101:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739063.464113:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306393c00 x1874579665691264/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739063.464141:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306393c00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665691264:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739063.464213:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739063.464265:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e840 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.464323:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665691264 00000100:00100000:0.0:1787739063.464330:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665691264 00000100:00100000:0.0:1787739063.464385:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3222 00000100:00100000:0.0:1787739063.464394:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306390b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665691264:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739063.465483:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306390b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665691264:12345-0@lo:1000:osp_up1-0.0 Request processed in 1087us (1267us total) trans 4294968393 rc 0/0 00000100:00100000:0.0:1787739063.465493:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3222 00000100:00100000:2.0:1787739063.466673:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306393c00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665691264:0@lo:1000:osp_up1-0.0 00010000:00010000:3.0:1787739063.466968:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad54c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x151c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.466984:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad54c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x151c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.466995:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad54c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x151c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.467025:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad553 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xc8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.467033:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad553 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.467040:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad553 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.467073:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad55a lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.467081:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad55a lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.467091:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad55a lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739063.467110:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f1680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665691392:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739063.467163:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.467220:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241180 going to sleep for 11 seconds 00000100:00100000:2.0:1787739063.467400:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665691392 00000100:00100000:2.0:1787739063.467407:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665691392 00000100:00100000:2.0:1787739063.467476:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3223 00000100:00100000:2.0:1787739063.467486:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6523:x1874579665691392:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739063.467502:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739063.467518:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad561 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xc8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad55a expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.467540:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad561 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad55a expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.467552:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad561 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad55a expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.467560:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc8:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:2.0:1787739063.467563:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739063.467566:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d200, rc 0) 00000100:00100000:2.0:1787739063.467603:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:6523:x1874579665691392:12345-0@lo:101:mdt00_000.0 Request processed in 118us (439us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.467609:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3223 00000100:00100000:3.0:1787739063.467713:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f1680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665691392:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739063.467726:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad55a lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad561 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.467736:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad55a lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad561 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739063.467760:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000053ef5a14 reply state 0000000088526b90 transno 4294983404 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad553 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.467772:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad553 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.467778:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc8:0x0].0x0 (ffffa0730608b400) 00010000:00010000:3.0:1787739063.467781:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739063.467784:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787739063.467790:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad55a lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad561 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.467797:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad55a lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad561 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739063.467807:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xc8:0x0] 00000004:00010000:3.0:1787739063.467813:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000053ef5a14 reply state 0000000088526b90 transno 4294983404 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad54c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x151c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.467821:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad54c lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x151c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.467837:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x151c:0x0].0x0 (ffffa0730608b500) 00010000:00010000:3.0:1787739063.467840:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739063.467842:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:3.0:1787739063.467847:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x151c:0x0]: returning size 4096 00010000:00010000:3.0:1787739063.467856:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad53e lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.467863:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad53e lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.467869:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad53e lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.467876:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad53e lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.467881:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739063.467885:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad53e lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.467892:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad53e lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.467901:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730608b300) 00010000:00010000:3.0:1787739063.467903:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739063.467907:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad53e lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.467913:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad53e lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739063.467930:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000053ef5a14 reply state 0000000088526b90 transno 4294983404 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad545 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.467940:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad545 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.467945:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x49ac0f22 (ffffa0730608b900) 00010000:00010000:3.0:1787739063.467948:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739063.467950:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 2 00000100:00100000:3.0:1787739063.468031:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304246d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17616:x1874579652484736:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 6749us (6914us total) trans 4294983404 rc 0/0 00000100:00100000:3.0:1787739063.468039:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23485 00000100:00100000:2.0:1787739063.499823:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739063.501641:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652484992 00000100:00100000:3.0:1787739063.501655:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652484992 00000100:00100000:3.0:1787739063.501720:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23486 00000100:00100000:3.0:1787739063.501732:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306390780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17623:x1874579652484992:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:3.0:1787739063.501911:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306390780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17623:x1874579652484992:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 182us (2093us total) trans 0 rc -17/-17 00000100:00100000:3.0:1787739063.501919:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23486 00000100:00100000:0.0:1787739063.536430:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739063.536473:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652485120 00000100:00100000:0.0:1787739063.536481:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652485120 00000100:00100000:0.0:1787739063.536537:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23487 00000100:00100000:0.0:1787739063.536548:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17624:x1874579652485120:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739063.536569:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739063.536593:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad568 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095088 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.536655:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad56f lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.536669:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad56f lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.536680:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad56f lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.536692:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad576 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.536700:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad576 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.536707:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad576 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.536782:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad57d lrc: 3/1,0 mode: --/PR res: [0x200000402:0x151c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.536790:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad57d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x151c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.536797:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad57d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x151c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739063.536869:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x151c:0x0]: returning size 4096 00000004:00200000:0.0:1787739063.536874:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:0.0:1787739063.536892:0:13793:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad57d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x151c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.536905:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad56f lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.536912:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad56f lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.536918:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad56f lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.536925:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad56f lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.536931:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609bc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739063.536935:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad56f lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.536942:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad56f lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.536953:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07307064200) 00010000:00010000:0.0:1787739063.536956:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739063.536961:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad56f lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.536967:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad56f lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.536993:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad576 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.537000:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad576 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.539190:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad576 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.539199:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739063.539204:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad576 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.539213:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad576 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.539228:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x49ac0f22 (ffffa0730608b900) 00010000:00010000:0.0:1787739063.539231:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739063.539235:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad576 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.539242:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad576 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.539294:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad568 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095088 expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.539313:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad57d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x151c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095088 expref: 16 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.539321:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x151c:0x0].0x0 (ffffa0730608b500) 00010000:00010000:0.0:1787739063.539324:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787739063.539326:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b600, rc 0) 00000100:00100000:0.0:1787739063.539391:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17624:x1874579652485120:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 2844us (3418us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.539399:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23487 00000100:00100000:1.0:1787739063.546600:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739063.546756:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652485248 00000100:00100000:0.0:1787739063.546766:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652485248 00000100:00100000:0.0:1787739063.546827:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23488 00000100:00100000:0.0:1787739063.546836:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042443c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17624:x1874579652485248:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739063.546857:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739063.546876:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad584 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xc8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09508f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.546931:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad58b lrc: 3/1,0 mode: --/PR res: [0x200000400:0xc8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.546943:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad58b lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.546954:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad58b lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739063.547020:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xc8:0x0]: returning size 4096 00000004:00200000:0.0:1787739063.547026:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739063.547059:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad584 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09508f expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.547076:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad58b lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09508f expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.547085:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc8:0x0].0x0 (ffffa0730608b400) 00010000:00010000:0.0:1787739063.547087:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739063.547090:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b800, rc 0) 00000100:00100000:0.0:1787739063.547146:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042443c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17624:x1874579652485248:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 311us (556us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.547153:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23488 00000100:00100000:3.0:1787739063.552677:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739063.552756:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652485376 00000100:00100000:0.0:1787739063.552765:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652485376 00000100:00100000:0.0:1787739063.552823:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23489 00000100:00100000:0.0:1787739063.552833:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17624:x1874579652485376:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739063.552856:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739063.552876:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad592 lrc: 2/0,0 mode: --/CR res: [0x240000401:0xc8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095096 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.552934:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad599 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.552949:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad561 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad55a expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.552963:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad599 lrc: 4/1,0 mode: --/PR res: [0x240000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.553021:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad561 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ad55a expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.553036:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665691520:0@lo:104: 00000100:00100000:0.0:1787739063.553086:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.553130:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068480c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.553152:0:6507:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787739063.553160:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665691520 00000100:00100000:0.0:1787739063.553163:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665691520 00000100:00100000:0.0:1787739063.553190:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1168 00000100:00100000:0.0:1787739063.553197:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073041ed850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665691520:12345-0@lo:104: 00010000:00010000:0.0:1787739063.553209:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad55a lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad561 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.553254:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787739063.553260:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073041ed850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665691520:12345-0@lo:104: Request processed in 63us (173us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.553266:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1168 00010000:00010000:0.0:1787739063.553309:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739063.553315:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad55a lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad561 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.553324:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aba00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739063.553330:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad55a lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xc8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ad561 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739063.553339:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad55a lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xc8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ad561 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.553364:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad55a lrc: 2/0,0 mode: --/PW res: [0x240000401:0xc8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad561 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.553369:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739063.553388:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad55a lrc: 1/0,0 mode: --/PW res: [0x240000401:0xc8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad561 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.553396:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aba00/0x7ee4453e4a1ad55a lrc: 0/0,0 mode: --/PW res: [0x240000401:0xc8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad561 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.553436:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739063.553438:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739063.553440:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739063.553461:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665691520:0@lo:104: 00010000:00010000:0.0:1787739063.553475:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad599 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.554518:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787739063.554538:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665691648:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739063.554588:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.554673:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665691648 00000100:00100000:3.0:1787739063.554680:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665691648 00010000:00010000:3.0:1787739063.554707:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad561 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ad55a expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739063.554742:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1527 00000100:00100000:3.0:1787739063.554749:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415701a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+9:3652:x1874579665691648:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739063.554762:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:3.0:1787739063.557130:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968393 is committed 00000020:00080000:3.0:1787739063.557144:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 368 batchid updated 00000020:00080000:0.0:1787739063.557179:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 368 00000004:00080000:0.0:1787739063.557235:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa073068480c0 with version 693 00000020:00080000:0.0:1787739063.557258:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 368 cancel update log [0x200000401:0x3:0x0].362: rc = 0 00000020:00080000:0.0:1787739063.557323:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 368 cancel update log [0x240000400:0x3:0x0].361: rc = 0 00000020:00080000:0.0:1787739063.557331:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 368 00000001:00080000:3.0:1787739063.558124:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968391 is committed 00000020:00080000:3.0:1787739063.558133:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07410404c00 refcount 2 committed 0 result 0 batchid 369 00000020:00080000:3.0:1787739063.558137:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2600 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07410404f00 00000020:00080000:3.0:1787739063.558140:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].363 00000020:00080000:3.0:1787739063.558143:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2700 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073040e9400 00000020:00080000:3.0:1787739063.558146:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].362 00010000:00010000:3.0:1787739063.558219:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad561 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ad55a expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.558235:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc8:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739063.558241:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad599 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.558250:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad599 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.558258:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad599 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.558283:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739063.558376:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415701a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+5:3652:x1874579665691648:12345-0@lo:103:ldlm_bl.0 Request processed in 3627us (3786us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.558386:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1527 00000020:00080000:3.0:1787739063.558430:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 370 committed batchid 368 00000020:00080000:3.0:1787739063.558467:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 368 00010000:00010000:3.0:1787739063.558481:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad599 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.558488:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad599 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739063.558537:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xc8:0x0]: returning size 4096 00000004:00200000:3.0:1787739063.558542:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739063.558584:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad592 lrc: 0/0,0 mode: --/CR res: [0x240000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095096 expref: 10 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.558600:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad599 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095096 expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.558608:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc8:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739063.558611:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739063.558613:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609ac00, rc 0) 00000100:00100000:3.0:1787739063.558654:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17624:x1874579652485376:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 5822us (5987us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.558778:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665691648:0@lo:103:ldlm_bl.0 00000004:00080000:2.0:1787739063.558804:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 693 rpc_version 693 00000020:00100000:2.0:1787739063.558838:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739063.558847:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073063903c0 x1874579665691904/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739063.558870:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073063903c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665691904:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739063.558929:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739063.558958:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e840 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.558959:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665691904 00000100:00100000:0.0:1787739063.558967:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665691904 00010000:00010000:1.0:1787739063.559178:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad45e lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.559188:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad45e lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.559200:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad45e lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.559206:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739063.559210:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad45e lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.559218:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad45e lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.559230:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17c:0x0].0x0 (ffffa07307064e00) 00010000:00010000:1.0:1787739063.559233:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739063.559237:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad45e lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.559242:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad45e lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.559259:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad465 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.559265:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad465 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.559272:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad465 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.559277:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ba00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739063.559280:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad465 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.559285:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad465 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.559292:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc5:0x0].0x0 (ffffa07307064700) 00010000:00010000:1.0:1787739063.559294:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739063.559296:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad465 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.559302:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad465 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.559703:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3224 00000100:00100000:0.0:1787739063.559717:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306392940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+5:7467:x1874579665691904:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739063.559898:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306392940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:7467:x1874579665691904:12345-0@lo:1000:osp_up0-1.0 Request processed in 184us (968us total) trans 4294983405 rc 0/0 00000100:00100000:0.0:1787739063.559905:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3224 00000100:00100000:2.0:1787739063.560124:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073063903c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665691904:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739063.560294:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23489 00000100:00100000:3.0:1787739063.566742:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739063.566999:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652485632 00000100:00100000:0.0:1787739063.567025:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652485632 00000100:00100000:0.0:1787739063.567080:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23490 00000100:00100000:0.0:1787739063.567091:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17624:x1874579652485632:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:0.0:1787739063.567145:0:6524:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739063.567172:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad57d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x151c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095088 expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.567189:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x151c:0x0].0x0 (ffffa0730608b500) 00010000:00010000:0.0:1787739063.567194:0:6524:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739063.567218:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad5a0 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.567230:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad5a0 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.567239:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad5a0 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.567254:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad5a7 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.567261:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad5a7 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.567268:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad5a7 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.567335:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad5ae lrc: 3/0,1 mode: --/EX res: [0x200000402:0x151c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.567343:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad5ae lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x151c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.567350:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad5ae lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x151c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.567367:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad5b5 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.567376:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad58b lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09508f expref: 16 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.567385:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad5b5 lrc: 4/0,1 mode: --/EX res: [0x200000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.567445:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad58b lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09508f expref: 16 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.567458:0:6524:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad58b lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09508f expref: 16 pid: 6523 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.567473:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665692032:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739063.567543:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306848e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739063.570467:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665692032:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739063.570495:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad5b5 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739063.576567:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739063.576674:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652485760 00000100:00100000:1.0:1787739063.576685:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652485760 00010000:00010000:1.0:1787739063.579524:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad58b lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09508f expref: 17 pid: 6523 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739063.579622:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1528 00000100:00100000:1.0:1787739063.579637:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652485760:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739063.579654:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739063.579670:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad58b lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09508f expref: 17 pid: 6523 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.579690:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad58b lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09508f expref: 17 pid: 6523 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.579713:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad58b lrc: 0/0,0 mode: --/PR res: [0x200000400:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09508f expref: 17 pid: 6523 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.579727:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc8:0x0].0x0 (ffffa0730608b400) 00010000:00010000:1.0:1787739063.579732:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad5b5 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.579741:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad5b5 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.579748:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad5b5 lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.579776:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739063.580460:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:2409:x1874579652485760:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 826us (3898us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739063.580470:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1528 00010000:00010000:0.0:1787739063.583278:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad5b5 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.583296:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad5b5 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.583348:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad5bc lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.583355:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad5bc lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.583367:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad5bc lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.583388:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665692160:0@lo:101:mdt00_001.0 00000100:00100000:0.0:1787739063.583459:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.583494:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306848e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.584187:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665692160 00000100:00100000:3.0:1787739063.584195:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665692160 00000100:00100000:3.0:1787739063.584249:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3225 00000100:00100000:3.0:1787739063.584257:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6524:x1874579665692160:12345-0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787739063.584273:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739063.584289:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad5c3 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad5bc expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.584315:0:6529:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad599 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095096 expref: 7 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.584325:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad5c3 lrc: 4/0,0 mode: --/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad5bc expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.584347:0:6529:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad599 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095096 expref: 7 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.584361:0:6529:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad599 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095096 expref: 7 pid: 7944 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739063.584372:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb4f00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665692288:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739063.584452:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241180 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.595850:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739063.595887:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652485888 00000100:00100000:0.0:1787739063.595894:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652485888 00010000:00010000:0.0:1787739063.595932:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad599 lrc: 5/0,0 mode: PR/PR res: [0x240000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095096 expref: 8 pid: 7944 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.595963:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1529 00000100:00100000:0.0:1787739063.595973:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f77840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2408:x1874579652485888:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739063.595990:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739063.595997:0:6510:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad599 lrc: 5/0,0 mode: PR/PR res: [0x240000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095096 expref: 8 pid: 7944 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.596026:0:6510:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad599 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095096 expref: 8 pid: 7944 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.596050:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc8:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:0.0:1787739063.596056:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad5c3 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad5bc expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.596066:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad5c3 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad5bc expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.596075:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad5c3 lrc: 4/0,0 mode: EX/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad5bc expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.596109:0:6510:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad5c3 lrc: 4/0,0 mode: EX/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad5bc expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.596122:0:6510:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ec3c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6510:1874579665692416:0@lo:105: 00000100:00100000:0.0:1787739063.596166:0:6510:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.596196:0:6510:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306848180 going to sleep for 11 seconds 00010000:00010000:3.0:1787739063.596468:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad599 lrc: 0/0,0 mode: --/PR res: [0x240000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095096 expref: 8 pid: 7944 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739063.596485:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb4f00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665692288:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739063.596510:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad5c3 lrc: 4/0,0 mode: EX/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad5bc expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.596518:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc8:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739063.596521:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739063.596523:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa200, rc 0) 00000100:00100000:3.0:1787739063.596566:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6524:x1874579665692160:12345-0@lo:101:mdt00_001.0 Request processed in 12310us (13108us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.596575:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3225 00000100:00100000:3.0:1787739063.596601:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665692416 00000100:00100000:3.0:1787739063.596604:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665692416 00000100:00100000:3.0:1787739063.596629:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1169 00000100:00100000:3.0:1787739063.596635:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6510:x1874579665692416:12345-0@lo:105: 00010000:00010000:3.0:1787739063.596647:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad5bc lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad5c3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.596656:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad5bc lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad5c3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.596681:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad5bc lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad5c3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.596689:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad5bc lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad5c3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.596696:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad5bc lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad5c3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.596704:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad5bc lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ad5c3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.596715:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609b000) 00000100:00100000:3.0:1787739063.596747:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6510:x1874579665692416:12345-0@lo:105: Request processed in 111us (580us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.596753:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1169 00000100:00100000:3.0:1787739063.596777:0:6510:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ec3c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6510:1874579665692416:0@lo:105: 00010000:00010000:3.0:1787739063.596797:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739063.596830:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f77840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652485888:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 859us (990us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.596837:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1529 00000100:00100000:0.0:1787739063.597639:0:6524:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968393 gen 2 00000004:00080000:0.0:1787739063.597648:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968390 committed transno 4294968393 00000020:00080000:0.0:1787739063.597654:0:6524:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa073039370c0 refcount 2 committed 0 result 0 batchid 378 00000020:00080000:0.0:1787739063.597669:0:6524:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2200 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07303937000 00000020:00080000:0.0:1787739063.597673:0:6524:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].374 00000020:00080000:0.0:1787739063.597676:0:6524:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2900 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073040e9e00 00000020:00080000:0.0:1787739063.597679:0:6524:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].375 00000004:00080000:0.0:1787739063.597704:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968392 committed transno 4294968393 00000004:00080000:0.0:1787739063.597712:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968393 committed transno 4294968393 00000100:00100000:0.0:1787739063.597728:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665692160:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739063.597744:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad5bc lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad5c3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1787739063.597857:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 378 committed batchid 377 00000020:00080000:3.0:1787739063.597916:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 377 00000100:00100000:0.0:1787739063.598686:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665692544:0@lo:403:mdt00_001.0 00000100:00100000:0.0:1787739063.598753:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.598886:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306848e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.599769:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665692544 00000100:00100000:3.0:1787739063.599777:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665692544 00000100:00100000:3.0:1787739063.599836:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3226 00000100:00100000:3.0:1787739063.599845:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6524:x1874579665692544:12345-0@lo:403:mdt00_001.0 00000100:00100000:3.0:1787739063.602174:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6524:x1874579665692544:12345-0@lo:403:mdt00_001.0 Request processed in 2321us (3412us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.602188:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3226 00000100:00100000:1.0:1787739063.602401:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156efc00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665692544:0@lo:403:mdt00_001.0 00000040:00080000:1.0:1787739063.603558:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].377, 952 off576504 00000040:00080000:1.0:1787739063.603871:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].377, 1032 off625992 00000004:00080000:1.0:1787739063.603877:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730623d540 with version 718 00000004:00080000:3.0:1787739063.603966:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 718 rpc_version 718 00000020:00100000:3.0:1787739063.603999:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739063.604025:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07409fb4f00 x1874579665692800/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1992/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739063.604048:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb4f00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665692800:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739063.604107:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.604155:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241180 going to sleep for 11 seconds 00000100:00100000:1.0:1787739063.604270:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665692800 00000100:00100000:1.0:1787739063.604277:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665692800 00000100:00100000:1.0:1787739063.604329:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3227 00000100:00100000:1.0:1787739063.604337:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665692800:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739063.604557:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665692800:12345-0@lo:1000:osp_up1-0.0 Request processed in 220us (449us total) trans 4294968394 rc 0/0 00000100:00100000:1.0:1787739063.604564:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3227 00000100:00100000:3.0:1787739063.604676:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb4f00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665692800:0@lo:1000:osp_up1-0.0 00000004:00200000:1.0:1787739063.604774:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x151c:0x0]: returning size 4096 00010000:00010000:1.0:1787739063.604806:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad5b5 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.604816:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad5b5 lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.604824:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad5b5 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.604830:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739063.604835:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad5b5 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.604842:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad5b5 lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.604857:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc8:0x0].0x0 (ffffa0730608b400) 00010000:00010000:1.0:1787739063.604860:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739063.604865:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad5b5 lrc: 2/0,0 mode: --/EX res: [0x200000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.604872:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad5b5 lrc: 0/0,0 mode: --/EX res: [0x200000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.604886:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad5bc lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad5c3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.604900:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad5bc lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad5c3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.604907:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad5bc lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad5c3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.604930:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739063.604939:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xc8:0x0] 00010000:00010000:1.0:1787739063.604946:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad5ae lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x151c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.604952:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad5ae lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x151c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.604958:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad5ae lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x151c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.604963:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609aa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739063.604966:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad5ae lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x151c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.604972:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad5ae lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x151c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.604979:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x151c:0x0].0x0 (ffffa0730608b500) 00010000:00010000:1.0:1787739063.604982:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739063.604984:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad5ae lrc: 2/0,0 mode: --/EX res: [0x200000402:0x151c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.604990:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad5ae lrc: 0/0,0 mode: --/EX res: [0x200000402:0x151c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.605133:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad5a0 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.605140:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad5a0 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.605145:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad5a0 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.605150:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad5a0 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.605155:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609be00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739063.605158:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad5a0 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.605163:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad5a0 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.605172:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07307064200) 00010000:00010000:1.0:1787739063.605174:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739063.605177:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad5a0 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.605181:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad5a0 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739063.605201:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e61b9676 reply state 00000000bc6b9bc3 transno 4294983406 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad5a7 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.605213:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad5a7 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.605220:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x49ac0f22 (ffffa0730608b900) 00010000:00010000:1.0:1787739063.605222:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739063.605225:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787739063.605286:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17624:x1874579652485632:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 38197us (38549us total) trans 4294983406 rc 0/0 00000100:00100000:1.0:1787739063.605293:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23490 00010000:00010000:3.0:1787739063.606267:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739063.606276:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad5bc lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad5c3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.606284:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739063.606290:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad5bc lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad5c3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739063.606299:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad5bc lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad5c3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.606331:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad5bc lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad5c3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.606338:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739063.606370:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad5bc lrc: 1/0,0 mode: --/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad5c3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.606379:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b000/0x7ee4453e4a1ad5bc lrc: 0/0,0 mode: --/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad5c3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.606397:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739063.606399:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739063.606400:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739063.606409:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787739063.606497:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665692928:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739063.606545:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.606596:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665692928 00000100:00100000:3.0:1787739063.606603:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665692928 00000100:00100000:3.0:1787739063.606644:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1332 00000100:00100000:3.0:1787739063.606651:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba6450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+7:3651:x1874579665692928:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739063.606725:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739063.606744:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad5c3 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad5bc expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.606756:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc8:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739063.606764:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739063.606865:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba6450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+6:3651:x1874579665692928:12345-0@lo:103:ldlm_bl.0 Request processed in 214us (319us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.606872:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1332 00000100:00100000:3.0:1787739063.606934:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb7840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665692928:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739063.622740:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739063.640810:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739063.641080:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652486016 00000100:00100000:3.0:1787739063.641089:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652486016 00000100:00100000:3.0:1787739063.641152:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23491 00000100:00100000:3.0:1787739063.641163:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156efc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17625:x1874579652486016:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739063.641185:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739063.641218:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad5ca lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09509d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.641725:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad5d1 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.641742:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad5d1 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.641755:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad5d1 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.641770:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad5d8 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.641777:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad5d8 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.641784:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad5d8 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.641821:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad5d1 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.641828:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad5d1 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.641834:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad5d1 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.641844:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad5d1 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.641850:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739063.641855:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad5d1 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.641862:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad5d1 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.641875:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2c00) 00010000:00010000:3.0:1787739063.641879:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739063.641883:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad5d1 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.641889:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad5d1 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.641904:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad5d8 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.641909:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad5d8 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.641915:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad5d8 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.641920:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739063.641923:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad5d8 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.641934:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad5d8 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.641942:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x917910c6 (ffffa074331c2e00) 00010000:00010000:3.0:1787739063.641945:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739063.641947:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad5d8 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.641953:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ad5d8 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.643399:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad5ca lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09509d expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.643445:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2c00) 00010000:00010000:3.0:1787739063.643450:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739063.643455:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad5ca lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09509d expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.643472:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa200, rc 0) 00000100:00100000:3.0:1787739063.643546:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156efc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17625:x1874579652486016:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 2384us (2743us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739063.643556:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23491 00000100:00100000:1.0:1787739063.650354:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739063.650477:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652486144 00000100:00100000:3.0:1787739063.650487:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652486144 00000100:00100000:3.0:1787739063.650556:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23492 00000100:00100000:3.0:1787739063.650566:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304247c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17625:x1874579652486144:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739063.650595:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739063.650623:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ad5df lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0950a4 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.650693:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad5e6 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.650706:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad5e6 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.650723:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad5e6 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.650746:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad5ed lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.650753:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad5ed lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.650760:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad5ed lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.650808:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad5e6 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.650820:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad5e6 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.650827:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad5e6 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.650833:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad5e6 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.650846:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739063.650855:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad5e6 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.650862:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad5e6 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.650884:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2300) 00010000:00010000:3.0:1787739063.650887:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739063.650892:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad5e6 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.650902:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad5e6 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.650918:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad5ed lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.650924:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad5ed lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.650941:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad5ed lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.650946:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aaa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739063.650949:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad5ed lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.650954:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad5ed lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.650975:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x917910c6 (ffffa074331c2800) 00010000:00010000:3.0:1787739063.650978:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739063.650980:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad5ed lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.650986:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad5ed lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.651040:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ad5df lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950a4 expref: 18 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.651050:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2300) 00010000:00010000:3.0:1787739063.651052:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739063.651056:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ad5df lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950a4 expref: 18 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.651078:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab600, rc 0) 00000100:00100000:3.0:1787739063.651158:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304247c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17625:x1874579652486144:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 587us (807us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739063.651166:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23492 00000100:00100000:0.0:1787739063.660168:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739063.660470:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652486272 00000100:00100000:1.0:1787739063.660479:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652486272 00000100:00100000:1.0:1787739063.660536:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23493 00000100:00100000:1.0:1787739063.660545:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17625:x1874579652486272:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739063.660564:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739063.660586:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad5f4 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0950ab expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.660642:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ad5fb lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.660667:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ad5fb lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.660679:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ad5fb lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739063.660730:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:1.0:1787739063.660735:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739063.660770:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad5f4 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950ab expref: 19 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.660793:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ad5fb lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950ab expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.660802:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235200) 00010000:00010000:1.0:1787739063.660806:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739063.660809:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c600, rc 0) 00000100:00100000:1.0:1787739063.660865:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17625:x1874579652486272:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 321us (707us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739063.660872:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23493 00000100:00100000:0.0:1787739063.679124:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739063.679216:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652486528 00000100:00100000:3.0:1787739063.679227:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652486528 00000100:00100000:3.0:1787739063.679289:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23494 00000100:00100000:3.0:1787739063.679299:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17625:x1874579652486528:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:3.0:1787739063.679448:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb7840 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665693056:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739063.679502:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.679567:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739063.679711:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665693056 00000100:00100000:1.0:1787739063.679717:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665693056 00000100:00100000:1.0:1787739063.679761:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3228 00000100:00100000:1.0:1787739063.679768:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:6525:x1874579665693056:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739063.679864:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:6525:x1874579665693056:12345-0@lo:1000:mdt00_002.0 Request processed in 96us (361us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739063.679870:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3228 00000100:00100000:0.0:1787739063.679985:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb7840 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665693056:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739063.680044:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ad2c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665693184:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739063.680072:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.680098:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306848780 going to sleep for 11 seconds 00000100:00100000:1.0:1787739063.680121:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665693184 00000100:00100000:1.0:1787739063.680125:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665693184 00000100:00100000:1.0:1787739063.680155:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3229 00000100:00100000:1.0:1787739063.680161:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:6525:x1874579665693184:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739063.680236:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:6525:x1874579665693184:12345-0@lo:1000:mdt00_002.0 Request processed in 75us (163us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739063.680242:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3229 00000100:00100000:1.0:1787739063.680269:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ad2c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665693184:0@lo:1000:mdt00_002.0 00010000:00010000:1.0:1787739063.680315:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ad602 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.680750:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ad602 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.680765:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ad602 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739063.681059:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304245680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665693312:0@lo:101:mdt00_002.0 00000100:00100000:0.0:1787739063.681182:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.681219:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623da80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739063.681278:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665693312 00000100:00100000:1.0:1787739063.681284:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665693312 00000100:00100000:1.0:1787739063.681364:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3230 00000100:00100000:1.0:1787739063.681372:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:6525:x1874579665693312:12345-0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787739063.681386:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739063.681401:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad609 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad602 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.681448:0:12410:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ad5fb lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950ab expref: 17 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.681458:0:12410:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad609 lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad602 expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.681484:0:12410:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ad5fb lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950ab expref: 17 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.681497:0:12410:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ad5fb lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950ab expref: 17 pid: 6524 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739063.681514:0:12410:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304244f00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:12410:1874579665693440:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739063.681578:0:12410:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d600 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.686711:0:12410:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304244f00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:12410:1874579665693440:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739063.686752:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad609 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad602 expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.686769:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235200) 00010000:00010000:0.0:1787739063.686772:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739063.686786:0:12410:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad609 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad602 expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.686796:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1da00, rc 0) 00000100:00100000:0.0:1787739063.686871:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6525:x1874579665693312:12345-0@lo:101:mdt00_002.0 Request processed in 5500us (5689us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.686880:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3230 00000100:00100000:2.0:1787739063.687041:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304245680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665693312:0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787739063.687060:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ad602 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad609 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.687071:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ad602 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad609 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.687607:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739063.687809:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652486656 00000100:00100000:3.0:1787739063.687816:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652486656 00010000:00010000:3.0:1787739063.687847:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ad5fb lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950ab expref: 18 pid: 6524 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739063.687884:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1530 00000100:00100000:3.0:1787739063.687893:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ac780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2409:x1874579652486656:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739063.687907:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739063.687914:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ad5fb lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950ab expref: 18 pid: 6524 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.687935:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ad5fb lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950ab expref: 18 pid: 6524 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.687958:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ad5fb lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950ab expref: 18 pid: 6524 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.687971:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235200) 00010000:00010000:3.0:1787739063.687975:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad609 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad602 expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.687985:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad609 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad602 expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.687993:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad609 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad602 expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.688039:0:6509:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad609 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad602 expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739063.688054:0:6509:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb61c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665693568:0@lo:105: 00000100:00100000:3.0:1787739063.688104:0:6509:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.688161:0:6509:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739063.688212:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665693568 00000100:00100000:2.0:1787739063.688216:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665693568 00000100:00100000:2.0:1787739063.688251:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1170 00000100:00100000:2.0:1787739063.688259:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb70c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665693568:12345-0@lo:105: 00010000:00010000:2.0:1787739063.688270:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ad602 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad609 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.688279:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ad602 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad609 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.688292:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ad602 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad609 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.688299:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ad602 lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad609 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.688306:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ad602 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad609 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.688314:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ad602 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ad609 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.688334:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07441a1cc00) 00000100:00100000:2.0:1787739063.688368:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb70c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665693568:12345-0@lo:105: Request processed in 110us (264us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.688375:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1170 00010000:00010000:2.0:1787739063.688395:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ad602 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad609 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.688403:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ad602 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad609 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739063.688449:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739063.688481:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad610 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.688488:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad610 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.688498:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad610 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739063.688512:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306393c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665693696:0@lo:101:mdt00_002.0 00000100:00100000:2.0:1787739063.688536:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739063.688551:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e840 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.688713:0:6509:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb61c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665693568:0@lo:105: 00010000:00010000:3.0:1787739063.688726:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739063.688771:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ac780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652486656:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 879us (1165us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.688780:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1530 00000100:00100000:3.0:1787739063.688804:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665693696 00000100:00100000:3.0:1787739063.688808:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665693696 00000100:00100000:3.0:1787739063.688833:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3231 00000100:00100000:3.0:1787739063.688840:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306392580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:6525:x1874579665693696:12345-0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739063.688855:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739063.688869:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ad617 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad610 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.688888:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ad617 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad610 expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.688899:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ad617 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad610 expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.688906:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x917910c6 (ffffa074331c2300) 00010000:00010000:3.0:1787739063.688909:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739063.688911:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab600, rc 0) 00000100:00100000:2.0:1787739063.690514:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306393c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665693696:0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787739063.690534:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad610 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad617 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.690545:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad610 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad617 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739063.690562:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:2.0:1787739063.690607:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306393c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665693824:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739063.690643:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.690773:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665693824 00000100:00100000:0.0:1787739063.690780:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665693824 00000100:00100000:0.0:1787739063.690824:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3232 00000100:00100000:0.0:1787739063.690832:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6525:x1874579665693824:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739063.691063:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.691135:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6525:x1874579665693824:12345-0@lo:1000:mdt00_002.0 Request processed in 304us (490us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.691143:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3232 00000100:00100000:2.0:1787739063.691220:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306393c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665693824:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739063.691260:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c3c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665693952:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739063.691282:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739063.691301:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.691348:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665693952 00000100:00100000:0.0:1787739063.691352:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665693952 00000100:00100000:0.0:1787739063.691372:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3233 00000100:00100000:0.0:1787739063.691378:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306393c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6525:x1874579665693952:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739063.691465:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306393c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6525:x1874579665693952:12345-0@lo:1000:mdt00_002.0 Request processed in 87us (182us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739063.691471:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3233 00000100:00100000:2.0:1787739063.691538:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c3c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665693952:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739063.691632:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c3c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665694080:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739063.691652:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739063.691856:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.691880:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665694080 00000100:00100000:0.0:1787739063.691883:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665694080 00000100:00100000:0.0:1787739063.691903:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3234 00000100:00100000:0.0:1787739063.691908:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6525:x1874579665694080:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739063.692132:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6525:x1874579665694080:12345-0@lo:1000:mdt00_002.0 Request processed in 222us (477us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739063.692138:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3234 00000100:00100000:2.0:1787739063.692201:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c3c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665694080:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739063.692252:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562c000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665694208:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739063.692274:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739063.692295:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.692342:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665694208 00000100:00100000:0.0:1787739063.692347:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665694208 00000100:00100000:0.0:1787739063.692364:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3235 00000100:00100000:0.0:1787739063.692369:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6525:x1874579665694208:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739063.692443:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6525:x1874579665694208:12345-0@lo:1000:mdt00_002.0 Request processed in 73us (167us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.692449:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3235 00000100:00100000:2.0:1787739063.692535:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562c000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665694208:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739063.692598:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306392580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6525:x1874579665693696:12345-0@lo:101:mdt00_002.0 Request processed in 1422us (1724us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.692607:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3231 00000100:00100000:2.0:1787739063.693001:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562c000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665694336:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739063.693041:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739063.693101:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.693133:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665694336 00000100:00100000:0.0:1787739063.693138:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665694336 00000100:00100000:0.0:1787739063.693174:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3236 00000100:00100000:0.0:1787739063.693180:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6525:x1874579665694336:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739063.693526:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6525:x1874579665694336:12345-0@lo:1000:mdt00_002.0 Request processed in 344us (482us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.693533:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3236 00000100:00100000:2.0:1787739063.693557:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562c000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665694336:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739063.693596:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562c000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665694464:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739063.693617:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739063.693717:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.693750:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665694464 00000100:00100000:3.0:1787739063.693754:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665694464 00000100:00100000:3.0:1787739063.693883:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3237 00000100:00100000:3.0:1787739063.693890:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6525:x1874579665694464:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739063.694242:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6525:x1874579665694464:12345-0@lo:1000:mdt00_002.0 Request processed in 352us (623us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.694247:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3237 00000100:00100000:2.0:1787739063.694290:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562c000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665694464:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739063.694327:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562c000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665694592:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739063.694351:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739063.694409:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.694497:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665694592 00000100:00100000:0.0:1787739063.694502:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665694592 00000100:00100000:0.0:1787739063.694564:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3238 00000100:00100000:0.0:1787739063.694572:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6525:x1874579665694592:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739063.694615:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6525:x1874579665694592:12345-0@lo:1000:mdt00_002.0 Request processed in 44us (263us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.694621:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3238 00000100:00100000:2.0:1787739063.694714:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562c000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665694592:0@lo:1000:mdt00_002.0 00000040:00080000:2.0:1787739063.695557:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].363, 2168 off551568 00000040:00080000:2.0:1787739063.695633:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].364, 2368 off603896 00000004:00080000:2.0:1787739063.695638:0:6525:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730610e840 with version 694 00000004:00080000:1.0:1787739063.695825:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 694 rpc_version 694 00000020:00100000:1.0:1787739063.695858:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739063.695866:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304245680 x1874579665694848/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739063.695884:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304245680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665694848:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739063.695941:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739063.696279:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.696364:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665694848 00000100:00100000:3.0:1787739063.696370:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665694848 00000100:00100000:3.0:1787739063.696403:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3239 00000100:00100000:3.0:1787739063.696439:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304244f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665694848:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739063.697380:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304244f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665694848:12345-0@lo:1000:osp_up0-1.0 Request processed in 967us (1436us total) trans 4294983407 rc 0/0 00000100:00100000:3.0:1787739063.697389:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3239 00000100:00100000:1.0:1787739063.697603:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304245680 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665694848:0@lo:1000:osp_up0-1.0 00010000:00010000:3.0:1787739063.697891:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad61e lrc: 3/0,1 mode: --/PW res: [0x240000402:0x17d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.697907:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad61e lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.697918:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad61e lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.697992:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad625 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.698000:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad625 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.698006:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad625 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.698104:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad62c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.698111:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad62c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.698121:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad62c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739063.698143:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb61c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665694976:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1787739063.698186:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.698235:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241600 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.698312:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665694976 00000100:00100000:0.0:1787739063.698320:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665694976 00000100:00100000:0.0:1787739063.698396:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3240 00000100:00100000:0.0:1787739063.698405:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6525:x1874579665694976:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787739063.698452:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739063.698470:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad633 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad62c expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.698490:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad633 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad62c expref: 10 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.698501:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad633 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad62c expref: 10 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.698508:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc6:0x0].0x0 (ffffa07307064d00) 00010000:00010000:0.0:1787739063.698511:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739063.698514:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b000, rc 0) 00000100:00100000:0.0:1787739063.698547:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6525:x1874579665694976:12345-0@lo:101:mdt00_002.0 Request processed in 143us (360us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.698552:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3240 00000100:00100000:1.0:1787739063.698620:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb61c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665694976:0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787739063.698632:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad62c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad633 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.698640:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad62c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad633 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739063.698670:0:6525:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f24a1050 reply state 000000002006e155 transno 4294968395 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad625 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.698681:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad625 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.698687:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc6:0x0].0x0 (ffffa074331c2c00) 00010000:00010000:1.0:1787739063.698689:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739063.698692:0:6525:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787739063.698698:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad62c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad633 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.698705:0:6525:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad62c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad633 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739063.698714:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xc6:0x0] 00000004:00010000:1.0:1787739063.698720:0:6525:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f24a1050 reply state 000000002006e155 transno 4294968395 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad61e lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.698725:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad61e lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x17d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.698730:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17d:0x0].0x0 (ffffa074331c2800) 00010000:00010000:1.0:1787739063.698732:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739063.698733:0:6525:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:1.0:1787739063.698738:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17d:0x0]: returning size 4096 00000004:00080000:1.0:1787739063.698744:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739063.698752:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ad602 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad609 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.698759:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ad602 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad609 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.698765:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ad602 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad609 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.698792:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739063.698797:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739063.698804:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739063.698810:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ad602 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad609 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.698817:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1cc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739063.698822:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ad602 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad609 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739063.698831:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ad602 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad609 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.698873:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ad602 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad609 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.698880:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739063.698900:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ad602 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad609 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.698906:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1cc00/0x7ee4453e4a1ad602 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad609 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.698934:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739063.698936:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739063.698937:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739063.698950:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665695104:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739063.698980:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739063.699034:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665695104 00000100:00100000:1.0:1787739063.699039:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665695104 00000100:00100000:1.0:1787739063.699071:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1333 00000100:00100000:1.0:1787739063.699078:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+10:3651:x1874579665695104:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739063.699091:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739063.699107:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad609 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad602 expref: 10 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.699119:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235200) 00010000:00010000:1.0:1787739063.699127:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739063.699155:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce03c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+9:3651:x1874579665695104:12345-0@lo:103:ldlm_bl.0 Request processed in 78us (174us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739063.699161:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1333 00000100:00100000:1.0:1787739063.699186:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665695104:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739063.699203:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad610 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad617 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.699210:0:6525:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad610 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad617 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739063.699277:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17625:x1874579652486528:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 19978us (20160us total) trans 4294968395 rc 0/0 00000100:00100000:1.0:1787739063.699284:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23494 00000100:00100000:0.0:1787739063.718669:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739063.718967:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652486912 00000100:00100000:3.0:1787739063.718978:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652486912 00000100:00100000:3.0:1787739063.719053:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23495 00000100:00100000:3.0:1787739063.719063:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ae580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17626:x1874579652486912:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:2.0:1787739063.722573:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ae580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17626:x1874579652486912:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 3502us (3904us total) trans 0 rc -17/-17 00000100:00100000:2.0:1787739063.722586:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23495 00000100:00100000:3.0:1787739063.746700:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787739063.746724:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356afc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665695232:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787739063.746799:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.746893:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665695360:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1787739063.747050:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.747522:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665695232 00000100:00100000:0.0:1787739063.747530:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665695232 00000100:00100000:0.0:1787739063.747578:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665695360 00000100:00100000:0.0:1787739063.747582:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665695360 00000100:00100000:0.0:1787739063.747605:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 358 00000100:00100000:0.0:1787739063.747615:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb61c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874579665695232:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787739063.747645:0:8441:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8646080 pending 0 free 3923529728 avail 3692007424 00000100:00080000:3.0:1787739063.747743:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787739063.747862:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb61c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874579665695232:12345-0@lo:13:osp-pre-0-0.0 Request processed in 248us (1063us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.747870:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 358 00000100:00100000:0.0:1787739063.747887:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 359 00000100:00100000:0.0:1787739063.747893:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415700b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874579665695360:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1787739063.747903:0:8441:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8646080 pending 0 free 3923529728 avail 3692007424 00000100:00100000:0.0:1787739063.747928:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415700b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874579665695360:12345-0@lo:13:osp-pre-0-1.0 Request processed in 36us (879us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.747933:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 359 00000100:00100000:3.0:1787739063.748003:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356afc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665695232:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1787739063.748217:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665695360:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787739063.749264:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739063.752598:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652487040 00000100:00100000:0.0:1787739063.752614:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652487040 00000100:00100000:0.0:1787739063.752767:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23496 00000100:00100000:0.0:1787739063.752781:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17627:x1874579652487040:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739063.752954:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739063.752978:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad63a lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0950b2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.753056:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad641 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.753070:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad641 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.753082:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad641 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.753094:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad648 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.753104:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ad617 lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad610 expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.753114:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad648 lrc: 4/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.753152:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ad617 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ad610 expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.753170:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356afc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665695488:0@lo:104: 00000100:00100000:0.0:1787739063.753234:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.753323:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306848a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.757000:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665695488 00000100:00100000:3.0:1787739063.757022:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665695488 00000100:00100000:3.0:1787739063.757150:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1171 00000100:00100000:3.0:1787739063.757163:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:13793:x1874579665695488:12345-0@lo:104: 00010000:00010000:3.0:1787739063.757181:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad610 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad617 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.757298:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739063.757304:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:13793:x1874579665695488:12345-0@lo:104: Request processed in 144us (4070us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.757311:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1171 00010000:00010000:1.0:1787739063.757356:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739063.757364:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad610 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad617 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.757373:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739063.757379:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad610 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ad617 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739063.757393:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad610 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ad617 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.757453:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad610 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad617 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.757461:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739063.757482:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad610 lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad617 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.757489:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad610 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad617 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.757509:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739063.757511:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739063.757513:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739063.757528:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665695616:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739063.757569:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.759565:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356afc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665695488:0@lo:104: 00010000:00010000:0.0:1787739063.759593:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad648 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739063.762637:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665695616 00000100:00100000:2.0:1787739063.762652:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665695616 00010000:00010000:2.0:1787739063.762705:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ad617 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ad610 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739063.762747:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1531 00000100:00100000:2.0:1787739063.762756:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce1a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3650:x1874579665695616:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739063.762770:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:2.0:1787739063.768839:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983407 is committed 00000001:00080000:2.0:1787739063.768868:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983406 is committed 00000020:00080000:2.0:1787739063.768877:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 378 batchid updated 00000020:00080000:2.0:1787739063.768891:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07306241300 refcount 2 committed 0 result 0 batchid 379 00000020:00080000:2.0:1787739063.768896:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07404f86c00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa073062413c0 00000020:00080000:2.0:1787739063.768900:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].376 00000020:00080000:2.0:1787739063.768903:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07404f86a00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07302faaa00 00000020:00080000:2.0:1787739063.768906:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].376 00000020:00080000:2.0:1787739063.768922:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 379 committed batchid 378 00000004:00080000:2.0:1787739063.769042:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730610e780 with version 719 00000020:00080000:2.0:1787739063.769053:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 378 cancel update log [0x240000401:0x3:0x0].374: rc = 0 00000020:00080000:2.0:1787739063.769094:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 378 cancel update log [0x200000400:0x3:0x0].375: rc = 0 00000020:00080000:2.0:1787739063.769102:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 378 00000004:00080000:2.0:1787739063.769110:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 719 rpc_version 719 00000020:00100000:2.0:1787739063.769141:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739063.769152:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743562c000 x1874579665695872/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739063.769176:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562c000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665695872:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739063.769245:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739063.769274:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ea80 going to sleep for 11 seconds 00010000:00010000:3.0:1787739063.769546:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad545 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769557:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad545 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769564:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad545 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769571:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739063.769575:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad545 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769583:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad545 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769596:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x49ac0f22 (ffffa0730608b900) 00010000:00010000:3.0:1787739063.769600:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739063.769604:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad545 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769611:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad545 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769625:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad54c lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x151c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769631:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad54c lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x151c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769637:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad54c lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x151c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769641:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739063.769644:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad54c lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x151c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769650:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad54c lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x151c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769655:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x151c:0x0].0x0 (ffffa0730608b500) 00010000:00010000:3.0:1787739063.769668:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739063.769671:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad54c lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x151c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769677:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad54c lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x151c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769691:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad553 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769698:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad553 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769704:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad553 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769710:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aae00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739063.769713:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad553 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769719:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad553 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769726:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc8:0x0].0x0 (ffffa0730608b400) 00010000:00010000:3.0:1787739063.769729:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739063.769732:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad553 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769738:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad553 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769753:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad5a7 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769759:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad5a7 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769765:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad5a7 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769770:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739063.769773:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad5a7 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769779:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad5a7 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769785:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x49ac0f22 (ffffa0730608b900) 00010000:00010000:3.0:1787739063.769788:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739063.769790:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad5a7 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.769796:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad5a7 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x49ac0f22 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739063.769817:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665695872 00000100:00100000:3.0:1787739063.769824:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665695872 00000100:00100000:3.0:1787739063.769866:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3241 00000100:00100000:3.0:1787739063.769875:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665695872:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739063.770024:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665695872:12345-0@lo:1000:osp_up1-0.0 Request processed in 138us (767us total) trans 4294968396 rc 0/0 00000100:00100000:3.0:1787739063.770031:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3241 00010000:00010000:2.0:1787739063.770172:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ad617 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ad610 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.770184:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x917910c6 (ffffa074331c2300) 00010000:00010000:2.0:1787739063.770188:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad648 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.770198:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad648 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.770206:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad648 lrc: 4/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739063.770246:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562c000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665695872:0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1787739063.770269:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739063.770304:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce1a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+6:3650:x1874579665695616:12345-0@lo:103:ldlm_bl.0 Request processed in 7549us (12735us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.770312:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1531 00000100:00100000:2.0:1787739063.770750:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665695616:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739063.772470:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad648 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.772486:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad648 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739063.772606:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665696000:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739063.772686:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.772729:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062416c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739063.772957:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665696000 00000100:00100000:2.0:1787739063.772966:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665696000 00000100:00100000:2.0:1787739063.773025:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3242 00000100:00100000:2.0:1787739063.773034:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665696000:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1787739063.773123:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:13793:x1874579665696000:12345-0@lo:1000:mdt00_004.0 Request processed in 90us (437us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.773132:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3242 00000100:00100000:2.0:1787739063.773171:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07415701a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665696000:0@lo:1000:mdt00_004.0 00010000:00010000:2.0:1787739063.773222:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad64f lrc: 3/1,0 mode: --/PR res: [0x240000402:0x17d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.773236:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad64f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.773247:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad64f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739063.773272:0:13793:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad64f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.773286:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad641 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.773293:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad641 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.773299:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad641 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.773305:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad641 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.773312:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739063.773316:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad641 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.774494:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad641 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.774522:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07307064900) 00010000:00010000:2.0:1787739063.774525:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739063.774531:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad641 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.774539:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad641 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.774556:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad648 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.774563:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad648 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.774569:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad648 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.774575:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739063.774578:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad648 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.774584:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad648 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.774591:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x917910c6 (ffffa074331c2300) 00010000:00010000:2.0:1787739063.774593:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739063.774595:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad648 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.774601:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad648 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.774639:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad63a lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950b2 expref: 15 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.775003:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad64f lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x17d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950b2 expref: 14 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.775021:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17d:0x0].0x0 (ffffa07403a6f100) 00010000:00010000:2.0:1787739063.775024:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739063.775026:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633ce00, rc 0) 00000100:00100000:2.0:1787739063.775142:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17627:x1874579652487040:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 22363us (25879us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.775151:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23496 00000100:00100000:3.0:1787739063.790752:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739063.790820:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652487168 00000100:00100000:1.0:1787739063.790829:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652487168 00000100:00100000:1.0:1787739063.790885:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23497 00000100:00100000:1.0:1787739063.790895:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f77840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17627:x1874579652487168:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739063.790918:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739063.790935:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad656 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x17d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0950b9 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.790987:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad65d lrc: 3/1,0 mode: --/PR res: [0x240000402:0x17d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.791000:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad65d lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.791021:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad65d lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739063.791110:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17d:0x0]: returning size 4096 00000004:00200000:1.0:1787739063.791115:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739063.791147:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad656 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x17d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950b9 expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.791162:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad65d lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x17d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950b9 expref: 10 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.791170:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17d:0x0].0x0 (ffffa074331c2800) 00010000:00010000:1.0:1787739063.791173:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739063.791176:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1cc00, rc 0) 00000100:00100000:1.0:1787739063.791229:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f77840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17627:x1874579652487168:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 336us (487us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739063.791236:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23497 00000100:00100000:3.0:1787739063.814245:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739063.814354:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652487296 00000100:00100000:2.0:1787739063.814364:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652487296 00000100:00100000:2.0:1787739063.814583:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23498 00000100:00100000:2.0:1787739063.814595:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17627:x1874579652487296:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739063.814617:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739063.814637:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ad664 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0950c0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.814711:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad66b lrc: 3/1,0 mode: --/PR res: [0x240000400:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.814726:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad66b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.814738:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad66b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739063.814805:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xc6:0x0]: returning size 4096 00000004:00200000:2.0:1787739063.814811:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739063.814846:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ad664 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950c0 expref: 12 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.814865:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad66b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950c0 expref: 11 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.814875:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc6:0x0].0x0 (ffffa074331c2c00) 00010000:00010000:2.0:1787739063.814878:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739063.814882:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d400, rc 0) 00000100:00100000:2.0:1787739063.814962:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17627:x1874579652487296:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 369us (726us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.814970:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23498 00000100:00100000:3.0:1787739063.819865:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739063.819953:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652487424 00000100:00100000:0.0:1787739063.819963:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652487424 00000100:00100000:0.0:1787739063.820043:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23499 00000100:00100000:0.0:1787739063.820057:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17627:x1874579652487424:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739063.820077:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739063.820096:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad672 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0950c7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.820160:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad679 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.820176:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad633 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad62c expref: 5 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.820190:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad679 lrc: 4/1,0 mode: --/PR res: [0x200000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.820223:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad633 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ad62c expref: 5 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.820239:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356afc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665696128:0@lo:104: 00000100:00100000:0.0:1787739063.820289:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.820333:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306848a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.820586:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665696128 00000100:00100000:3.0:1787739063.820591:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665696128 00000100:00100000:3.0:1787739063.820629:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1172 00000100:00100000:3.0:1787739063.820636:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:13793:x1874579665696128:12345-0@lo:104: 00010000:00010000:3.0:1787739063.820649:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad62c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad633 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.820734:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739063.820740:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:13793:x1874579665696128:12345-0@lo:104: Request processed in 105us (451us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.820747:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1172 00010000:00010000:3.0:1787739063.820769:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739063.820775:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad62c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad633 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.820783:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739063.820789:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad62c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xc6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ad633 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739063.820798:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad62c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xc6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ad633 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.820825:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad62c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xc6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad633 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.820831:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739063.820852:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad62c lrc: 1/0,0 mode: --/PW res: [0x200000401:0xc6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad633 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.820861:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad62c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xc6:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad633 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.820878:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739063.820880:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739063.820882:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739063.820889:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787739063.820899:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665696256:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739063.821232:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356afc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665696128:0@lo:104: 00010000:00010000:1.0:1787739063.821253:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad679 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739063.821385:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.821510:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665696256 00000100:00100000:0.0:1787739063.821515:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665696256 00010000:00010000:0.0:1787739063.821539:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad633 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ad62c expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.821560:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1532 00000100:00100000:0.0:1787739063.821567:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d87480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+6:3650:x1874579665696256:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739063.821578:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00080000:3.0:1787739063.824649:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 379 batchid updated 00000020:00080000:2.0:1787739063.824716:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 379 00000004:00080000:2.0:1787739063.824775:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07410404300 with version 720 00000020:00080000:2.0:1787739063.824803:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 379 cancel update log [0x240000401:0x3:0x0].376: rc = 0 00000020:00080000:2.0:1787739063.824872:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 379 cancel update log [0x200000400:0x3:0x0].376: rc = 0 00000020:00080000:2.0:1787739063.824881:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 379 00000004:00080000:3.0:1787739063.825005:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 720 rpc_version 720 00000020:00100000:3.0:1787739063.825060:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739063.825071:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304d87840 x1874579665696512/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739063.825095:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d87840 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665696512:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739063.825162:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739063.825363:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665696512 00000100:00100000:2.0:1787739063.825370:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665696512 00000100:00100000:2.0:1787739063.825441:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3243 00000100:00100000:2.0:1787739063.825450:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d852c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665696512:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739063.825595:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062410c0 going to sleep for 11 seconds 00010000:00010000:0.0:1787739063.825881:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad633 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ad62c expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.825898:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc6:0x0].0x0 (ffffa07307064d00) 00010000:00010000:0.0:1787739063.825903:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad679 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.825912:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad679 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.825920:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad679 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.825980:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739063.826061:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d852c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665696512:12345-0@lo:1000:osp_up1-0.0 Request processed in 612us (898us total) trans 4294968397 rc 0/0 00000100:00100000:2.0:1787739063.826070:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3243 00000100:00100000:3.0:1787739063.826197:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d87840 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665696512:0@lo:1000:osp_up1-0.0 00010000:00010000:1.0:1787739063.826234:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad679 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.826242:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad679 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739063.826295:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xc6:0x0]: returning size 4096 00000004:00200000:1.0:1787739063.826300:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739063.826341:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad672 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950c7 expref: 16 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.826356:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad679 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950c7 expref: 15 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.826364:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc6:0x0].0x0 (ffffa07307064d00) 00010000:00010000:1.0:1787739063.826366:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739063.826369:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a400, rc 0) 00000100:00100000:1.0:1787739063.826450:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17627:x1874579652487424:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 6397us (6596us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739063.826458:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23499 00000100:00100000:0.0:1787739063.831102:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d87480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+5:3650:x1874579665696256:12345-0@lo:103:ldlm_bl.0 Request processed in 9526us (9708us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.831117:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1532 00000100:00100000:3.0:1787739063.834337:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665696256:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739063.834480:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739063.834516:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739063.834596:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652487552 00000100:00100000:1.0:1787739063.834602:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652487552 00000100:00100000:1.0:1787739063.834668:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1334 00000100:00100000:1.0:1787739063.834678:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304261a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2408:x1874579652487552:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:1.0:1787739063.834694:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739063.834722:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad64f lrc: 0/0,0 mode: --/PR res: [0x240000402:0x17d:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950b2 expref: 15 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.834741:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17d:0x0].0x0 (ffffa07403a6f100) 00010000:00010000:1.0:1787739063.834750:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739063.834813:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304261a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2408:x1874579652487552:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 136us (334us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739063.834823:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1334 00000100:00100000:3.0:1787739063.835638:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652487808 00000100:00100000:3.0:1787739063.835647:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652487808 00000100:00100000:3.0:1787739063.835707:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23500 00000100:00100000:3.0:1787739063.835715:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042621c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17627:x1874579652487808:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:3.0:1787739063.835765:0:6524:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739063.835881:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad65d lrc: 0/0,0 mode: --/PR res: [0x240000402:0x17d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950b9 expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.835899:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17d:0x0].0x0 (ffffa074331c2800) 00010000:00010000:3.0:1787739063.835903:0:6524:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739063.835957:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ad680 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.835965:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ad680 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.835976:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ad680 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739063.835998:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d84b40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665696640:0@lo:101:mdt00_001.0 00000100:00100000:3.0:1787739063.836141:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.836220:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062410c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.837774:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665696640 00000100:00100000:0.0:1787739063.837848:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665696640 00000100:00100000:1.0:1787739063.837993:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3244 00000100:00100000:1.0:1787739063.838003:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+5:6524:x1874579665696640:12345-0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739063.838031:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739063.838059:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad687 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad680 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.838086:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad687 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad680 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.838100:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad687 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad680 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.838108:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235200) 00010000:00010000:1.0:1787739063.838111:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739063.838114:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1ca00, rc 0) 00000100:00100000:1.0:1787739063.838219:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:6524:x1874579665696640:12345-0@lo:101:mdt00_001.0 Request processed in 217us (2078us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739063.838226:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3244 00000100:00100000:3.0:1787739063.838452:0:6524:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983407 gen 2 00000004:00080000:3.0:1787739063.838459:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983405 committed transno 4294983407 00000004:00080000:3.0:1787739063.838479:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983407 committed transno 4294983407 00000100:00100000:3.0:1787739063.838499:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d84b40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665696640:0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787739063.838516:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ad680 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad687 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.838526:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ad680 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad687 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739063.838543:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739063.838573:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad68e lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.838581:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad68e lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.838589:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad68e lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739063.838606:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d84b40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665696768:0@lo:101:mdt00_001.0 00000100:00100000:3.0:1787739063.838644:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.838853:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062410c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739063.838922:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665696768 00000100:00100000:2.0:1787739063.838927:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665696768 00000100:00100000:2.0:1787739063.838970:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3245 00000100:00100000:2.0:1787739063.838978:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304245680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:6524:x1874579665696768:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739063.838989:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739063.839004:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ad695 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad68e expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.839031:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ad695 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad68e expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.839113:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ad695 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad68e expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.839121:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x917910c6 (ffffa07403a6f000) 00010000:00010000:2.0:1787739063.839124:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739063.839126:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633dc00, rc 0) 00000100:00100000:2.0:1787739063.839208:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304245680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665696768:12345-0@lo:101:mdt00_001.0 Request processed in 231us (564us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.839215:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3245 00000100:00100000:0.0:1787739063.839407:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d84b40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665696768:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739063.839649:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad68e lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad695 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.839723:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad68e lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad695 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739063.839741:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:1.0:1787739063.844776:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce21c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665696896:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739063.845179:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739063.845262:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739063.846531:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665696896 00000100:00100000:2.0:1787739063.846544:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665696896 00000100:00100000:2.0:1787739063.846648:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3246 00000100:00100000:2.0:1787739063.846667:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665696896:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739063.846777:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665696896:12345-0@lo:1000:mdt00_001.0 Request processed in 120us (1598us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.847766:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3246 00000100:00100000:1.0:1787739063.848975:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce21c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665696896:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739063.849063:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce21c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665697024:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739063.849117:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739063.850792:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.850982:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665697024 00000100:00100000:0.0:1787739063.850991:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665697024 00000100:00100000:0.0:1787739063.851057:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3247 00000100:00100000:0.0:1787739063.851067:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665697024:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739063.851201:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665697024:12345-0@lo:1000:mdt00_001.0 Request processed in 136us (2084us total) trans 0 rc 1/1 00000100:00100000:0.0:1787739063.851209:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3247 00000100:00100000:1.0:1787739063.851363:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce21c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665697024:0@lo:1000:mdt00_001.0 00010000:00010000:1.0:1787739063.851453:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ad69c lrc: 3/0,1 mode: --/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.851463:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ad69c lrc: 3/0,1 mode: --/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.851474:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ad69c lrc: 3/0,1 mode: --/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739063.851492:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce21c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665697152:0@lo:101:mdt00_001.0 00000100:00100000:1.0:1787739063.851530:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739063.851555:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d900 going to sleep for 11 seconds 00000100:00100000:2.0:1787739063.851609:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665697152 00000100:00100000:2.0:1787739063.851614:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665697152 00000100:00100000:2.0:1787739063.851977:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3248 00000100:00100000:2.0:1787739063.851986:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6524:x1874579665697152:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739063.851999:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739063.852029:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad6a3 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad69c expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.852073:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad6a3 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad69c expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.852088:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad6a3 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad69c expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.852097:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17d:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:2.0:1787739063.852100:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739063.852103:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633ce00, rc 0) 00000100:00100000:2.0:1787739063.852143:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6524:x1874579665697152:12345-0@lo:101:mdt00_001.0 Request processed in 157us (612us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.852149:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3248 00000100:00100000:2.0:1787739063.852178:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce21c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665697152:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739063.852191:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ad69c lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad6a3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.852200:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ad69c lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad6a3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.852223:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ad6aa lrc: 3/0,1 mode: --/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.852232:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ad6aa lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.852241:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ad6aa lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.852258:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad6b1 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.852269:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad66b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950c0 expref: 10 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.852278:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad6b1 lrc: 4/0,1 mode: --/EX res: [0x240000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.852302:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad66b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950c0 expref: 10 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.852315:0:6524:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad66b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950c0 expref: 10 pid: 7944 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739063.852326:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562cb40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665697280:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739063.852382:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074104049c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739063.857379:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562cb40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665697280:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739063.857404:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad6b1 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.857517:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739063.857858:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652487936 00000100:00100000:2.0:1787739063.857864:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652487936 00010000:00010000:2.0:1787739063.857895:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad66b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950c0 expref: 11 pid: 7944 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739063.857928:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1533 00000100:00100000:2.0:1787739063.857936:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304263840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2409:x1874579652487936:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739063.857949:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739063.857955:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad66b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950c0 expref: 11 pid: 7944 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.857967:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad66b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950c0 expref: 11 pid: 7944 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.857987:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad66b lrc: 0/0,0 mode: --/PR res: [0x240000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950c0 expref: 11 pid: 7944 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.858001:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc6:0x0].0x0 (ffffa074331c2c00) 00010000:00010000:2.0:1787739063.858006:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad6b1 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.858026:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad6b1 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.858040:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad6b1 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.858064:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739063.858106:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304263840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2409:x1874579652487936:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 170us (590us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.858114:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1533 00010000:00010000:0.0:1787739063.858239:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad6b1 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.858251:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad6b1 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.858291:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1ad6b8 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.858297:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1ad6b8 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.858307:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1ad6b8 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.858325:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042612c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665697408:0@lo:101:mdt00_001.0 00000100:00100000:0.0:1787739063.858367:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.858391:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068483c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.858403:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665697408 00000100:00100000:0.0:1787739063.858407:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665697408 00000100:00100000:0.0:1787739063.858457:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3249 00000100:00100000:0.0:1787739063.858464:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6524:x1874579665697408:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739063.858478:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739063.858490:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad6bf lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad6b8 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.858510:0:6528:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad679 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950c7 expref: 13 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.858518:0:6528:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad6bf lrc: 4/0,0 mode: --/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad6b8 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.858543:0:6528:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad679 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950c7 expref: 13 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.858555:0:6528:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad679 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950c7 expref: 13 pid: 13793 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.858566:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305656940 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6528:1874579665697536:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739063.858601:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068489c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.876492:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305656940 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6528:1874579665697536:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739063.876523:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00010000:00010000:3.0:1787739063.876529:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad6bf lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad6b8 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.876543:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc6:0x0].0x0 (ffffa07307064d00) 00010000:00010000:3.0:1787739063.876546:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739063.876550:0:6528:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad6bf lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad6b8 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739063.876554:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652488064 00010000:00010000:3.0:1787739063.876559:0:6528:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad6bf lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad6b8 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739063.876560:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652488064 00010000:00010000:3.0:1787739063.876566:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609bc00, rc 0) 00010000:00010000:2.0:1787739063.876589:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad679 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950c7 expref: 14 pid: 13793 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739063.876619:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1534 00000100:00100000:2.0:1787739063.876628:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba7c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652488064:12345-192.168.203.15@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787739063.876633:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665697408:12345-0@lo:101:mdt00_001.0 Request processed in 18170us (18265us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.876639:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3249 00010000:00010000:2.0:1787739063.876642:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739063.876647:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad679 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950c7 expref: 14 pid: 13793 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.880980:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad679 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950c7 expref: 14 pid: 13793 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.881023:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad679 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950c7 expref: 14 pid: 13793 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.881040:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc6:0x0].0x0 (ffffa07307064d00) 00010000:00010000:2.0:1787739063.881047:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad6bf lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad6b8 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.881057:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad6bf lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad6b8 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.881066:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad6bf lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad6b8 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.881112:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad6bf lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad6b8 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739063.881128:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562cb40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665697664:0@lo:105: 00000100:00100000:2.0:1787739063.881189:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739063.881232:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074104049c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.883295:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665697664 00000100:00100000:3.0:1787739063.883313:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665697664 00000100:00100000:3.0:1787739063.883390:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1173 00000100:00100000:3.0:1787739063.883400:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562cf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665697664:12345-0@lo:105: 00010000:00010000:3.0:1787739063.883435:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1ad6b8 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad6bf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.883446:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1ad6b8 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad6bf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.883454:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1ad6b8 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad6bf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.883462:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1ad6b8 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad6bf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.883470:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1ad6b8 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad6bf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.883478:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1ad6b8 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ad6bf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.883496:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609b000) 00000100:00100000:3.0:1787739063.883585:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562cf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665697664:12345-0@lo:105: Request processed in 186us (2397us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.883593:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1173 00000100:00100000:3.0:1787739063.883636:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562cb40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665697664:0@lo:105: 00010000:00010000:3.0:1787739063.883665:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739063.883711:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba7c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:2409:x1874579652488064:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 7081us (7187us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.883723:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1534 00000100:00100000:0.0:1787739063.887981:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042612c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665697408:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739063.888007:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1ad6b8 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad6bf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.894328:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305654b40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665697792:0@lo:403:mdt00_001.0 00000100:00100000:0.0:1787739063.894411:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.894464:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068483c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739063.894468:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665697792 00000100:00100000:2.0:1787739063.894474:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665697792 00000100:00100000:2.0:1787739063.894527:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3250 00000100:00100000:2.0:1787739063.894536:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305654780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665697792:12345-0@lo:403:mdt00_001.0 00000100:00100000:3.0:1787739063.897803:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305654780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665697792:12345-0@lo:403:mdt00_001.0 Request processed in 3260us (3384us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.897818:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3250 00000100:00100000:0.0:1787739063.897957:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305654b40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665697792:0@lo:403:mdt00_001.0 00000040:00080000:0.0:1787739063.899573:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].364, 664 off552232 00000040:00080000:0.0:1787739063.899847:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].365, 744 off604640 00000004:00080000:0.0:1787739063.899854:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa073068483c0 with version 695 00000004:00080000:2.0:1787739063.899894:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 695 rpc_version 695 00000020:00100000:2.0:1787739063.899927:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739063.899936:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743562cb40 x1874579665698048/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1808/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739063.899958:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562cb40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665698048:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739063.900031:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739063.900063:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.902403:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665698048 00000100:00100000:3.0:1787739063.902443:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665698048 00000100:00100000:2.0:1787739063.902557:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3251 00000100:00100000:2.0:1787739063.902569:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410b09050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665698048:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739063.903027:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410b09050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665698048:12345-0@lo:1000:osp_up0-1.0 Request processed in 460us (2995us total) trans 4294983408 rc 0/0 00000100:00100000:2.0:1787739063.903036:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3251 00000100:00100000:3.0:1787739063.903105:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562cb40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665698048:0@lo:1000:osp_up0-1.0 00000004:00200000:0.0:1787739063.903167:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17d:0x0]: returning size 4096 00010000:00010000:0.0:1787739063.903188:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad6b1 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903197:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad6b1 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903209:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad6b1 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903215:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739063.903220:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad6b1 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903228:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad6b1 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903243:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc6:0x0].0x0 (ffffa074331c2c00) 00010000:00010000:0.0:1787739063.903246:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739063.903249:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad6b1 lrc: 2/0,0 mode: --/EX res: [0x240000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903255:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad6b1 lrc: 0/0,0 mode: --/EX res: [0x240000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903268:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1ad6b8 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad6bf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903275:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1ad6b8 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad6bf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903283:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1ad6b8 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad6bf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903304:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739063.903312:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xc6:0x0] 00010000:00010000:0.0:1787739063.903318:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ad6aa lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903325:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ad6aa lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903330:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ad6aa lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903334:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739063.903337:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ad6aa lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903342:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ad6aa lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903349:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17d:0x0].0x0 (ffffa074331c2800) 00010000:00010000:0.0:1787739063.903351:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739063.903354:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ad6aa lrc: 2/0,0 mode: --/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903359:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c600/0x7ee4453e4a1ad6aa lrc: 0/0,0 mode: --/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903369:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ad69c lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad6a3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903377:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ad69c lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad6a3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903384:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ad69c lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad6a3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903409:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:0.0:1787739063.903449:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739063.903454:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ad680 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad687 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.903458:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787739063.903461:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ad680 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad687 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.903464:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1ad6b8 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad6bf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903468:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ad680 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad687 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.903471:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739063.903475:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1ad6b8 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad6bf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903481:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00010000:2.0:1787739063.903483:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1ad6b8 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad6bf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739063.903484:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739063.903488:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad68e lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad695 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903494:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad68e lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad695 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.903510:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1ad6b8 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad6bf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.903516:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739063.903535:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1ad6b8 lrc: 1/0,0 mode: --/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad6bf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.903542:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b000/0x7ee4453e4a1ad6b8 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad6bf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.903551:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042621c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17627:x1874579652487808:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 67836us (69033us total) trans 4294968398 rc 0/0 00010000:00010000:2.0:1787739063.903556:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00000100:00100000:0.0:1787739063.903558:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23500 00010000:00010000:2.0:1787739063.903559:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ad69c lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad6a3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.903565:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1de00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739063.903568:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ad69c lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad6a3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739063.903575:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ad69c lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad6a3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903577:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739063.903581:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ad680 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad687 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903588:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aac00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739063.903590:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ad69c lrc: 2/0,0 mode: --/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad6a3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903592:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ad680 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad687 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.903597:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000004:00010000:0.0:1787739063.903599:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ad680 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad687 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.903603:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ad69c lrc: 1/0,0 mode: --/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad6a3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.903610:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1de00/0x7ee4453e4a1ad69c lrc: 0/0,0 mode: --/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad6a3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903614:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ad680 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad687 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903619:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739063.903620:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739063.903622:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739063.903623:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739063.903632:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ad680 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad687 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739063.903635:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665698176:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739063.903640:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ad680 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad687 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.903649:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739063.903651:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739063.903652:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739063.903660:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305654b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665698432:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739063.903668:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.903683:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787739063.903732:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739063.903796:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787739063.903803:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665698304:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739063.906451:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739063.906454:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665698176 00000100:00100000:0.0:1787739063.906463:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665698176 00000100:00100000:0.0:1787739063.906513:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665698432 00000100:00100000:0.0:1787739063.906516:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665698432 00000100:00100000:0.0:1787739063.906532:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665698304 00000100:00100000:0.0:1787739063.906534:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665698304 00000100:00100000:0.0:1787739063.906548:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1335 00000100:00100000:0.0:1787739063.906557:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562ed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+12:3652:x1874579665698176:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739063.906570:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739063.906593:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad6bf lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad6b8 expref: 12 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.906611:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc6:0x0].0x0 (ffffa07307064d00) 00010000:00010000:0.0:1787739063.906620:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739063.906671:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562ed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+11:3652:x1874579665698176:12345-0@lo:103:ldlm_bl.0 Request processed in 116us (3002us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.906678:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1335 00000100:00100000:0.0:1787739063.906693:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1336 00000100:00100000:0.0:1787739063.906698:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073056552c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+10:3650:x1874579665698432:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739063.906705:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739063.906714:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad687 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad680 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.906725:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235200) 00010000:00010000:0.0:1787739063.906732:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739063.906760:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073056552c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+9:3650:x1874579665698432:12345-0@lo:103:ldlm_bl.0 Request processed in 62us (3075us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.906766:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1336 00000100:00100000:2.0:1787739063.906938:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665698176:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739063.907247:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1337 00000100:00100000:0.0:1787739063.907257:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562da40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+8:3652:x1874579665698304:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739063.907265:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739063.907277:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad6a3 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x17d:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad69c expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.907290:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17d:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:0.0:1787739063.907296:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739063.907336:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562da40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+7:3652:x1874579665698304:12345-0@lo:103:ldlm_bl.0 Request processed in 82us (892us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.907380:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665698304:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739063.914212:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1337 00000100:00100000:0.0:1787739063.914292:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305654b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665698432:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739063.935388:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739063.935758:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652488192 00000100:00100000:1.0:1787739063.935768:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652488192 00000100:00100000:1.0:1787739063.935830:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23501 00000100:00100000:1.0:1787739063.935842:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304245e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17628:x1874579652488192:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739063.935866:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739063.935886:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad6c6 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0950ce expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.935949:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad6cd lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.935963:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad6cd lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.935975:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad6cd lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.935994:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad6d4 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.936002:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad6d4 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.936019:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad6d4 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.936058:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad6cd lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.936066:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad6cd lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.936073:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad6cd lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.936079:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad6cd lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.936085:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739063.936089:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad6cd lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.936097:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad6cd lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.936110:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064a00) 00010000:00010000:1.0:1787739063.936113:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739063.936118:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad6cd lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.936124:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad6cd lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.936135:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad6d4 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.936142:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad6d4 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.936147:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad6d4 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.936151:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739063.936154:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad6d4 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.936159:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad6d4 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.936166:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x1640014e (ffffa07433235100) 00010000:00010000:1.0:1787739063.936168:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739063.936171:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad6d4 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.936177:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad6d4 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.936206:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad6c6 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950ce expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.936222:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064a00) 00010000:00010000:1.0:1787739063.936224:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739063.936228:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad6c6 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950ce expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.936237:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1dc00, rc 0) 00000100:00100000:1.0:1787739063.936297:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304245e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17628:x1874579652488192:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 458us (928us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739063.936305:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23501 00000100:00100000:3.0:1787739063.945183:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739063.945278:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652488320 00000100:00100000:0.0:1787739063.945289:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652488320 00000100:00100000:0.0:1787739063.945347:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23502 00000100:00100000:0.0:1787739063.945358:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304244780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17628:x1874579652488320:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739063.945388:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739063.945410:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad6db lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0950d5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.945490:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad6e2 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.945503:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad6e2 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.945514:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad6e2 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739063.945560:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:0.0:1787739063.945565:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739063.945599:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad6db lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950d5 expref: 15 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.945614:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad6e2 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950d5 expref: 14 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.945627:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9f00) 00010000:00010000:0.0:1787739063.945630:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739063.945634:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a600, rc 0) 00000100:00100000:0.0:1787739063.945695:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304244780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17628:x1874579652488320:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 338us (520us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.945702:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23502 00000100:00100000:0.0:1787739063.971257:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739063.971336:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652488576 00000100:00100000:1.0:1787739063.971346:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652488576 00000100:00100000:1.0:1787739063.971404:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23503 00000100:00100000:1.0:1787739063.971433:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305654b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17628:x1874579652488576:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:1.0:1787739063.971539:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665698560:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739063.971622:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739063.971674:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739063.971738:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665698560 00000100:00100000:2.0:1787739063.971743:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665698560 00000100:00100000:2.0:1787739063.971791:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3252 00000100:00100000:2.0:1787739063.971798:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6524:x1874579665698560:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739063.971878:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6524:x1874579665698560:12345-0@lo:1000:mdt00_001.0 Request processed in 80us (254us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.971886:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3252 00000100:00100000:1.0:1787739063.971904:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665698560:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739063.971948:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665698688:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739063.971970:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739063.971993:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739063.972005:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665698688 00000100:00100000:2.0:1787739063.972019:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665698688 00000100:00100000:2.0:1787739063.972058:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3253 00000100:00100000:2.0:1787739063.972065:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6524:x1874579665698688:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739063.972141:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:6524:x1874579665698688:12345-0@lo:1000:mdt00_001.0 Request processed in 76us (169us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739063.972147:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3253 00000100:00100000:1.0:1787739063.972302:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665698688:0@lo:1000:mdt00_001.0 00010000:00010000:1.0:1787739063.972348:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad6e9 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.972363:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad6e9 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.972372:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad6e9 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739063.972385:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665698816:0@lo:101:mdt00_001.0 00000100:00100000:1.0:1787739063.972410:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739063.972446:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739063.972448:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665698816 00000100:00100000:2.0:1787739063.972452:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665698816 00000100:00100000:2.0:1787739063.972483:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3254 00000100:00100000:2.0:1787739063.972489:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6524:x1874579665698816:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739063.972498:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739063.972513:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad6f0 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad6e9 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.972538:0:6529:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad43b lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095050 expref: 9 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.972548:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad6f0 lrc: 4/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad6e9 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.972570:0:6529:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad43b lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095050 expref: 9 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.972585:0:6529:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad43b lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095050 expref: 9 pid: 6524 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739063.972595:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562c3c0 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665698944:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739063.972732:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ea80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.977631:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739063.977788:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652488704 00000100:00100000:1.0:1787739063.977797:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652488704 00010000:00010000:1.0:1787739063.977834:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad43b lrc: 5/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095050 expref: 10 pid: 6524 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739063.977874:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1535 00000100:00100000:1.0:1787739063.977884:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305657c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652488704:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739063.977898:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739063.977905:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad43b lrc: 5/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095050 expref: 10 pid: 6524 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.977919:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad43b lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095050 expref: 10 pid: 6524 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.977941:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064a00) 00010000:00010000:1.0:1787739063.977948:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad6f0 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad6e9 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.977964:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad6f0 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad6e9 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.977973:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad6f0 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad6e9 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.978003:0:10340:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad6f0 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad6e9 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739063.978027:0:10340:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce2d00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665699072:0@lo:105: 00000100:00100000:1.0:1787739063.978076:0:10340:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739063.978112:0:10340:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.978232:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665699072 00000100:00100000:0.0:1787739063.978236:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665699072 00000100:00100000:0.0:1787739063.978278:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1174 00000100:00100000:0.0:1787739063.978285:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665699072:12345-0@lo:105: 00010000:00010000:0.0:1787739063.978296:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad6e9 lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad6f0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.978305:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad6e9 lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad6f0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.978314:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad6e9 lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad6f0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.978321:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad6e9 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad6f0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.978328:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad6e9 lrc: 5/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad6f0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.978341:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad6e9 lrc: 5/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ad6f0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.978354:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07441a1dc00) 00000100:00100000:0.0:1787739063.978408:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665699072:12345-0@lo:105: Request processed in 124us (331us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.978437:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1174 00000100:00100000:0.0:1787739063.978474:0:10340:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce2d00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665699072:0@lo:105: 00010000:00010000:0.0:1787739063.978488:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739063.978522:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305657c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652488704:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 640us (901us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.978528:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1535 00010000:00010000:2.0:1787739063.979498:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad43b lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095050 expref: 9 pid: 6524 timeout: 750 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739063.979523:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562c3c0 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665698944:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739063.979542:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad6f0 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad6e9 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.979552:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064a00) 00010000:00010000:2.0:1787739063.979555:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739063.979558:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d600, rc 0) 00000100:00100000:2.0:1787739063.979601:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665698816:12345-0@lo:101:mdt00_001.0 Request processed in 7112us (7190us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.979609:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3254 00000100:00100000:1.0:1787739063.980464:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665698816:0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739063.980485:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad6e9 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad6f0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739063.980507:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739063.980545:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d000/0x7ee4453e4a1ad6f7 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.980552:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d000/0x7ee4453e4a1ad6f7 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.980560:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d000/0x7ee4453e4a1ad6f7 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739063.980577:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665699200:0@lo:101:mdt00_001.0 00000100:00100000:1.0:1787739063.980624:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739063.980646:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739063.980869:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665699200 00000100:00100000:0.0:1787739063.980875:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665699200 00000100:00100000:0.0:1787739063.980921:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3255 00000100:00100000:0.0:1787739063.980930:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:6524:x1874579665699200:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739063.980946:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739063.980961:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad6fe lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad6f7 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.980986:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad6fe lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad6f7 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.980999:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad6fe lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad6f7 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.981021:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x1640014e (ffffa073068a9000) 00010000:00010000:0.0:1787739063.981024:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739063.981027:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609bc00, rc 0) 00000100:00100000:0.0:1787739063.981066:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:6524:x1874579665699200:12345-0@lo:101:mdt00_001.0 Request processed in 138us (442us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739063.981072:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3255 00000100:00100000:3.0:1787739063.982928:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665699200:0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787739063.982955:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d000/0x7ee4453e4a1ad6f7 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad6fe expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.982968:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d000/0x7ee4453e4a1ad6f7 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad6fe expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739063.982988:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:3.0:1787739063.983050:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042452c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665699328:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739063.983104:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.983136:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.983155:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665699328 00000100:00100000:3.0:1787739063.983160:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665699328 00000100:00100000:3.0:1787739063.983203:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3256 00000100:00100000:3.0:1787739063.983211:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304247840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:6524:x1874579665699328:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739063.983396:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304247840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:6524:x1874579665699328:12345-0@lo:1000:mdt00_001.0 Request processed in 184us (290us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.983404:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3256 00000100:00100000:3.0:1787739063.983457:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042452c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665699328:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739063.983494:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042452c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665699456:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739063.983517:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.983533:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.983544:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665699456 00000100:00100000:3.0:1787739063.983547:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665699456 00000100:00100000:3.0:1787739063.983564:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3257 00000100:00100000:3.0:1787739063.983570:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304244000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:6524:x1874579665699456:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739063.983635:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304244000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:6524:x1874579665699456:12345-0@lo:1000:mdt00_001.0 Request processed in 64us (117us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739063.983641:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3257 00000100:00100000:3.0:1787739063.983664:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042452c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665699456:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739063.983836:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042452c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665699584:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739063.983863:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.983879:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.983890:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665699584 00000100:00100000:3.0:1787739063.983893:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665699584 00000100:00100000:3.0:1787739063.983911:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3258 00000100:00100000:3.0:1787739063.983916:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304244b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:6524:x1874579665699584:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739063.983962:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304244b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:6524:x1874579665699584:12345-0@lo:1000:mdt00_001.0 Request processed in 44us (97us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739063.983969:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3258 00000100:00100000:3.0:1787739063.983992:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042452c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665699584:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739063.985216:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042452c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665699712:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739063.985261:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.985787:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.985897:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665699712 00000100:00100000:3.0:1787739063.985903:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665699712 00000100:00100000:3.0:1787739063.985942:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3259 00000100:00100000:3.0:1787739063.985949:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042470c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:6524:x1874579665699712:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739063.986205:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042470c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:6524:x1874579665699712:12345-0@lo:1000:mdt00_001.0 Request processed in 255us (942us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.986213:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3259 00000100:00100000:3.0:1787739063.986255:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042452c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665699712:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739063.986318:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042452c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665699840:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739063.986347:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.986365:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.986377:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665699840 00000100:00100000:3.0:1787739063.986380:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665699840 00000100:00100000:3.0:1787739063.986402:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3260 00000100:00100000:3.0:1787739063.986407:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304246580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:6524:x1874579665699840:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739063.986487:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304246580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:6524:x1874579665699840:12345-0@lo:1000:mdt00_001.0 Request processed in 79us (139us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.986493:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3260 00000100:00100000:3.0:1787739063.986518:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042452c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665699840:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739063.986552:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042452c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665699968:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739063.986574:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.986589:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.986601:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665699968 00000100:00100000:3.0:1787739063.986604:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665699968 00000100:00100000:3.0:1787739063.986623:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3261 00000100:00100000:3.0:1787739063.986628:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304244f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:6524:x1874579665699968:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739063.986670:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304244f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:6524:x1874579665699968:12345-0@lo:1000:mdt00_001.0 Request processed in 42us (95us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.986677:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3261 00000100:00100000:3.0:1787739063.986715:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042452c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665699968:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739063.986745:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042452c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665700096:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739063.986767:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.986782:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.986792:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665700096 00000100:00100000:3.0:1787739063.986795:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665700096 00000100:00100000:3.0:1787739063.986815:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3262 00000100:00100000:3.0:1787739063.986821:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304245680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:6524:x1874579665700096:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739063.986864:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304245680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:6524:x1874579665700096:12345-0@lo:1000:mdt00_001.0 Request processed in 43us (96us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739063.986871:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3262 00000100:00100000:3.0:1787739063.986895:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042452c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665700096:0@lo:1000:mdt00_001.0 00000040:00080000:3.0:1787739063.987665:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].378, 2168 off578672 00000040:00080000:3.0:1787739063.987741:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].378, 2368 off628360 00000004:00080000:3.0:1787739063.987747:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306241480 with version 721 00000004:00080000:2.0:1787739063.988944:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 721 rpc_version 721 00000020:00100000:2.0:1787739063.988987:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739063.988996:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743562c3c0 x1874579665700352/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739063.989027:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562c3c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665700352:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739063.989091:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.989118:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665700352 00000100:00100000:2.0:1787739063.989120:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ea80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739063.989124:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665700352 00000100:00100000:3.0:1787739063.989171:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3263 00000100:00100000:3.0:1787739063.989179:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665700352:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739063.992169:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+11:7740:x1874579665700352:12345-0@lo:1000:osp_up1-0.0 Request processed in 2984us (3070us total) trans 4294968399 rc 0/0 00000100:00100000:3.0:1787739063.992182:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3263 00000100:00100000:2.0:1787739063.992240:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562c3c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665700352:0@lo:1000:osp_up1-0.0 00010000:00010000:1.0:1787739063.992629:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad705 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x151f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.992646:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad705 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x151f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.992951:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad705 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x151f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.992975:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad70c lrc: 3/0,1 mode: --/PW res: [0x200000400:0xc9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.992983:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad70c lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.992989:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad70c lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.993482:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad713 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.993492:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad713 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739063.993503:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad713 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739063.993523:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce0b40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665700480:0@lo:101:mdt00_001.0 00000100:00100000:1.0:1787739063.993573:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739063.993602:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d900 going to sleep for 11 seconds 00000100:00100000:2.0:1787739063.997511:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665700480 00000100:00100000:2.0:1787739063.997526:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665700480 00000100:00100000:2.0:1787739063.997583:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3264 00000100:00100000:2.0:1787739063.997592:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+11:6524:x1874579665700480:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739063.997616:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739063.997638:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ad71a lrc: 2/0,0 mode: --/PW res: [0x240000401:0xc9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad713 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.997685:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ad71a lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad713 expref: 12 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.997700:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ad71a lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad713 expref: 12 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739063.997709:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc9:0x0].0x0 (ffffa07403a6fd00) 00010000:00010000:2.0:1787739063.997712:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739063.997715:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633da00, rc 0) 00000100:00100000:2.0:1787739063.997806:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+12:6524:x1874579665700480:12345-0@lo:101:mdt00_001.0 Request processed in 215us (4232us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739063.997813:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3264 00000100:00100000:0.0:1787739063.998481:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce0b40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665700480:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739063.998504:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad713 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad71a expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.998517:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad713 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad71a expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739063.998542:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a05c8b58 reply state 00000000de3feae0 transno 4294983409 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad70c lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.998555:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad70c lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.998562:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc9:0x0].0x0 (ffffa07433235300) 00010000:00010000:0.0:1787739063.998564:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739063.998567:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739063.998573:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad713 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad71a expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.998580:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad713 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad71a expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739063.998599:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xc9:0x0] 00000004:00010000:0.0:1787739063.998606:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a05c8b58 reply state 00000000de3feae0 transno 4294983409 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad705 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x151f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.998614:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad705 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x151f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.998620:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x151f:0x0].0x0 (ffffa07433235b00) 00010000:00010000:0.0:1787739063.998622:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739063.998624:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787739063.998629:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x151f:0x0]: returning size 4096 00000004:00080000:0.0:1787739063.998638:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739063.998647:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad6e9 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad6f0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.998653:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad6e9 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad6f0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.998660:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad6e9 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad6f0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.998676:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739063.998680:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739063.998742:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d000/0x7ee4453e4a1ad6f7 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad6fe expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739063.998750:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d000/0x7ee4453e4a1ad6f7 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad6fe expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739063.998836:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305654b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17628:x1874579652488576:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 27421us (27591us total) trans 4294983409 rc 0/0 00000100:00100000:0.0:1787739063.998844:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23503 00010000:00010000:3.0:1787739063.999001:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739063.999019:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad6e9 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad6f0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.999028:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1dc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739063.999104:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad6e9 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad6f0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739063.999114:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad6e9 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad6f0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.999207:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad6e9 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad6f0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.999214:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739063.999241:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad6e9 lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad6f0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.999249:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1dc00/0x7ee4453e4a1ad6e9 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad6f0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739063.999556:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739063.999558:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739063.999560:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739063.999619:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042452c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665700608:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739063.999670:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739063.999894:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665700608 00000100:00100000:3.0:1787739063.999900:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665700608 00000100:00100000:3.0:1787739063.999952:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1338 00000100:00100000:3.0:1787739063.999961:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305656d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+12:3651:x1874579665700608:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739063.999973:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739063.999992:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad6f0 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad6e9 expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.000006:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064a00) 00010000:00010000:3.0:1787739064.001134:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739064.001211:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305656d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+11:3651:x1874579665700608:12345-0@lo:103:ldlm_bl.0 Request processed in 1249us (1539us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739064.001221:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1338 00000100:00100000:1.0:1787739064.004339:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042452c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665700608:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739064.036877:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739064.036962:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652488960 00000100:00100000:3.0:1787739064.036971:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652488960 00000100:00100000:3.0:1787739064.037044:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23504 00000100:00100000:3.0:1787739064.037055:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305656580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17629:x1874579652488960:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:3.0:1787739064.037203:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305656580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17629:x1874579652488960:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 150us (335us total) trans 0 rc -17/-17 00000100:00100000:3.0:1787739064.037210:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23504 00000100:00100000:0.0:1787739064.066823:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739064.067269:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652489088 00000100:00100000:3.0:1787739064.067279:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652489088 00000100:00100000:3.0:1787739064.067439:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23505 00000100:00100000:3.0:1787739064.067451:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305654000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17630:x1874579652489088:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739064.067471:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739064.067494:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad721 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0950dc expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.067559:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad728 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.067574:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad728 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.067586:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad728 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.067597:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad72f lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.067711:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad6fe lrc: 2/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad6f7 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.067719:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad72f lrc: 4/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.068281:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad6fe lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ad6f7 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.068297:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042452c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665700736:0@lo:104: 00000100:00100000:3.0:1787739064.068476:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739064.068520:0:6506:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:2.0:1787739064.068532:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665700736 00000100:00100000:2.0:1787739064.068536:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665700736 00000100:00100000:2.0:1787739064.068613:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1175 00000100:00100000:2.0:1787739064.068621:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073041e1450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6523:x1874579665700736:12345-0@lo:104: 00010000:00010000:2.0:1787739064.068634:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d000/0x7ee4453e4a1ad6f7 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad6fe expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.068689:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739064.068695:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073041e1450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:6523:x1874579665700736:12345-0@lo:104: Request processed in 76us (220us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.068703:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1175 00010000:00010000:3.0:1787739064.068889:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739064.068896:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d000/0x7ee4453e4a1ad6f7 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad6fe expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.068905:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739064.068911:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d000/0x7ee4453e4a1ad6f7 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ad6fe expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739064.068921:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d000/0x7ee4453e4a1ad6f7 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ad6fe expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.068949:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d000/0x7ee4453e4a1ad6f7 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad6fe expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.068956:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1787739064.068995:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665700864:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739064.069039:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787739064.069061:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d000/0x7ee4453e4a1ad6f7 lrc: 1/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad6fe expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.069069:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d000/0x7ee4453e4a1ad6f7 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad6fe expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.069087:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739064.069089:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739064.069091:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739064.069093:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665700864 00000100:00100000:1.0:1787739064.069097:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665700864 00010000:00010000:1.0:1787739064.069114:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad6fe lrc: 5/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ad6f7 expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739064.069137:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1536 00000100:00100000:1.0:1787739064.069144:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+11:3652:x1874579665700864:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739064.069154:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787739064.071606:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241c00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739064.071663:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042452c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665700736:0@lo:104: 00010000:00010000:3.0:1787739064.071688:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad72f lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.073084:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad68e lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad695 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739064.073103:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad68e lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad695 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.073148:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad68e lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad695 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.073155:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739064.073177:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa200/0x7ee4453e4a1ad68e lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad695 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1787739064.073196:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968399 is committed 00000001:00080000:3.0:1787739064.073220:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968398 is committed 00000020:00080000:3.0:1787739064.073232:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa0730610e0c0 refcount 2 committed 0 result 0 batchid 371 00000020:00080000:3.0:1787739064.073236:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa074035e8200 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730610e840 00000020:00080000:3.0:1787739064.073239:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].364 00000020:00080000:3.0:1787739064.073242:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa074035e8500 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306aade00 00000020:00080000:3.0:1787739064.073245:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].363 00000020:00080000:3.0:1787739064.073257:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 370 batchid updated 00000020:00080000:3.0:1787739064.073275:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 371 committed batchid 370 00000004:00080000:3.0:1787739064.073386:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07306241c00 with version 696 00000020:00080000:3.0:1787739064.073396:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 369 cancel update log [0x200000401:0x3:0x0].363: rc = 0 00000020:00080000:3.0:1787739064.073454:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 369 cancel update log [0x240000400:0x3:0x0].362: rc = 0 00000020:00080000:3.0:1787739064.073463:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 370 00010000:00010000:3.0:1787739064.073510:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad6fe lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ad6f7 expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.073523:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x1640014e (ffffa073068a9000) 00010000:00010000:3.0:1787739064.073528:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad72f lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.073538:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad72f lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.073546:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad72f lrc: 4/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.073572:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739064.073644:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+6:3652:x1874579665700864:12345-0@lo:103:ldlm_bl.0 Request processed in 4500us (4604us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.073716:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d84780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665700992:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:2.0:1787739064.073760:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739064.073814:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665700992 00000100:00100000:2.0:1787739064.073820:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665700992 00000100:00100000:3.0:1787739064.073907:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1536 00000100:00100000:2.0:1787739064.073931:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1339 00000100:00100000:1.0:1787739064.073939:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665700864:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739064.073941:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba7850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+7:3653:x1874579665700992:12345-0@lo:103:jbd2/dm-1-8.0 00000004:00080000:3.0:1787739064.073948:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 696 rpc_version 696 00010000:00010000:2.0:1787739064.073953:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000020:00100000:3.0:1787739064.073971:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00010000:00010000:2.0:1787739064.073972:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ad695 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x917910c6 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad68e expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.073982:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304d84b40 x1874579665701248/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739064.073999:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d84b40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665701248:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739064.074042:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787739064.074068:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x917910c6 (ffffa07403a6f000) 00000100:00100000:3.0:1787739064.074076:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241600 going to sleep for 11 seconds 00010000:00010000:2.0:1787739064.074077:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739064.074097:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad72f lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.074106:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad72f lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739064.074116:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba7850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+6:3653:x1874579665700992:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 176us (354us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.074123:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1339 00000100:00100000:2.0:1787739064.074179:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d84780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665700992:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:3.0:1787739064.074190:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665701376:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739064.074222:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739064.074249:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ea80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739064.074490:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665701248 00000100:00100000:1.0:1787739064.074495:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665701248 00000100:00100000:1.0:1787739064.074524:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665701376 00000100:00100000:1.0:1787739064.074527:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665701376 00000100:00100000:1.0:1787739064.074541:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3265 00000100:00100000:2.0:1787739064.074552:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3266 00000100:00100000:1.0:1787739064.074552:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d87480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665701248:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739064.074571:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665701376:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739064.074698:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d87480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665701248:12345-0@lo:1000:osp_up0-1.0 Request processed in 150us (654us total) trans 4294983410 rc 0/0 00000100:00100000:1.0:1787739064.074706:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3265 00000100:00100000:2.0:1787739064.074795:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665701376:12345-0@lo:1000:mdt00_000.0 Request processed in 233us (573us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.074803:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3266 00000100:00100000:3.0:1787739064.074884:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d84b40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665701248:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739064.074937:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665701376:0@lo:1000:mdt00_000.0 00010000:00010000:1.0:1787739064.074977:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad736 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x151f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.074989:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad736 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x151f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.075003:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad736 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x151f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739064.075031:0:6523:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad736 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x151f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.075045:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad728 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.075052:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad728 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.075058:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad728 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.075065:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad728 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.075071:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739064.075075:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad728 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.075083:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad728 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.075092:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074331c2000) 00010000:00010000:1.0:1787739064.075095:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739064.075098:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad728 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.075103:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad728 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.075115:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad72f lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.075122:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad72f lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.075126:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad72f lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.075131:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739064.075134:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad72f lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.075140:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad72f lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.075147:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x1640014e (ffffa073068a9000) 00010000:00010000:1.0:1787739064.075149:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739064.075152:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad72f lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.075158:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad72f lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.075194:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad721 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950dc expref: 9 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.075211:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad736 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x151f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950dc expref: 8 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.075222:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x151f:0x0].0x0 (ffffa07433235400) 00010000:00010000:1.0:1787739064.075224:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739064.075226:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d000, rc 0) 00000100:00100000:1.0:1787739064.075274:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305654000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17630:x1874579652489088:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 7827us (8461us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.075281:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23505 00010000:00010000:0.0:1787739064.075709:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad61e lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x17d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.075718:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad61e lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x17d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.075725:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad61e lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x17d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.075731:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aaa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739064.075734:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad61e lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x17d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.075740:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad61e lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x17d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.075749:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17d:0x0].0x0 (ffffa074331c2800) 00010000:00010000:0.0:1787739064.075751:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739064.075755:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad61e lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x17d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.075760:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ad61e lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x17d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.075774:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad625 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.075780:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad625 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.075786:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad625 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.075791:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739064.075793:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad625 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.075799:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad625 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.075806:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc6:0x0].0x0 (ffffa074331c2c00) 00010000:00010000:0.0:1787739064.075808:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739064.075811:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad625 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.075816:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ad625 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739064.078810:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739064.078865:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652489216 00000100:00100000:0.0:1787739064.078875:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652489216 00000100:00100000:0.0:1787739064.078944:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23506 00000100:00100000:0.0:1787739064.078954:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17630:x1874579652489216:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739064.078978:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739064.078999:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad73d lrc: 2/0,0 mode: --/CR res: [0x200000402:0x151f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0950e3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.079070:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad744 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x151f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.079084:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad744 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x151f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.079094:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad744 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x151f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739064.079188:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x151f:0x0]: returning size 4096 00000004:00200000:0.0:1787739064.079193:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739064.079230:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad73d lrc: 0/0,0 mode: --/CR res: [0x200000402:0x151f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950e3 expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.079248:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad744 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x151f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950e3 expref: 17 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.079256:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x151f:0x0].0x0 (ffffa07433235b00) 00010000:00010000:0.0:1787739064.079259:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739064.079263:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609aa00, rc 0) 00000100:00100000:0.0:1787739064.079327:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17630:x1874579652489216:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 376us (527us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739064.079335:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23506 00000100:00100000:2.0:1787739064.096580:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739064.096638:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652489344 00000100:00100000:0.0:1787739064.096647:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652489344 00000100:00100000:0.0:1787739064.096713:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23507 00000100:00100000:0.0:1787739064.096722:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17630:x1874579652489344:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739064.096744:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739064.096764:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad74b lrc: 2/0,0 mode: --/CR res: [0x200000400:0xc9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0950ea expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.096840:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad752 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xc9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.096854:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad752 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.096865:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad752 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739064.096910:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xc9:0x0]: returning size 4096 00000004:00200000:0.0:1787739064.096915:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739064.096948:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad74b lrc: 0/0,0 mode: --/CR res: [0x200000400:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950ea expref: 19 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.096967:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad752 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950ea expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.096976:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc9:0x0].0x0 (ffffa07433235300) 00010000:00010000:0.0:1787739064.096979:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739064.096982:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609ac00, rc 0) 00000100:00100000:0.0:1787739064.097053:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17630:x1874579652489344:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 332us (482us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739064.097061:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23507 00000100:00100000:1.0:1787739064.100599:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739064.100939:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652489472 00000100:00100000:3.0:1787739064.100949:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652489472 00000100:00100000:3.0:1787739064.101032:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23508 00000100:00100000:3.0:1787739064.101043:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17630:x1874579652489472:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739064.101064:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739064.101086:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad759 lrc: 2/0,0 mode: --/CR res: [0x240000401:0xc9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0950f1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.101142:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad760 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.101157:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ad71a lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad713 expref: 5 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.101166:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad760 lrc: 4/1,0 mode: --/PR res: [0x240000401:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.101198:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ad71a lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ad713 expref: 5 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.101211:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665701504:0@lo:104: 00000100:00100000:3.0:1787739064.101259:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739064.101402:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739064.101883:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665701504 00000100:00100000:1.0:1787739064.101889:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665701504 00000100:00100000:1.0:1787739064.101922:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1176 00000100:00100000:1.0:1787739064.101929:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042452c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6523:x1874579665701504:12345-0@lo:104: 00010000:00010000:1.0:1787739064.101942:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad713 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad71a expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.102117:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787739064.102125:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042452c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6523:x1874579665701504:12345-0@lo:104: Request processed in 196us (866us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.102132:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1176 00010000:00010000:1.0:1787739064.102156:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739064.102163:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad713 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad71a expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.102172:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739064.102177:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad713 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xc9:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ad71a expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739064.102192:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad713 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xc9:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ad71a expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.102219:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad713 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xc9:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad71a expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.102225:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739064.102255:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad713 lrc: 1/0,0 mode: --/PW res: [0x240000401:0xc9:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad71a expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.102320:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c800/0x7ee4453e4a1ad713 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xc9:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad71a expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.102341:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739064.102343:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739064.102344:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739064.102368:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787739064.102383:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce1e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665701632:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739064.102392:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665701504:0@lo:104: 00000100:00100000:2.0:1787739064.102442:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.102481:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665701632 00000100:00100000:1.0:1787739064.102486:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665701632 00010000:00010000:1.0:1787739064.102505:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ad71a lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ad713 expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739064.102537:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1537 00000100:00100000:1.0:1787739064.102545:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562cf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+6:3651:x1874579665701632:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739064.102554:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739064.103124:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad760 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1787739064.108985:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 371 batchid updated 00010000:00010000:1.0:1787739064.109122:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ad71a lrc: 0/0,0 mode: --/PW res: [0x240000401:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ad713 expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.109141:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc9:0x0].0x0 (ffffa07403a6fd00) 00010000:00010000:1.0:1787739064.109147:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad760 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.109157:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad760 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.109165:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad760 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.109198:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739064.109286:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562cf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665701632:12345-0@lo:103:ldlm_bl.0 Request processed in 6741us (6843us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.109297:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1537 00000100:00100000:1.0:1787739064.109352:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce1e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665701632:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739064.109454:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad760 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.109482:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad760 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:0.0:1787739064.109794:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 371 00000004:00080000:0.0:1787739064.109854:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa073068486c0 with version 697 00000020:00080000:0.0:1787739064.109867:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 371 cancel update log [0x200000401:0x3:0x0].364: rc = 0 00000020:00080000:0.0:1787739064.109944:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 371 cancel update log [0x240000400:0x3:0x0].363: rc = 0 00000020:00080000:0.0:1787739064.109957:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 371 00000004:00200000:3.0:1787739064.110634:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xc9:0x0]: returning size 4096 00000004:00200000:3.0:1787739064.110642:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739064.110775:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad759 lrc: 0/0,0 mode: --/CR res: [0x240000401:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950f1 expref: 9 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.110797:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad760 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950f1 expref: 8 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.110806:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc9:0x0].0x0 (ffffa07403a6fd00) 00010000:00010000:3.0:1787739064.110810:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739064.110813:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa200, rc 0) 00000100:00100000:3.0:1787739064.110871:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17630:x1874579652489472:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 9829us (10279us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739064.110880:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23508 00000004:00080000:3.0:1787739064.110912:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 697 rpc_version 697 00000020:00100000:3.0:1787739064.110940:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739064.110949:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304d852c0 x1874579665701888/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739064.110971:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665701888:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739064.111035:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739064.111075:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739064.113580:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665701888 00000100:00100000:0.0:1787739064.113593:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665701888 00000100:00100000:0.0:1787739064.113650:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3267 00000100:00100000:0.0:1787739064.113668:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665701888:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739064.114000:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665701888:12345-0@lo:1000:osp_up0-1.0 Request processed in 333us (2966us total) trans 4294983411 rc 0/0 00000100:00100000:0.0:1787739064.114019:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3267 00000100:00100000:3.0:1787739064.115730:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665701888:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739064.130921:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739064.130965:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739064.131037:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652489600 00000100:00100000:0.0:1787739064.131047:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652489600 00000100:00100000:0.0:1787739064.131116:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1340 00000100:00100000:0.0:1787739064.131127:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562e940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2408:x1874579652489600:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:0.0:1787739064.131143:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739064.131169:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ad736 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x151f:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950dc expref: 8 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.131188:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x151f:0x0].0x0 (ffffa07433235400) 00010000:00010000:0.0:1787739064.131197:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739064.131249:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562e940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652489600:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 125us (284us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739064.131257:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1340 00000100:00100000:3.0:1787739064.131704:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652489856 00000100:00100000:3.0:1787739064.131710:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652489856 00000100:00100000:3.0:1787739064.131755:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23509 00000100:00100000:3.0:1787739064.131763:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17630:x1874579652489856:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:3.0:1787739064.131810:0:6525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739064.131825:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad744 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x151f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950e3 expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.131845:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x151f:0x0].0x0 (ffffa07433235b00) 00010000:00010000:3.0:1787739064.131849:0:6525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739064.131894:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab400/0x7ee4453e4a1ad767 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.131902:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab400/0x7ee4453e4a1ad767 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.131911:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab400/0x7ee4453e4a1ad767 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.131933:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042612c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665702016:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1787739064.131979:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739064.132410:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739064.137380:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665702016 00000100:00100000:0.0:1787739064.137397:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665702016 00000100:00100000:0.0:1787739064.137532:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3268 00000100:00100000:0.0:1787739064.137544:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304261a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6525:x1874579665702016:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787739064.137566:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739064.137593:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad76e lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad767 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.137775:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad76e lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad767 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.137789:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad76e lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad767 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.137798:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9d00) 00010000:00010000:0.0:1787739064.137801:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739064.137804:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a000, rc 0) 00000100:00100000:0.0:1787739064.137924:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304261a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6525:x1874579665702016:12345-0@lo:101:mdt00_002.0 Request processed in 382us (5944us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739064.137932:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3268 00000100:00100000:3.0:1787739064.142523:0:6525:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968399 gen 2 00000004:00080000:3.0:1787739064.142534:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968394 committed transno 4294968399 00000020:00080000:3.0:1787739064.142541:0:6525:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa0730623d000 refcount 2 committed 0 result 0 batchid 380 00000020:00080000:3.0:1787739064.142545:0:6525:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2b00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730623d540 00000020:00080000:3.0:1787739064.142548:0:6525:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].377 00000020:00080000:3.0:1787739064.142551:0:6525:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2000 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306940c00 00000020:00080000:3.0:1787739064.142554:0:6525:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].377 00000020:00080000:1.0:1787739064.142601:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 380 committed batchid 379 00000020:00080000:1.0:1787739064.142665:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 379 00000004:00080000:3.0:1787739064.143455:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968396 committed transno 4294968399 00000004:00080000:3.0:1787739064.143478:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968397 committed transno 4294968399 00000004:00080000:3.0:1787739064.143486:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968399 committed transno 4294968399 00000100:00100000:3.0:1787739064.143506:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042612c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665702016:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739064.143533:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab400/0x7ee4453e4a1ad767 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad76e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.143549:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab400/0x7ee4453e4a1ad767 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad76e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739064.143568:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739064.143763:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ad775 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.143772:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ad775 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.143782:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ad775 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.143802:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042612c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665702144:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1787739064.143863:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739064.148348:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739064.155371:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665702144 00000100:00100000:1.0:1787739064.155387:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665702144 00000100:00100000:1.0:1787739064.155468:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3269 00000100:00100000:1.0:1787739064.155480:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304263840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6525:x1874579665702144:12345-0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787739064.155501:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739064.155525:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad77c lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad775 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.155556:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad77c lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad775 expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.155572:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad77c lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad775 expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.155586:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x1640014e (ffffa07433235800) 00010000:00010000:1.0:1787739064.155590:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739064.155593:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c800, rc 0) 00000100:00100000:1.0:1787739064.155684:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304263840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6525:x1874579665702144:12345-0@lo:101:mdt00_002.0 Request processed in 205us (11820us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.155691:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3269 00000100:00100000:1.0:1787739064.155752:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042612c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665702144:0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787739064.155770:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ad775 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad77c expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.155781:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ad775 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad77c expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739064.155804:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:1.0:1787739064.155852:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665702272:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739064.155887:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.155910:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623da80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739064.155927:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665702272 00000100:00100000:1.0:1787739064.155931:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665702272 00000100:00100000:1.0:1787739064.155956:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3270 00000100:00100000:1.0:1787739064.155962:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6525:x1874579665702272:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739064.158214:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6525:x1874579665702272:12345-0@lo:1000:mdt00_002.0 Request processed in 2244us (2319us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.158226:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3270 00000100:00100000:1.0:1787739064.158304:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665702272:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739064.158373:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665702400:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739064.158406:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.158449:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623da80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739064.158466:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665702400 00000100:00100000:1.0:1787739064.158471:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665702400 00000100:00100000:1.0:1787739064.158504:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3271 00000100:00100000:1.0:1787739064.158512:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6525:x1874579665702400:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739064.158595:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:6525:x1874579665702400:12345-0@lo:1000:mdt00_002.0 Request processed in 84us (189us total) trans 0 rc 1/1 00000100:00100000:1.0:1787739064.158601:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3271 00000100:00100000:1.0:1787739064.158626:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665702400:0@lo:1000:mdt00_002.0 00010000:00010000:1.0:1787739064.158677:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad783 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.158687:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad783 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.158697:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad783 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739064.158709:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665702528:0@lo:101:mdt00_002.0 00000100:00100000:1.0:1787739064.158730:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.158745:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739064.158756:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665702528 00000100:00100000:1.0:1787739064.158759:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665702528 00000100:00100000:1.0:1787739064.158780:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3272 00000100:00100000:1.0:1787739064.158785:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6525:x1874579665702528:12345-0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787739064.158795:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739064.158811:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad78a lrc: 2/0,0 mode: --/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad783 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.158833:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad78a lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad783 expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.158846:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad78a lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad783 expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.158853:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x151f:0x0].0x0 (ffffa07433235d00) 00010000:00010000:1.0:1787739064.158856:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739064.158859:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1de00, rc 0) 00000100:00100000:1.0:1787739064.158886:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6525:x1874579665702528:12345-0@lo:101:mdt00_002.0 Request processed in 101us (155us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.158892:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3272 00000100:00100000:1.0:1787739064.158916:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665702528:0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787739064.158928:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad783 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad78a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.158937:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad783 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad78a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.158957:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad791 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.158965:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad791 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.158972:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad791 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.159214:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad798 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.159227:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad752 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950ea expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.159235:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad798 lrc: 4/0,1 mode: --/EX res: [0x200000400:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.159307:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad752 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950ea expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.159325:0:6525:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad752 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950ea expref: 17 pid: 13793 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739064.159335:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6525:1874579665702656:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739064.159450:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739064.169590:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739064.169621:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652489984 00000100:00100000:3.0:1787739064.169629:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652489984 00010000:00010000:3.0:1787739064.169665:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad752 lrc: 5/0,0 mode: PR/PR res: [0x200000400:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950ea expref: 18 pid: 13793 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.169699:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1538 00000100:00100000:3.0:1787739064.169709:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042612c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2409:x1874579652489984:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739064.169723:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739064.169730:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad752 lrc: 5/0,0 mode: PR/PR res: [0x200000400:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950ea expref: 18 pid: 13793 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.169794:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad752 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950ea expref: 18 pid: 13793 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.169820:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc9:0x0].0x0 (ffffa07433235300) 00010000:00010000:3.0:1787739064.169825:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad798 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.169835:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad798 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.169842:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad798 lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.169861:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739064.169915:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042612c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2409:x1874579652489984:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 207us (334us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739064.169923:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1538 00010000:00010000:3.0:1787739064.170191:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad752 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950ea expref: 17 pid: 13793 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.170208:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce1e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6525:1874579665702656:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739064.170277:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad798 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.170288:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad798 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.170293:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad798 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.170335:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad79f lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.170343:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad79f lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.170352:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad79f lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.170370:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d87840 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665702784:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1787739064.170406:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739064.170448:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e780 going to sleep for 11 seconds 00000100:00100000:1.0:1787739064.170595:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665702784 00000100:00100000:1.0:1787739064.170600:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665702784 00000100:00100000:1.0:1787739064.170630:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3273 00000100:00100000:1.0:1787739064.170636:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306390f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6525:x1874579665702784:12345-0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787739064.170652:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739064.170666:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad7a6 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad79f expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.170687:0:6529:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad760 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950f1 expref: 6 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.170701:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad7a6 lrc: 4/0,0 mode: --/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad79f expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.170722:0:6529:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad760 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950f1 expref: 6 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.170735:0:6529:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad760 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950f1 expref: 6 pid: 6523 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739064.170746:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce2580 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665702912:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739064.170795:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623da80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739064.185055:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce2580 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665702912:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739064.185095:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad7a6 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad79f expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.185109:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc9:0x0].0x0 (ffffa07403a6fd00) 00010000:00010000:0.0:1787739064.185112:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739064.185117:0:6529:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad7a6 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad79f expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.185128:0:6529:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad7a6 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad79f expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.185140:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1da00, rc 0) 00000100:00100000:0.0:1787739064.185201:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306390f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:6525:x1874579665702784:12345-0@lo:101:mdt00_002.0 Request processed in 14565us (14793us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739064.185210:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3273 00000100:00100000:2.0:1787739064.185619:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739064.187824:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652490112 00000100:00100000:1.0:1787739064.187838:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652490112 00010000:00010000:1.0:1787739064.187884:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad760 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950f1 expref: 7 pid: 6523 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739064.187922:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1539 00000100:00100000:1.0:1787739064.187933:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2409:x1874579652490112:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739064.187950:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739064.187956:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad760 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950f1 expref: 7 pid: 6523 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.187970:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad760 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950f1 expref: 7 pid: 6523 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.187992:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad760 lrc: 0/0,0 mode: --/PR res: [0x240000401:0xc9:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950f1 expref: 7 pid: 6523 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.188005:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc9:0x0].0x0 (ffffa07403a6fd00) 00010000:00010000:1.0:1787739064.192751:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad7a6 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad79f expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.192769:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad7a6 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad79f expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.192777:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad7a6 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad79f expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.192827:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad7a6 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad79f expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739064.192842:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce03c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665703040:0@lo:105: 00000100:00100000:1.0:1787739064.192907:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.192957:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d600 going to sleep for 11 seconds 00000100:00100000:1.0:1787739064.192984:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665703040 00000100:00100000:1.0:1787739064.192990:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665703040 00000100:00100000:1.0:1787739064.193047:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1177 00000100:00100000:1.0:1787739064.193056:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce1a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665703040:12345-0@lo:105: 00010000:00010000:1.0:1787739064.193069:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad79f lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad7a6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.193078:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad79f lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad7a6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.193093:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad79f lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad7a6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.193101:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad79f lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad7a6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.193108:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad79f lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad7a6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.193116:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad79f lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ad7a6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.193131:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069ab800) 00000100:00100000:1.0:1787739064.193173:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce1a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665703040:12345-0@lo:105: Request processed in 118us (265us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.193180:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1177 00000100:00100000:1.0:1787739064.193220:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce03c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665703040:0@lo:105: 00010000:00010000:1.0:1787739064.193233:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739064.193531:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2409:x1874579652490112:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 5597us (7911us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.193539:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1539 00000100:00100000:3.0:1787739064.193935:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d87840 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665702784:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739064.193952:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad79f lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad7a6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.194989:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073063921c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665703168:0@lo:403:mdt00_002.0 00000100:00100000:3.0:1787739064.195060:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739064.195088:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e780 going to sleep for 11 seconds 00000100:00100000:1.0:1787739064.195162:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665703168 00000100:00100000:1.0:1787739064.195170:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665703168 00000100:00100000:1.0:1787739064.195222:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3274 00000100:00100000:1.0:1787739064.195230:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306392d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:6525:x1874579665703168:12345-0@lo:403:mdt00_002.0 00000100:00100000:1.0:1787739064.196920:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306392d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:6525:x1874579665703168:12345-0@lo:403:mdt00_002.0 Request processed in 1683us (1852us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.196934:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3274 00000100:00100000:3.0:1787739064.197230:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073063921c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665703168:0@lo:403:mdt00_002.0 00000040:00080000:3.0:1787739064.198249:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].379, 664 off579336 00000040:00080000:3.0:1787739064.198527:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].379, 744 off629104 00000004:00080000:3.0:1787739064.198535:0:6525:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730610e780 with version 722 00000004:00080000:1.0:1787739064.204446:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 722 rpc_version 722 00000020:00100000:1.0:1787739064.204491:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739064.204503:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304ce03c0 x1874579665703424/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2064/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739064.204528:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce03c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665703424:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739064.204602:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.204641:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d600 going to sleep for 11 seconds 00000100:00100000:1.0:1787739064.205085:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665703424 00000100:00100000:1.0:1787739064.205094:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665703424 00000100:00100000:1.0:1787739064.205152:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3275 00000100:00100000:1.0:1787739064.205162:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665703424:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739064.206637:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665703424:12345-0@lo:1000:osp_up1-0.0 Request processed in 1471us (2030us total) trans 4294968400 rc 0/0 00000100:00100000:1.0:1787739064.206649:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3275 00000100:00100000:0.0:1787739064.207041:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce03c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665703424:0@lo:1000:osp_up1-0.0 00000004:00200000:3.0:1787739064.207099:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x151f:0x0]: returning size 4096 00010000:00010000:3.0:1787739064.207128:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad798 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.207137:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad798 lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.207145:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad798 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.207152:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739064.207156:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad798 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.207163:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad798 lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.207176:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc9:0x0].0x0 (ffffa07433235300) 00010000:00010000:3.0:1787739064.207179:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739064.207183:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad798 lrc: 2/0,0 mode: --/EX res: [0x200000400:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.207188:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad798 lrc: 0/0,0 mode: --/EX res: [0x200000400:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.207201:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad79f lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad7a6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.207208:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad79f lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad7a6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.207214:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad79f lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad7a6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.207240:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739064.207249:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xc9:0x0] 00010000:00010000:3.0:1787739064.207256:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad791 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.207344:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739064.207351:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad79f lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad7a6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.207358:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739064.207362:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad79f lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad7a6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739064.207369:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad79f lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad7a6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.207401:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad79f lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad7a6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.207408:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739064.207445:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad79f lrc: 1/0,0 mode: --/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad7a6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.207452:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad79f lrc: 0/0,0 mode: --/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad7a6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.207466:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739064.207468:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739064.207470:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739064.207478:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787739064.207490:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305654f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665703552:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739064.207529:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739064.207569:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665703552 00000100:00100000:0.0:1787739064.207574:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665703552 00000100:00100000:0.0:1787739064.207615:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1341 00000100:00100000:0.0:1787739064.207624:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305655e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+11:3650:x1874579665703552:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739064.207636:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739064.207741:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad791 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.207748:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad791 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.207754:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ca00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739064.207757:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad791 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.207764:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad791 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.207773:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x151f:0x0].0x0 (ffffa07433235b00) 00010000:00010000:3.0:1787739064.207775:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739064.207778:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad791 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.207784:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad791 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.207794:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad783 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad78a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.207802:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad783 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad78a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.207809:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad783 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad78a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.207828:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739064.207840:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739064.207844:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab400/0x7ee4453e4a1ad767 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad76e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.207851:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab400/0x7ee4453e4a1ad767 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad76e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.207858:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab400/0x7ee4453e4a1ad767 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad76e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.207869:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:3.0:1787739064.207874:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739064.207878:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ad775 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad77c expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.207884:0:6525:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ad775 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad77c expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.207945:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17630:x1874579652489856:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 76181us (77032us total) trans 4294983412 rc 0/0 00000100:00100000:3.0:1787739064.207953:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23509 00010000:00010000:3.0:1787739064.207978:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787739064.207982:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad783 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad78a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.207990:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739064.207994:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad783 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad78a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739064.208001:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad783 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad78a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.208029:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad783 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad78a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.208036:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739064.208054:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad783 lrc: 1/0,0 mode: --/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad78a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.208060:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1d600/0x7ee4453e4a1ad783 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad78a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.208071:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739064.208074:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab400/0x7ee4453e4a1ad767 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad76e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.208080:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739064.208083:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab400/0x7ee4453e4a1ad767 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad76e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739064.208089:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab400/0x7ee4453e4a1ad767 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad76e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.208102:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab400/0x7ee4453e4a1ad767 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad76e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.208112:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739064.208118:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab400/0x7ee4453e4a1ad767 lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad76e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.208124:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab400/0x7ee4453e4a1ad767 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad76e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.208135:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739064.208136:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739064.208138:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739064.208144:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739064.208152:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073063912c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665703808:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739064.208181:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739064.208212:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787739064.208218:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073063921c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665703680:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739064.208241:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.208379:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665703808 00000100:00100000:1.0:1787739064.208383:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665703808 00000100:00100000:1.0:1787739064.208407:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665703680 00000100:00100000:1.0:1787739064.208410:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665703680 00000100:00100000:1.0:1787739064.208449:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1342 00000100:00100000:1.0:1787739064.208457:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073063930c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+12:3653:x1874579665703808:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739064.208465:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739064.208478:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad76e lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad767 expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.208489:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9d00) 00010000:00010000:1.0:1787739064.208497:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739064.208597:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad7a6 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xc9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad79f expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.208611:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc9:0x0].0x0 (ffffa07403a6fd00) 00010000:00010000:0.0:1787739064.208617:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739064.208648:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305655e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+10:3650:x1874579665703552:12345-0@lo:103:ldlm_bl.0 Request processed in 1026us (1119us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739064.208719:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1341 00000100:00100000:0.0:1787739064.208736:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1343 00000100:00100000:0.0:1787739064.208743:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba5850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+9:3653:x1874579665703680:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739064.208751:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739064.208762:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad78a lrc: 0/0,0 mode: --/EX res: [0x200000402:0x151f:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad783 expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.208773:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x151f:0x0].0x0 (ffffa07433235d00) 00010000:00010000:0.0:1787739064.208779:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739064.212154:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073063930c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+8:3653:x1874579665703808:12345-0@lo:103:ldlm_bl.0 Request processed in 3686us (3960us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.212169:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1342 00000100:00100000:0.0:1787739064.218448:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba5850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+7:3653:x1874579665703680:12345-0@lo:103:ldlm_bl.0 Request processed in 9694us (10194us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739064.218466:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1343 00000100:00100000:0.0:1787739064.218575:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305654f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665703552:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739064.219484:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073063921c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665703680:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739064.219543:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073063912c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665703808:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739064.256585:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739064.260833:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787739064.260853:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306390b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665703936:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1787739064.260917:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739064.260990:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665703936 00000100:00100000:3.0:1787739064.260997:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665703936 00000100:00100000:3.0:1787739064.261049:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3276 00000100:00100000:3.0:1787739064.261056:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306392940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:3651:x1874579665703936:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1787739064.261090:0:12410:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1445072896 avail 1298833408 00000100:00100000:3.0:1787739064.261132:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306392940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:3651:x1874579665703936:12345-0@lo:41:osp-pre-1-0.0 Request processed in 76us (215us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739064.261139:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3276 00000100:00100000:3.0:1787739064.261183:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306390b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665703936:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1787739064.265651:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652490240 00000100:00100000:0.0:1787739064.265863:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652490240 00000100:00100000:0.0:1787739064.266049:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23510 00000100:00100000:0.0:1787739064.266060:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306391e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17631:x1874579652490240:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739064.266085:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739064.266106:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad7ad lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0950f8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.266170:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad7b4 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.266184:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad7b4 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.266195:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad7b4 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.266211:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad7bb lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.266217:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad7bb lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.266224:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad7bb lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.266274:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad7b4 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.266281:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad7b4 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.266287:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad7b4 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.266292:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad7b4 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.266297:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739064.266301:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad7b4 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.266308:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad7b4 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.266319:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9f00) 00010000:00010000:0.0:1787739064.266322:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739064.266326:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad7b4 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.266332:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad7b4 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.266343:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad7bb lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.266349:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad7bb lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.266355:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad7bb lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.266360:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739064.266363:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad7bb lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.266379:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad7bb lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.266386:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x61ea7fc2 (ffffa073068a9400) 00010000:00010000:0.0:1787739064.266389:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739064.266392:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad7bb lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.266397:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ad7bb lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.266446:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad7ad lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950f8 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.266456:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9f00) 00010000:00010000:0.0:1787739064.266458:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739064.266461:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad7ad lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950f8 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.266469:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609be00, rc 0) 00000100:00100000:0.0:1787739064.266532:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306391e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17631:x1874579652490240:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 474us (9957us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739064.266539:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23510 00000100:00100000:2.0:1787739064.275779:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739064.275939:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652490368 00000100:00100000:3.0:1787739064.275949:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652490368 00000100:00100000:3.0:1787739064.276027:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23511 00000100:00100000:3.0:1787739064.276038:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17631:x1874579652490368:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739064.276060:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739064.276083:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad7c2 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0950ff expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.276143:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ad7c9 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.276157:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ad7c9 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.276168:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ad7c9 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739064.276215:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x5:0x0]: returning size 4096 00000004:00200000:3.0:1787739064.276220:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739064.276256:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad7c2 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950ff expref: 8 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.277578:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ad7c9 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950ff expref: 7 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.277593:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b000) 00010000:00010000:3.0:1787739064.277596:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739064.277600:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa600, rc 0) 00000100:00100000:3.0:1787739064.277665:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17631:x1874579652490368:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 1629us (1896us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739064.277673:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23511 00000100:00100000:0.0:1787739064.288938:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739064.289078:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652490624 00000100:00100000:3.0:1787739064.289090:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652490624 00000100:00100000:3.0:1787739064.289152:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23512 00000100:00100000:3.0:1787739064.289163:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305654f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17631:x1874579652490624:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:3.0:1787739064.289311:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306390b40 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665704064:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739064.289380:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739064.289508:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ef00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739064.290928:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665704064 00000100:00100000:2.0:1787739064.290940:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665704064 00000100:00100000:2.0:1787739064.296169:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3277 00000100:00100000:2.0:1787739064.296188:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306393c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6525:x1874579665704064:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739064.296486:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306393c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6525:x1874579665704064:12345-0@lo:1000:mdt00_002.0 Request processed in 304us (7120us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.296494:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3277 00000100:00100000:0.0:1787739064.296550:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306390b40 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665704064:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739064.296620:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305655e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665704192:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739064.296671:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739064.299536:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665704192 00000100:00100000:2.0:1787739064.299548:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665704192 00000100:00100000:0.0:1787739064.299580:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306848240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739064.299591:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3278 00000100:00100000:2.0:1787739064.299599:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073056552c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6525:x1874579665704192:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739064.299732:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073056552c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6525:x1874579665704192:12345-0@lo:1000:mdt00_002.0 Request processed in 134us (3060us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739064.299741:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3278 00000100:00100000:0.0:1787739064.299847:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305655e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665704192:0@lo:1000:mdt00_002.0 00010000:00010000:0.0:1787739064.299927:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad7d0 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.299937:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad7d0 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.299948:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad7d0 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739064.299969:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305655e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665704320:0@lo:101:mdt00_002.0 00000100:00100000:0.0:1787739064.300078:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739064.300104:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306848f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739064.300156:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665704320 00000100:00100000:2.0:1787739064.300160:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665704320 00000100:00100000:2.0:1787739064.300179:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3279 00000100:00100000:2.0:1787739064.300185:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305657c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6525:x1874579665704320:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787739064.300194:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739064.300209:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad7d7 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad7d0 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.300234:0:6529:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad6e2 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950d5 expref: 16 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.300250:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad7d7 lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad7d0 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.300345:0:6529:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad6e2 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950d5 expref: 16 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.300358:0:6529:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad6e2 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950d5 expref: 16 pid: 13793 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739064.300379:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562e940 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6529:1874579665704448:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739064.301062:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413900 going to sleep for 11 seconds 00000100:00100000:1.0:1787739064.307605:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739064.307910:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562e940 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6529:1874579665704448:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739064.307944:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad7d7 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad7d0 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.307958:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9f00) 00010000:00010000:2.0:1787739064.307961:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739064.307965:0:6529:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad7d7 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad7d0 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.307986:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c000, rc 0) 00000100:00100000:2.0:1787739064.308057:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305657c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6525:x1874579665704320:12345-0@lo:101:mdt00_002.0 Request processed in 7872us (7978us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.308065:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3279 00000100:00100000:1.0:1787739064.308186:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305655e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665704320:0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787739064.308203:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad7d0 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad7d7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.308213:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad7d0 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad7d7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.308447:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652490752 00000100:00100000:3.0:1787739064.308454:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652490752 00010000:00010000:3.0:1787739064.308484:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad6e2 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950d5 expref: 17 pid: 13793 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.308522:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1540 00000100:00100000:3.0:1787739064.308531:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce0f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652490752:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739064.308546:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739064.308551:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad6e2 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950d5 expref: 17 pid: 13793 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.308563:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad6e2 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950d5 expref: 17 pid: 13793 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.308583:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad6e2 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950d5 expref: 17 pid: 13793 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.308597:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9f00) 00010000:00010000:3.0:1787739064.308601:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad7d7 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad7d0 expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.308609:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad7d7 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad7d0 expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.308617:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad7d7 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad7d0 expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.308646:0:10340:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad7d7 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad7d0 expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.308671:0:10340:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:10340:1874579665704576:0@lo:105: 00000100:00100000:3.0:1787739064.308819:0:10340:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739064.308866:0:10340:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739064.310470:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665704576 00000100:00100000:3.0:1787739064.310484:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665704576 00000100:00100000:3.0:1787739064.310541:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1178 00000100:00100000:3.0:1787739064.310552:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073063903c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:10340:x1874579665704576:12345-0@lo:105: 00010000:00010000:3.0:1787739064.310567:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad7d0 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad7d7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.310579:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad7d0 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad7d7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.310590:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad7d0 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad7d7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.310599:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad7d0 lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad7d7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.310608:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad7d0 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad7d7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.310616:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad7d0 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ad7d7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.310637:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609be00) 00000100:00100000:3.0:1787739064.311404:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073063903c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:10340:x1874579665704576:12345-0@lo:105: Request processed in 853us (2583us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739064.311429:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1178 00000100:00100000:3.0:1787739064.311485:0:10340:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:10340:1874579665704576:0@lo:105: 00010000:00010000:3.0:1787739064.311503:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739064.311545:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce0f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:2409:x1874579652490752:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 3014us (3948us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739064.311554:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1540 00010000:00010000:1.0:1787739064.316105:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad7d0 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad7d7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.316123:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad7d0 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad7d7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739064.316160:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739064.316210:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad7de lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.316217:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad7de lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.316229:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad7de lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739064.316251:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce2580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665704704:0@lo:101:mdt00_002.0 00000100:00100000:1.0:1787739064.316331:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739064.316587:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665704704 00000100:00100000:0.0:1787739064.316594:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665704704 00000100:00100000:0.0:1787739064.316647:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3280 00000100:00100000:0.0:1787739064.316663:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6525:x1874579665704704:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787739064.316679:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739064.316703:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad7e5 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad7de expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.316729:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad7e5 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad7de expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.316748:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad7e5 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad7de expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.316757:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x61ea7fc2 (ffffa073068a9600) 00010000:00010000:0.0:1787739064.316760:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739064.316763:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b200, rc 0) 00000100:00100000:0.0:1787739064.316800:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6525:x1874579665704704:12345-0@lo:101:mdt00_002.0 Request processed in 145us (470us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739064.316807:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3280 00000100:00100000:1.0:1787739064.317000:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dc00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739064.317031:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce2580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665704704:0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787739064.317047:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad7de lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad7e5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.317057:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad7de lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad7e5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739064.317070:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:1.0:1787739064.317101:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce2580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665704832:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739064.317133:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.317163:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739064.317282:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665704832 00000100:00100000:3.0:1787739064.317287:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665704832 00000100:00100000:3.0:1787739064.317322:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3281 00000100:00100000:3.0:1787739064.317329:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6525:x1874579665704832:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739064.317410:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6525:x1874579665704832:12345-0@lo:1000:mdt00_002.0 Request processed in 82us (276us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739064.317436:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3281 00000100:00100000:1.0:1787739064.317625:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce2580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665704832:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739064.317667:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce2580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665704960:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739064.318057:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.318163:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739064.322264:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665704960 00000100:00100000:3.0:1787739064.322371:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665704960 00000100:00100000:3.0:1787739064.322521:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3282 00000100:00100000:3.0:1787739064.322533:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6525:x1874579665704960:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739064.322810:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6525:x1874579665704960:12345-0@lo:1000:mdt00_002.0 Request processed in 279us (4752us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739064.322819:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3282 00000100:00100000:1.0:1787739064.327585:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce2580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665704960:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739064.327872:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356acf00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665705088:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739064.327938:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.328049:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739064.328161:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665705088 00000100:00100000:3.0:1787739064.328168:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665705088 00000100:00100000:3.0:1787739064.328224:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3283 00000100:00100000:3.0:1787739064.328233:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6525:x1874579665705088:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739064.328343:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6525:x1874579665705088:12345-0@lo:1000:mdt00_002.0 Request processed in 110us (405us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739064.328351:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3283 00000100:00100000:1.0:1787739064.328811:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356acf00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665705088:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739064.328865:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356acf00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665705216:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739064.328894:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.328919:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739064.328963:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665705216 00000100:00100000:0.0:1787739064.328969:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665705216 00000100:00100000:0.0:1787739064.329022:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3284 00000100:00100000:0.0:1787739064.329030:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6525:x1874579665705216:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739064.329098:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6525:x1874579665705216:12345-0@lo:1000:mdt00_002.0 Request processed in 69us (202us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739064.329104:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3284 00000100:00100000:1.0:1787739064.329357:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356acf00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665705216:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739064.329545:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356acf00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665705344:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739064.329575:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.329634:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739064.329785:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665705344 00000100:00100000:3.0:1787739064.329790:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665705344 00000100:00100000:3.0:1787739064.329858:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3285 00000100:00100000:3.0:1787739064.329866:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356afc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6525:x1874579665705344:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739064.329951:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356afc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6525:x1874579665705344:12345-0@lo:1000:mdt00_002.0 Request processed in 86us (375us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739064.329958:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3285 00000100:00100000:2.0:1787739064.330034:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356acf00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665705344:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739064.330079:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562e940 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665705472:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739064.330107:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739064.330230:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c4136c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739064.330256:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665705472 00000100:00100000:1.0:1787739064.330269:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665705472 00000100:00100000:1.0:1787739064.330291:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3286 00000100:00100000:1.0:1787739064.330297:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6525:x1874579665705472:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739064.330351:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6525:x1874579665705472:12345-0@lo:1000:mdt00_002.0 Request processed in 54us (242us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.330357:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3286 00000100:00100000:1.0:1787739064.330400:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562e940 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665705472:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739064.330455:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356acf00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665705600:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739064.330481:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.330496:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623dc00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739064.330507:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665705600 00000100:00100000:1.0:1787739064.330510:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665705600 00000100:00100000:1.0:1787739064.330526:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3287 00000100:00100000:1.0:1787739064.330531:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6525:x1874579665705600:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739064.330576:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6525:x1874579665705600:12345-0@lo:1000:mdt00_002.0 Request processed in 44us (94us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.330583:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3287 00000100:00100000:1.0:1787739064.330606:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356acf00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665705600:0@lo:1000:mdt00_002.0 00000040:00080000:1.0:1787739064.332155:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].365, 2168 off554400 00000040:00080000:1.0:1787739064.332247:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].366, 2368 off607008 00000004:00080000:1.0:1787739064.332255:0:6525:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730623dcc0 with version 698 00000004:00080000:1.0:1787739064.332347:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 698 rpc_version 698 00000020:00100000:1.0:1787739064.332830:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739064.332841:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074356acf00 x1874579665705856/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739064.332863:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356acf00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665705856:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739064.332927:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.332954:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739064.333067:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665705856 00000100:00100000:1.0:1787739064.333073:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665705856 00000100:00100000:1.0:1787739064.333109:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3288 00000100:00100000:1.0:1787739064.333116:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665705856:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739064.334235:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:7467:x1874579665705856:12345-0@lo:1000:osp_up0-1.0 Request processed in 1115us (1303us total) trans 4294983413 rc 0/0 00000100:00100000:1.0:1787739064.334246:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3288 00000100:00100000:1.0:1787739064.334555:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356acf00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665705856:0@lo:1000:osp_up0-1.0 00010000:00010000:3.0:1787739064.334921:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad7ec lrc: 3/0,1 mode: --/PW res: [0x240000402:0x17f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.334936:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad7ec lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.334951:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad7ec lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.334969:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad7f3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xc7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.335051:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad7f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.335058:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad7f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.335089:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad7fa lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.335095:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad7fa lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.335104:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad7fa lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.335120:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665705984:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1787739064.335157:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739064.335230:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739064.335328:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665705984 00000100:00100000:2.0:1787739064.335334:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665705984 00000100:00100000:2.0:1787739064.335567:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3289 00000100:00100000:2.0:1787739064.335577:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306390b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6525:x1874579665705984:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787739064.335594:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739064.335611:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad801 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xc7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad7fa expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.335631:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad801 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad7fa expref: 12 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.335642:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad801 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad7fa expref: 12 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.335650:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc7:0x0].0x0 (ffffa07403a6f000) 00010000:00010000:2.0:1787739064.335665:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739064.335668:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d800, rc 0) 00000100:00100000:2.0:1787739064.335722:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306390b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:6525:x1874579665705984:12345-0@lo:101:mdt00_002.0 Request processed in 147us (564us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.335729:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3289 00000100:00100000:3.0:1787739064.335789:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665705984:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739064.335804:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad7fa lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad801 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.335813:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad7fa lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad801 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739064.335839:0:6525:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003fcce350 reply state 000000000b01284f transno 4294968401 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad7f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.335850:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad7f3 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.335857:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc7:0x0].0x0 (ffffa0730608ba00) 00010000:00010000:3.0:1787739064.335860:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739064.335862:0:6525:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787739064.335867:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad7fa lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad801 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.335875:0:6525:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad7fa lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad801 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739064.335886:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xc7:0x0] 00000004:00010000:3.0:1787739064.335891:0:6525:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003fcce350 reply state 000000000b01284f transno 4294968401 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad7ec lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x17f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.335900:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad7ec lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x17f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.335906:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17f:0x0].0x0 (ffffa0730608b700) 00010000:00010000:3.0:1787739064.335908:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739064.335910:0:6525:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:3.0:1787739064.335915:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17f:0x0]: returning size 4096 00000004:00080000:3.0:1787739064.335922:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739064.335930:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad7d0 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad7d7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.335943:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad7d0 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad7d7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.335949:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad7d0 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad7d7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.335966:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739064.335971:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739064.335986:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad7de lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad7e5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.335994:0:6525:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad7de lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad7e5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.336072:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305654f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17631:x1874579652490624:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 46909us (47142us total) trans 4294968401 rc 0/0 00000100:00100000:3.0:1787739064.336079:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23512 00010000:00010000:1.0:1787739064.337914:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739064.337926:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad7d0 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad7d7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.337939:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609be00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739064.337946:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad7d0 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad7d7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739064.337956:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad7d0 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad7d7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.338072:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad7d0 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad7d7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.338080:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739064.338114:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad7d0 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad7d7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.338121:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1ad7d0 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad7d7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.338138:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739064.338140:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739064.338141:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739064.338149:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787739064.338166:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665706112:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739064.338225:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.338584:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665706112 00000100:00100000:1.0:1787739064.338592:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665706112 00000100:00100000:1.0:1787739064.338640:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1344 00000100:00100000:1.0:1787739064.338650:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb5680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+12:3650:x1874579665706112:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739064.338902:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739064.338924:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad7d7 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad7d0 expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.338940:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9f00) 00010000:00010000:1.0:1787739064.338949:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739064.339113:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb5680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+11:3650:x1874579665706112:12345-0@lo:103:ldlm_bl.0 Request processed in 465us (887us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.339121:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1344 00000100:00100000:3.0:1787739064.339346:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665706112:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739064.360342:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739064.360476:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652491008 00000100:00100000:2.0:1787739064.360487:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652491008 00000100:00100000:2.0:1787739064.360551:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23513 00000100:00100000:2.0:1787739064.360563:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17632:x1874579652491008:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:2.0:1787739064.360864:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17632:x1874579652491008:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 302us (528us total) trans 0 rc -17/-17 00000100:00100000:2.0:1787739064.360871:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23513 00000100:00100000:0.0:1787739064.387570:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739064.387636:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652491136 00000100:00100000:2.0:1787739064.387646:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652491136 00000100:00100000:2.0:1787739064.387705:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23514 00000100:00100000:2.0:1787739064.387716:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305656d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17633:x1874579652491136:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739064.387736:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739064.387760:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad808 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095106 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.387823:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad80f lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.387836:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad80f lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.387860:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad80f lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.387870:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad816 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.387880:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad7e5 lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad7de expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.387889:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad816 lrc: 4/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.387926:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad7e5 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ad7de expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739064.387942:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562e940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665706240:0@lo:104: 00000100:00100000:2.0:1787739064.387993:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739064.388049:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739064.388065:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665706240 00000100:00100000:2.0:1787739064.388069:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665706240 00000100:00100000:2.0:1787739064.388097:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1179 00000100:00100000:2.0:1787739064.388104:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f75a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665706240:12345-0@lo:104: 00010000:00010000:2.0:1787739064.388115:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad7de lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad7e5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.388163:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739064.388169:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f75a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665706240:12345-0@lo:104: Request processed in 66us (177us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.388175:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1179 00010000:00010000:2.0:1787739064.388199:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739064.388205:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad7de lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad7e5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.388214:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739064.388219:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad7de lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ad7e5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739064.388228:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad7de lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ad7e5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.388256:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad7de lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad7e5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.388263:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739064.388281:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad7de lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad7e5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.388290:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d800/0x7ee4453e4a1ad7de lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad7e5 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.388309:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739064.388311:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739064.388313:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739064.388323:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060061c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665706368:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739064.388352:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739064.388398:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562e940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665706240:0@lo:104: 00010000:00010000:2.0:1787739064.388444:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad816 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739064.388624:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665706368 00000100:00100000:0.0:1787739064.388629:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665706368 00010000:00010000:0.0:1787739064.388652:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad7e5 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ad7de expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739064.388687:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1541 00000100:00100000:0.0:1787739064.388695:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+11:3652:x1874579665706368:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739064.388706:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739064.398097:0:6502:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ad775 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad77c expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739064.398121:0:6502:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ad775 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad77c expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.398177:0:6502:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ad775 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad77c expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.398186:0:6502:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739064.398231:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1ad775 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad77c expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787739064.398252:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983413 is committed 00000001:00080000:0.0:1787739064.398272:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983412 is committed 00000100:00100000:3.0:1787739064.398274:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000020:00080000:0.0:1787739064.398283:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 380 batchid updated 00000100:00100000:3.0:1787739064.398296:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305654780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665706496:0@lo:103:jbd2/dm-0-8.0 00000020:00080000:0.0:1787739064.398308:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07306241900 refcount 2 committed 0 result 0 batchid 381 00000020:00080000:0.0:1787739064.398313:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07404f86a00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07306241480 00000020:00080000:0.0:1787739064.398317:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].378 00000020:00080000:0.0:1787739064.398319:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07404f86400 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07302faac00 00000020:00080000:0.0:1787739064.398322:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].378 00000020:00080000:0.0:1787739064.398350:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 381 committed batchid 380 00080000:02000000:0.0:1787739064.398372:0:6555:0:(osd_handler.c:2218:osd_trans_start()) *** cfs_fail_loc=19a, val=0*** 00000100:00100000:3.0:1787739064.398374:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787739064.398454:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad705 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x151f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739064.398469:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306848c00 with version 723 00000020:00080000:0.0:1787739064.398481:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 380 cancel update log [0x240000401:0x3:0x0].377: rc = 0 00010000:00010000:1.0:1787739064.398506:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad7e5 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ad7de expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.398523:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x61ea7fc2 (ffffa073068a9600) 00010000:00010000:1.0:1787739064.398529:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad816 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.398539:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad816 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.398547:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad816 lrc: 4/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:0.0:1787739064.398547:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 380 cancel update log [0x200000400:0x3:0x0].377: rc = 0 00000020:00080000:0.0:1787739064.398562:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 381 committed batchid 380 00010000:00010000:1.0:1787739064.398585:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:0.0:1787739064.398595:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 723 rpc_version 723 00000020:00100000:0.0:1787739064.398619:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739064.398629:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07305656940 x1874579665706752/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739064.398640:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561e1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+6:3652:x1874579665706368:12345-0@lo:103:ldlm_bl.0 Request processed in 9944us (10283us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739064.398652:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305656940 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665706752:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739064.398655:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1541 00000100:00100000:1.0:1787739064.398683:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665706496 00000100:00100000:1.0:1787739064.398688:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665706496 00000100:00100000:0.0:1787739064.398695:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739064.398714:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306848f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739064.398733:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1345 00000100:00100000:1.0:1787739064.398741:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306392580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+7:3652:x1874579665706496:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:1.0:1787739064.398755:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739064.398769:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad77c lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x1640014e bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad775 expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.398780:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x1640014e (ffffa07433235800) 00010000:00010000:1.0:1787739064.398787:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739064.398812:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306392580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+6:3652:x1874579665706496:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 71us (440us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.398822:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1345 00010000:00010000:1.0:1787739064.398847:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad816 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.398855:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad816 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.398874:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad705 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x151f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.398885:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad705 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x151f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.398891:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ce00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739064.398896:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad705 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x151f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.398903:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad705 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x151f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.398914:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x151f:0x0].0x0 (ffffa07433235b00) 00010000:00010000:3.0:1787739064.398917:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739064.398920:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad705 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x151f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.398926:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ad705 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x151f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739064.398935:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb7840 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665706880:0@lo:1000:mdt00_003.0 00010000:00010000:3.0:1787739064.398942:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad70c lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.398950:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad70c lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.398957:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad70c lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.398963:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c000 already unused, calling callback (ffffffffc18da4f0) 00000100:00100000:1.0:1787739064.398968:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787739064.398969:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad70c lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.398980:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad70c lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.398991:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc9:0x0].0x0 (ffffa07433235300) 00010000:00010000:3.0:1787739064.398994:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739064.398996:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241540 going to sleep for 11 seconds 00010000:00010000:3.0:1787739064.398998:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad70c lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.399019:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad70c lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xc9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.399077:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305654780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665706496:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:3.0:1787739064.399090:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060061c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665706368:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739064.399190:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665706752 00000100:00100000:2.0:1787739064.399195:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665706752 00000100:00100000:2.0:1787739064.399223:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3290 00000100:00100000:2.0:1787739064.399231:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305655e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665706752:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739064.399344:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305655e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665706752:12345-0@lo:1000:osp_up1-0.0 Request processed in 114us (649us total) trans 4294968402 rc 0/0 00000100:00100000:2.0:1787739064.399352:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3290 00000100:00100000:2.0:1787739064.399364:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665706880 00000100:00100000:2.0:1787739064.399368:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665706880 00000100:00100000:2.0:1787739064.399382:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3291 00000100:00100000:2.0:1787739064.399387:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665706880:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739064.399462:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665706880:12345-0@lo:1000:mdt00_003.0 Request processed in 74us (496us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.399468:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3291 00000100:00100000:0.0:1787739064.408288:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305656940 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665706752:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739064.408652:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb7840 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665706880:0@lo:1000:mdt00_003.0 00010000:00010000:1.0:1787739064.408734:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad81d lrc: 3/1,0 mode: --/PR res: [0x240000402:0x17f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.408751:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad81d lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.408762:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad81d lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739064.408786:0:7944:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad81d lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.408802:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad80f lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.408809:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad80f lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.408815:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad80f lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.408823:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad80f lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.408829:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739064.408834:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad80f lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.408842:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad80f lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.408853:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:1.0:1787739064.408856:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739064.408861:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad80f lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.408867:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad80f lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.408880:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad816 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.408887:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad816 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.408894:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad816 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.408899:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633ce00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739064.408902:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad816 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.408912:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad816 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.408921:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x61ea7fc2 (ffffa073068a9600) 00010000:00010000:1.0:1787739064.408923:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739064.408926:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad816 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.408931:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1ad816 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.408969:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad808 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095106 expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.408987:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad81d lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x17f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095106 expref: 14 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.408995:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17f:0x0].0x0 (ffffa07433235e00) 00010000:00010000:1.0:1787739064.409015:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739064.409018:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1de00, rc 0) 00000100:00100000:1.0:1787739064.409081:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305656d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17633:x1874579652491136:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 21365us (21520us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.409090:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23514 00000100:00100000:3.0:1787739064.421978:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739064.422141:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652491264 00000100:00100000:1.0:1787739064.422151:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652491264 00000100:00100000:1.0:1787739064.422216:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23515 00000100:00100000:1.0:1787739064.422228:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073063912c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17633:x1874579652491264:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739064.422258:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739064.422282:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad824 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x17f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09510d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.422347:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad82b lrc: 3/1,0 mode: --/PR res: [0x240000402:0x17f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.422363:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad82b lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.422374:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad82b lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739064.422490:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17f:0x0]: returning size 4096 00000004:00200000:1.0:1787739064.422496:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739064.422535:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad824 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x17f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09510d expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.422553:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad82b lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x17f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09510d expref: 10 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.422562:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17f:0x0].0x0 (ffffa0730608b700) 00010000:00010000:1.0:1787739064.422565:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739064.422568:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1ca00, rc 0) 00000100:00100000:1.0:1787739064.422625:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073063912c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17633:x1874579652491264:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 400us (664us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.422633:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23515 00000100:00100000:1.0:1787739064.432990:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739064.433101:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652491392 00000100:00100000:3.0:1787739064.433110:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652491392 00000100:00100000:3.0:1787739064.433167:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23516 00000100:00100000:3.0:1787739064.433178:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17633:x1874579652491392:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739064.433197:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739064.433217:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ad832 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xc7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095114 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.433275:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad839 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xc7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.433290:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad839 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.433304:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad839 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739064.433348:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xc7:0x0]: returning size 4096 00000004:00200000:3.0:1787739064.433353:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739064.433400:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1ad832 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095114 expref: 12 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.433437:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad839 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095114 expref: 11 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.433446:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc7:0x0].0x0 (ffffa0730608ba00) 00010000:00010000:3.0:1787739064.433449:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739064.433452:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab400, rc 0) 00000100:00100000:3.0:1787739064.433516:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17633:x1874579652491392:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 339us (534us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739064.433523:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23516 00000100:00100000:0.0:1787739064.442754:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739064.455248:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652491520 00000100:00100000:1.0:1787739064.455263:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652491520 00000100:00100000:1.0:1787739064.455426:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23517 00000100:00100000:1.0:1787739064.455440:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305654780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17633:x1874579652491520:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739064.455464:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739064.455489:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad840 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xc7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09511b expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.455551:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad847 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.455567:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad801 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad7fa expref: 5 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.455576:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad847 lrc: 4/1,0 mode: --/PR res: [0x200000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.455608:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad801 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ad7fa expref: 5 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739064.455625:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d30c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665707008:0@lo:104: 00000100:00100000:1.0:1787739064.455690:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.455742:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739064.455807:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665707008 00000100:00100000:2.0:1787739064.455811:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665707008 00000100:00100000:2.0:1787739064.455863:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1180 00000100:00100000:2.0:1787739064.455870:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d2580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665707008:12345-0@lo:104: 00010000:00010000:2.0:1787739064.455884:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad7fa lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad801 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.455940:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739064.455946:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d2580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:7944:x1874579665707008:12345-0@lo:104: Request processed in 78us (257us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.455954:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1180 00010000:00010000:2.0:1787739064.455977:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739064.455983:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad7fa lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad801 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.455991:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739064.455996:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad7fa lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xc7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ad801 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739064.456015:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad7fa lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xc7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ad801 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.456048:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad7fa lrc: 2/0,0 mode: --/PW res: [0x200000401:0xc7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad801 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.456055:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739064.456075:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad7fa lrc: 1/0,0 mode: --/PW res: [0x200000401:0xc7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad801 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.456083:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aa000/0x7ee4453e4a1ad7fa lrc: 0/0,0 mode: --/PW res: [0x200000401:0xc7:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad801 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.456097:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739064.456099:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739064.456101:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739064.456112:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665707136:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739064.456140:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.457105:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665707136 00000100:00100000:1.0:1787739064.457113:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665707136 00010000:00010000:1.0:1787739064.457140:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad801 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ad7fa expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739064.457175:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1542 00000100:00100000:1.0:1787739064.457182:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562e940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+6:3650:x1874579665707136:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739064.457198:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787739064.457701:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d30c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665707008:0@lo:104: 00010000:00010000:1.0:1787739064.457722:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad847 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739064.462831:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 381 batchid updated 00010000:00010000:1.0:1787739064.463195:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad801 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ad7fa expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.463215:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc7:0x0].0x0 (ffffa07403a6f000) 00000020:00080000:0.0:1787739064.463218:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 381 00010000:00010000:1.0:1787739064.463221:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad847 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.463230:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad847 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.463238:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad847 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.463273:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:0.0:1787739064.463281:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0740a5f6180 with version 724 00000020:00080000:0.0:1787739064.463291:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 381 cancel update log [0x240000401:0x3:0x0].378: rc = 0 00000020:00080000:0.0:1787739064.463365:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 381 cancel update log [0x200000400:0x3:0x0].378: rc = 0 00000020:00080000:0.0:1787739064.463374:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 381 00000004:00080000:0.0:1787739064.463384:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 724 rpc_version 724 00000020:00100000:0.0:1787739064.463433:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739064.463444:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073042612c0 x1874579665707392/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739064.463450:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562e940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+5:3650:x1874579665707136:12345-0@lo:103:ldlm_bl.0 Request processed in 6268us (7308us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.463461:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1542 00000100:00100000:0.0:1787739064.463463:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042612c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665707392:0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1787739064.463473:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad847 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.463484:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad847 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739064.463508:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.463520:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562c3c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665707136:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739064.463534:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306848840 going to sleep for 11 seconds 00000004:00200000:2.0:1787739064.463535:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xc7:0x0]: returning size 4096 00000004:00200000:2.0:1787739064.463542:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739064.463585:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad840 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09511b expref: 16 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.463602:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad847 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09511b expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.463610:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc7:0x0].0x0 (ffffa07403a6f000) 00010000:00010000:2.0:1787739064.463613:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739064.463616:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c800, rc 0) 00000100:00100000:2.0:1787739064.463679:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305654780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17633:x1874579652491520:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 8242us (20932us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.463687:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23517 00000100:00100000:2.0:1787739064.463721:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665707392 00000100:00100000:2.0:1787739064.463726:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665707392 00000100:00100000:2.0:1787739064.463778:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3292 00000100:00100000:2.0:1787739064.463787:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07415701a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665707392:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739064.463921:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07415701a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665707392:12345-0@lo:1000:osp_up1-0.0 Request processed in 135us (413us total) trans 4294968403 rc 0/0 00000100:00100000:2.0:1787739064.463927:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3292 00000100:00100000:0.0:1787739064.464950:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042612c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665707392:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739064.484295:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739064.485055:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652491904 00000100:00100000:2.0:1787739064.485067:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652491904 00000100:00100000:2.0:1787739064.485133:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23518 00000100:00100000:2.0:1787739064.485144:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17633:x1874579652491904:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:2.0:1787739064.485433:0:6523:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739064.485462:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ad82b lrc: 0/0,0 mode: --/PR res: [0x240000402:0x17f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09510d expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.485479:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17f:0x0].0x0 (ffffa0730608b700) 00010000:00010000:2.0:1787739064.485485:0:6523:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739064.485537:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad84e lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.485550:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad84e lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.485560:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad84e lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739064.485580:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562c3c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665707520:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1787739064.485633:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739064.485672:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413600 going to sleep for 11 seconds 00000100:00100000:0.0:1787739064.485770:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665707520 00000100:00100000:0.0:1787739064.485775:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665707520 00000100:00100000:0.0:1787739064.485841:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3293 00000100:00100000:0.0:1787739064.485849:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+5:6523:x1874579665707520:12345-0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739064.485860:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739064.485875:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad855 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad84e expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.485898:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad855 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad84e expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.485912:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad855 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad84e expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.485919:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07307064900) 00010000:00010000:0.0:1787739064.485922:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739064.485925:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b800, rc 0) 00000100:00100000:0.0:1787739064.485979:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:6523:x1874579665707520:12345-0@lo:101:mdt00_000.0 Request processed in 130us (345us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739064.485985:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3293 00000100:00100000:1.0:1787739064.487146:0:6523:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983413 gen 2 00000004:00080000:1.0:1787739064.487157:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983408 committed transno 4294983413 00000020:00080000:1.0:1787739064.487163:0:6523:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa073068489c0 refcount 2 committed 0 result 0 batchid 372 00000020:00080000:1.0:1787739064.487167:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2500 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa073068483c0 00000020:00080000:1.0:1787739064.487172:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].365 00000020:00080000:1.0:1787739064.487175:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2c00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073040e9e00 00000020:00080000:1.0:1787739064.487177:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].364 00000004:00080000:1.0:1787739064.487213:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983410 committed transno 4294983413 00000004:00080000:1.0:1787739064.487225:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983411 committed transno 4294983413 00000004:00080000:1.0:1787739064.487235:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983413 committed transno 4294983413 00000100:00100000:1.0:1787739064.487255:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562c3c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665707520:0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739064.487275:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad84e lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad855 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.487289:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad84e lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad855 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739064.487307:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739064.487341:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad85c lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.487349:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad85c lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.487358:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad85c lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739064.487375:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d3840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665707648:0@lo:101:mdt00_000.0 00000100:00100000:1.0:1787739064.487441:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.487469:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739064.487510:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665707648 00000100:00100000:2.0:1787739064.487516:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665707648 00000100:00100000:2.0:1787739064.487569:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3294 00000100:00100000:2.0:1787739064.487577:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:6523:x1874579665707648:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739064.487591:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739064.487609:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad863 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad85c expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.487631:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad863 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad85c expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.487643:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad863 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad85c expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.487652:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x61ea7fc2 (ffffa07403a6fd00) 00010000:00010000:2.0:1787739064.487654:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739064.487657:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d000, rc 0) 00000100:00100000:2.0:1787739064.487705:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665707648:12345-0@lo:101:mdt00_000.0 Request processed in 129us (264us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.487715:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3294 00000020:00080000:0.0:1787739064.487847:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 372 committed batchid 371 00000020:00080000:0.0:1787739064.487918:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 371 00000100:00100000:1.0:1787739064.488116:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d3840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665707648:0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739064.488130:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad85c lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad863 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.488140:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad85c lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad863 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739064.488154:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:1.0:1787739064.488189:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d3840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665707776:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739064.488217:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.488242:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739064.488309:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665707776 00000100:00100000:0.0:1787739064.488314:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665707776 00000100:00100000:0.0:1787739064.488350:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3295 00000100:00100000:0.0:1787739064.488357:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665707776:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739064.488465:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665707776:12345-0@lo:1000:mdt00_000.0 Request processed in 107us (245us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739064.488473:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3295 00000100:00100000:1.0:1787739064.488497:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d3840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665707776:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739064.488533:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d3840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665707904:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739064.488555:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.488578:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739064.488627:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665707904 00000100:00100000:2.0:1787739064.488632:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665707904 00000100:00100000:2.0:1787739064.488661:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3296 00000100:00100000:2.0:1787739064.488668:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665707904:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739064.488754:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665707904:12345-0@lo:1000:mdt00_000.0 Request processed in 86us (197us total) trans 0 rc 1/1 00000100:00100000:2.0:1787739064.488760:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3296 00000100:00100000:0.0:1787739064.488828:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d3840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665707904:0@lo:1000:mdt00_000.0 00010000:00010000:0.0:1787739064.488876:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ad86a lrc: 3/0,1 mode: --/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.488890:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ad86a lrc: 3/0,1 mode: --/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.488898:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ad86a lrc: 3/0,1 mode: --/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739064.488911:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d2580 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665708032:0@lo:101:mdt00_000.0 00000100:00100000:0.0:1787739064.488937:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739064.489022:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306848840 going to sleep for 11 seconds 00000100:00100000:0.0:1787739064.489588:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739064.490965:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652491648 00000100:00100000:3.0:1787739064.490976:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652491648 00000100:00100000:3.0:1787739064.491053:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1346 00000100:00100000:3.0:1787739064.491063:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d0b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2407:x1874579652491648:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:3.0:1787739064.491078:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739064.491101:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ad81d lrc: 0/0,0 mode: --/PR res: [0x240000402:0x17f:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095106 expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.491117:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17f:0x0].0x0 (ffffa07433235e00) 00010000:00010000:3.0:1787739064.491129:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739064.491182:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d0b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2407:x1874579652491648:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 121us (1597us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739064.491189:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1346 00000100:00100000:1.0:1787739064.491920:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665708032 00000100:00100000:1.0:1787739064.491927:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665708032 00000100:00100000:1.0:1787739064.492014:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3297 00000100:00100000:1.0:1787739064.492023:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665708032:12345-0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739064.492039:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739064.492057:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad871 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad86a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.492082:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad871 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad86a expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.492096:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad871 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad86a expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.492104:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17f:0x0].0x0 (ffffa07433235100) 00010000:00010000:1.0:1787739064.492108:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739064.492111:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1cc00, rc 0) 00000100:00100000:1.0:1787739064.492190:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665708032:12345-0@lo:101:mdt00_000.0 Request processed in 168us (3251us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.492196:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3297 00000100:00100000:1.0:1787739064.492876:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d2580 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665708032:0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739064.492898:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ad86a lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad871 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.492909:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ad86a lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad871 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.492942:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad878 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.492952:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad878 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.492959:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad878 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.492980:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad87f lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.492991:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad839 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095114 expref: 10 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.492999:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad87f lrc: 4/0,1 mode: --/EX res: [0x240000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.493038:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad839 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095114 expref: 10 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.493052:0:6523:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad839 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095114 expref: 10 pid: 6524 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739064.493065:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d2d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665708160:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739064.493195:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739064.501649:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739064.501928:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d2d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665708160:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739064.501954:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad87f lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739064.502063:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652492032 00000100:00100000:2.0:1787739064.502070:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652492032 00010000:00010000:2.0:1787739064.502104:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad839 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095114 expref: 11 pid: 6524 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739064.502145:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1543 00000100:00100000:2.0:1787739064.502156:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba6050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2408:x1874579652492032:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739064.502171:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739064.502177:0:6510:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad839 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095114 expref: 11 pid: 6524 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.502190:0:6510:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad839 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095114 expref: 11 pid: 6524 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.502214:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ad839 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095114 expref: 11 pid: 6524 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.502227:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc7:0x0].0x0 (ffffa0730608ba00) 00010000:00010000:2.0:1787739064.502233:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad87f lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.502243:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad87f lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.502252:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad87f lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.502278:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739064.502336:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba6050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652492032:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 182us (696us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.502343:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1543 00010000:00010000:1.0:1787739064.502505:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad87f lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.502513:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad87f lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.502564:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad886 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.502572:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad886 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.502582:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad886 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739064.502601:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080d2d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665708288:0@lo:101:mdt00_000.0 00000100:00100000:1.0:1787739064.502641:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.502669:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739064.502739:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665708288 00000100:00100000:2.0:1787739064.502744:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665708288 00000100:00100000:2.0:1787739064.502782:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3298 00000100:00100000:2.0:1787739064.502790:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665708288:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739064.502806:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739064.502819:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad88d lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad886 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.502839:0:6528:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad847 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09511b expref: 13 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.502846:0:6528:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad88d lrc: 4/0,0 mode: --/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad886 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.502869:0:6528:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad847 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09511b expref: 13 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.502886:0:6528:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad847 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09511b expref: 13 pid: 7944 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739064.502897:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f1a40 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6528:1874579665708416:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739064.502950:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413240 going to sleep for 11 seconds 00000100:00100000:1.0:1787739064.511804:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f1a40 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6528:1874579665708416:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739064.511846:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad88d lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad886 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.511861:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc7:0x0].0x0 (ffffa07403a6f000) 00010000:00010000:1.0:1787739064.511865:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739064.511869:0:6528:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad88d lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad886 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.511881:0:6528:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad88d lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad886 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.511889:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d400, rc 0) 00000100:00100000:1.0:1787739064.512030:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665708288:12345-0@lo:101:mdt00_000.0 Request processed in 9240us (9388us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.512039:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3298 00000100:00100000:1.0:1787739064.514132:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739064.514298:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652492160 00000100:00100000:3.0:1787739064.514309:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652492160 00010000:00010000:3.0:1787739064.514347:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad847 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09511b expref: 14 pid: 7944 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.514389:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1544 00000100:00100000:3.0:1787739064.514400:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080d3840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652492160:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739064.514437:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739064.514444:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad847 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09511b expref: 14 pid: 7944 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.514459:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad847 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09511b expref: 14 pid: 7944 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.514484:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ad847 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09511b expref: 14 pid: 7944 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.514499:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc7:0x0].0x0 (ffffa07403a6f000) 00010000:00010000:3.0:1787739064.514506:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad88d lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad886 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.514518:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad88d lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad886 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.514533:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad88d lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad886 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.514567:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad88d lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad886 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.514579:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073063921c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665708544:0@lo:105: 00000100:00100000:0.0:1787739064.514598:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080d2d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665708288:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739064.514621:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787739064.514624:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad886 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad88d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.514637:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad886 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad88d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.514665:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e840 going to sleep for 11 seconds 00000100:00100000:3.0:1787739064.514691:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665708544 00000100:00100000:3.0:1787739064.514693:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665708544 00000100:00100000:3.0:1787739064.514729:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1181 00000100:00100000:3.0:1787739064.514737:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073063930c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665708544:12345-0@lo:105: 00010000:00010000:3.0:1787739064.514748:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad886 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad88d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.514758:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad886 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad88d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.514767:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad886 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad88d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.514774:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad886 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad88d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.514847:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad886 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad88d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.514857:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad886 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ad88d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.514877:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07441a1d600) 00000100:00100000:3.0:1787739064.514923:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073063930c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665708544:12345-0@lo:105: Request processed in 186us (300us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739064.514932:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1181 00010000:00010000:0.0:1787739064.515081:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad886 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad88d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.515091:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad886 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad88d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739064.515789:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073063921c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665708544:0@lo:105: 00010000:00010000:2.0:1787739064.515803:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739064.515960:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d3840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:2409:x1874579652492160:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 1560us (1834us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.515969:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1544 00000100:00100000:0.0:1787739064.516170:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d3480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665708672:0@lo:403:mdt00_000.0 00000100:00100000:0.0:1787739064.516219:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739064.516252:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306848cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739064.517092:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665708672 00000100:00100000:2.0:1787739064.517102:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665708672 00000100:00100000:2.0:1787739064.517158:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3299 00000100:00100000:2.0:1787739064.517167:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665708672:12345-0@lo:403:mdt00_000.0 00000100:00100000:2.0:1787739064.519596:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665708672:12345-0@lo:403:mdt00_000.0 Request processed in 2422us (3368us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.519611:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3299 00000100:00100000:0.0:1787739064.519724:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d3480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665708672:0@lo:403:mdt00_000.0 00000040:00080000:0.0:1787739064.520684:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].366, 664 off555064 00000040:00080000:0.0:1787739064.520867:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].367, 744 off607752 00000004:00080000:0.0:1787739064.520873:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07306848cc0 with version 699 00000004:00080000:2.0:1787739064.521083:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 699 rpc_version 699 00000020:00100000:2.0:1787739064.521118:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739064.521126:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074080f2580 x1874579665708928/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1808/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739064.521149:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f2580 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665708928:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739064.521212:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739064.521250:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739064.522230:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665708928 00000100:00100000:3.0:1787739064.522239:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665708928 00000100:00100000:3.0:1787739064.522292:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3300 00000100:00100000:3.0:1787739064.522302:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665708928:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739064.522697:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665708928:12345-0@lo:1000:osp_up0-1.0 Request processed in 396us (1484us total) trans 4294983414 rc 0/0 00000100:00100000:3.0:1787739064.522706:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3300 00000100:00100000:2.0:1787739064.523157:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f2580 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665708928:0@lo:1000:osp_up0-1.0 00000004:00200000:0.0:1787739064.523218:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17f:0x0]: returning size 4096 00010000:00010000:0.0:1787739064.523240:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad87f lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523250:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad87f lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523257:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad87f lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523263:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739064.523268:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad87f lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523276:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad87f lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523290:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc7:0x0].0x0 (ffffa0730608ba00) 00010000:00010000:0.0:1787739064.523294:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739064.523298:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad87f lrc: 2/0,0 mode: --/EX res: [0x240000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523304:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ad87f lrc: 0/0,0 mode: --/EX res: [0x240000400:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523316:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad886 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad88d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523330:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad886 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad88d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523337:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad886 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad88d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523362:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739064.523371:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xc7:0x0] 00010000:00010000:0.0:1787739064.523377:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad878 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523384:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad878 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523389:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad878 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523394:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739064.523397:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad878 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523403:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad878 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523436:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17f:0x0].0x0 (ffffa0730608b700) 00010000:00010000:0.0:1787739064.523439:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739064.523442:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad878 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523448:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ad878 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523459:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ad86a lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad871 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523466:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ad86a lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad871 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523473:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ad86a lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad871 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523491:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:0.0:1787739064.523502:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739064.523506:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad84e lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad855 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523513:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad84e lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad855 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523519:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad84e lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad855 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523532:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:0.0:1787739064.523535:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739064.523544:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1787739064.523549:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad886 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad88d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523556:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739064.523560:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad886 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad88d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739064.523568:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad886 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad88d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523599:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad886 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad88d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523607:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739064.523627:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad886 lrc: 1/0,0 mode: --/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad88d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523635:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1d600/0x7ee4453e4a1ad886 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad88d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523650:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787739064.523653:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ad86a lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad871 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523659:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739064.523663:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ad86a lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad871 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739064.523670:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ad86a lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad871 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523687:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ad86a lrc: 2/0,0 mode: --/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad871 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523693:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739064.523707:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ad86a lrc: 1/0,0 mode: --/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad871 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523715:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b600/0x7ee4453e4a1ad86a lrc: 0/0,0 mode: --/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad871 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523726:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739064.523730:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad84e lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad855 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523735:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739064.523738:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad84e lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad855 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739064.523744:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad84e lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad855 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523757:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad84e lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad855 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523762:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739064.523767:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad84e lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad855 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523773:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633c600/0x7ee4453e4a1ad84e lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad855 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.523784:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739064.523786:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739064.523787:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739064.523801:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665709056:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739064.523846:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739064.523893:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665709312:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739064.523914:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739064.523932:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787739064.523937:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665709184:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739064.523957:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739064.523982:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665709056 00000100:00100000:0.0:1787739064.523987:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665709056 00010000:00010000:3.0:1787739064.525141:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739064.525226:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739064.526242:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665709312 00000100:00100000:0.0:1787739064.526251:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665709312 00000100:00100000:2.0:1787739064.529205:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665709184 00000100:00100000:2.0:1787739064.529223:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665709184 00000100:00100000:2.0:1787739064.529297:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1347 00000100:00100000:2.0:1787739064.529307:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d3840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+12:3651:x1874579665709056:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739064.529323:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739064.529349:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad88d lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad886 expref: 12 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.529364:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc7:0x0].0x0 (ffffa07403a6f000) 00010000:00010000:2.0:1787739064.529374:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739064.529469:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d3840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+11:3651:x1874579665709056:12345-0@lo:103:ldlm_bl.0 Request processed in 163us (5622us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.529476:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1347 00000100:00100000:2.0:1787739064.529498:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1348 00000100:00100000:2.0:1787739064.529504:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d2940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+10:3653:x1874579665709312:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739064.529512:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739064.529524:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ad855 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad84e expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.529534:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07307064900) 00010000:00010000:2.0:1787739064.529541:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739064.529571:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d2940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+9:3653:x1874579665709312:12345-0@lo:103:ldlm_bl.0 Request processed in 67us (5655us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.529577:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1348 00000100:00100000:2.0:1787739064.529591:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1349 00000100:00100000:2.0:1787739064.529597:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d0780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+8:3653:x1874579665709184:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739064.529605:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739064.529615:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad871 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x17f:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad86a expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.529626:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17f:0x0].0x0 (ffffa07433235100) 00010000:00010000:2.0:1787739064.529632:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739064.529654:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d0780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+7:3653:x1874579665709184:12345-0@lo:103:ldlm_bl.0 Request processed in 58us (5696us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.529661:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1349 00000100:00100000:2.0:1787739064.529707:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665709184:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739064.529717:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665709312:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739064.530772:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665709056:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739064.530809:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad85c lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad863 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.530884:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad85c lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad863 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739064.530977:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080d30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17633:x1874579652491904:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 45833us (46689us total) trans 4294968404 rc 0/0 00000100:00100000:0.0:1787739064.530985:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23518 00000100:00100000:3.0:1787739064.559674:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739064.559704:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652492288 00000100:00100000:3.0:1787739064.559712:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652492288 00000100:00100000:3.0:1787739064.559751:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23519 00000100:00100000:3.0:1787739064.559757:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073063921c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17643:x1874579652492288:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739064.559774:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739064.559792:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ad894 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095122 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.559851:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad89b lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.559872:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad89b lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.559883:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad89b lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.559898:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad8a2 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.559909:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad8a2 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.559921:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad8a2 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.559966:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad89b lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.559973:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad89b lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.559979:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad89b lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.559986:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad89b lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.559993:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739064.559998:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad89b lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.560019:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad89b lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.560034:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730608b800) 00010000:00010000:3.0:1787739064.560037:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739064.560042:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad89b lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.560048:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad89b lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.560062:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad8a2 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.560068:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad8a2 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.560074:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad8a2 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.560079:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aae00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739064.560082:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad8a2 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.560088:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad8a2 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.560096:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x101e480e (ffffa0730608b500) 00010000:00010000:3.0:1787739064.560098:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739064.560100:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad8a2 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.560105:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad8a2 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.560137:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ad894 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095122 expref: 14 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.560147:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730608b800) 00010000:00010000:3.0:1787739064.560150:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739064.560153:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1ad894 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095122 expref: 14 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.560163:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab800, rc 0) 00000100:00100000:3.0:1787739064.560217:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073063921c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17643:x1874579652492288:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 458us (548us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739064.560224:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23519 00000100:00100000:2.0:1787739064.573518:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739064.574576:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652492416 00000100:00100000:2.0:1787739064.574588:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652492416 00000100:00100000:2.0:1787739064.574642:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23520 00000100:00100000:2.0:1787739064.576448:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17643:x1874579652492416:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739064.576479:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739064.576508:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad8a9 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095129 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.576575:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad8b0 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.576589:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad8b0 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.576604:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad8b0 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.576616:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad8b7 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.576623:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad8b7 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.576629:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad8b7 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.577172:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad8b0 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.577181:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad8b0 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.577186:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad8b0 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.577193:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad8b0 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.577199:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633cc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739064.577203:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad8b0 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.577210:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad8b0 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.577223:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403a6f000) 00010000:00010000:2.0:1787739064.577227:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739064.577231:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad8b0 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.577238:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1ad8b0 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.577248:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad8b7 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.577254:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad8b7 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.577259:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad8b7 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.577264:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633de00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739064.577267:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad8b7 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.577272:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad8b7 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.577279:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x101e480e (ffffa07403a6f500) 00010000:00010000:2.0:1787739064.577281:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739064.577284:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad8b7 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.577289:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad8b7 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.577323:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad8a9 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095129 expref: 14 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.577338:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403a6f000) 00010000:00010000:2.0:1787739064.577340:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739064.577743:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad8a9 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095129 expref: 14 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.577758:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d400, rc 0) 00000100:00100000:2.0:1787739064.577828:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17643:x1874579652492416:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 3177us (4319us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739064.577837:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23520 00000100:00100000:0.0:1787739064.586250:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739064.586289:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652492544 00000100:00100000:0.0:1787739064.586300:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652492544 00000100:00100000:0.0:1787739064.586775:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23521 00000100:00100000:0.0:1787739064.586787:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17643:x1874579652492544:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739064.586809:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739064.586832:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad8be lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095130 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.586890:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad8c5 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.586904:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad8c5 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.586915:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad8c5 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739064.586958:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:0.0:1787739064.586963:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739064.586995:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad8be lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095130 expref: 15 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.587022:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad8c5 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095130 expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.587030:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07307064d00) 00010000:00010000:0.0:1787739064.587033:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739064.587036:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a200, rc 0) 00000100:00100000:0.0:1787739064.588249:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17643:x1874579652492544:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 1460us (2005us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739064.588260:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23521 00000100:00100000:2.0:1787739064.601542:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739064.601657:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652492800 00000100:00100000:0.0:1787739064.601667:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652492800 00000100:00100000:0.0:1787739064.601734:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23522 00000100:00100000:0.0:1787739064.601745:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17643:x1874579652492800:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:0.0:1787739064.601799:0:6525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739064.601824:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad8c5 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095130 expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.601841:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07307064d00) 00010000:00010000:0.0:1787739064.601851:0:6525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739064.601910:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad8cc lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.601924:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad8cc lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.601934:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad8cc lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.601945:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad8d3 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x101e480e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.601951:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad8d3 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.601957:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad8d3 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1787739064.602837:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].380, 2168 off581504 00000040:00080000:0.0:1787739064.602910:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].380, 2368 off631472 00000004:00080000:0.0:1787739064.602915:0:6525:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa073068483c0 with version 725 00000004:00080000:1.0:1787739064.603166:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 725 rpc_version 725 00000020:00100000:1.0:1787739064.603202:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739064.603212:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074356acf00 x1874579665709568/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 3976/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739064.603233:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356acf00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665709568:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739064.603296:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.603345:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241540 going to sleep for 11 seconds 00000100:00100000:2.0:1787739064.604287:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665709568 00000100:00100000:2.0:1787739064.604296:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665709568 00000100:00100000:2.0:1787739064.604362:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3301 00000100:00100000:2.0:1787739064.604371:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665709568:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739064.604973:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665709568:12345-0@lo:1000:osp_up1-0.0 Request processed in 600us (1673us total) trans 4294968405 rc 0/0 00000100:00100000:2.0:1787739064.604981:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3301 00000100:00100000:0.0:1787739064.605170:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356acf00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665709568:0@lo:1000:osp_up1-0.0 00010000:00010000:2.0:1787739064.605326:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad8da lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1521:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.605341:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad8da lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1521:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.605352:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad8da lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1521:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.605371:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad8e1 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xca:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.605380:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad8e1 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.605386:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad8e1 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.606541:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad8e8 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xca:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.606563:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad8e8 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.606574:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad8e8 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739064.606596:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f1a40 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665709696:0@lo:101:mdt00_002.0 00000100:00100000:2.0:1787739064.606654:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.606703:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665709696 00000100:00100000:2.0:1787739064.606706:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c413000 going to sleep for 11 seconds 00000100:00100000:1.0:1787739064.606711:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665709696 00000100:00100000:1.0:1787739064.606756:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3302 00000100:00100000:1.0:1787739064.606765:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410b0b450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:6525:x1874579665709696:12345-0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787739064.606796:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739064.606812:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad8ef lrc: 2/0,0 mode: --/PW res: [0x240000401:0xca:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad8e8 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.606835:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad8ef lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad8e8 expref: 10 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.606848:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad8ef lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad8e8 expref: 10 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.606857:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xca:0x0].0x0 (ffffa07433235400) 00010000:00010000:1.0:1787739064.606860:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739064.606864:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1da00, rc 0) 00000100:00100000:1.0:1787739064.606900:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410b0b450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+10:6525:x1874579665709696:12345-0@lo:101:mdt00_002.0 Request processed in 136us (245us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.606907:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3302 00000100:00100000:2.0:1787739064.607674:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f1a40 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665709696:0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787739064.607701:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad8e8 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad8ef expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.607714:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad8e8 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad8ef expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739064.607737:0:6525:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000029860801 reply state 00000000a34bd6f0 transno 4294983415 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad8e1 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.607751:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad8e1 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.607759:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xca:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:2.0:1787739064.607762:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739064.607765:0:6525:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787739064.607771:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad8e8 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad8ef expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.607779:0:6525:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad8e8 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad8ef expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739064.607791:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xca:0x0] 00000004:00010000:2.0:1787739064.607797:0:6525:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000029860801 reply state 00000000a34bd6f0 transno 4294983415 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad8da lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1521:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.607805:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad8da lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1521:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.607812:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1521:0x0].0x0 (ffffa07403a6f000) 00010000:00010000:2.0:1787739064.607814:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739064.607816:0:6525:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:2.0:1787739064.607821:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1521:0x0]: returning size 4096 00010000:00010000:2.0:1787739064.607832:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad8cc lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.607839:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad8cc lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.607845:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad8cc lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.607852:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad8cc lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.607862:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ba00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739064.607867:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad8cc lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.607875:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad8cc lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.607885:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07307064a00) 00010000:00010000:2.0:1787739064.607887:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739064.607890:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad8cc lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.607896:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ad8cc lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739064.607913:0:6525:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000029860801 reply state 00000000a34bd6f0 transno 4294983415 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad8d3 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.607920:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad8d3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.607926:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x101e480e (ffffa07307064c00) 00010000:00010000:2.0:1787739064.607928:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739064.607930:0:6525:0:(service.c:179:ptlrpc_save_lock()) nlocks 2 00000100:00100000:2.0:1787739064.608021:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17643:x1874579652492800:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 6276us (6486us total) trans 4294983415 rc 0/0 00000100:00100000:2.0:1787739064.608028:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23522 00000100:00100000:0.0:1787739064.625160:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739064.625311:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652493056 00000100:00100000:2.0:1787739064.625322:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652493056 00000100:00100000:2.0:1787739064.625380:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23523 00000100:00100000:2.0:1787739064.625391:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17644:x1874579652493056:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:2.0:1787739064.625717:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17644:x1874579652493056:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 328us (567us total) trans 0 rc -17/-17 00000100:00100000:2.0:1787739064.625725:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23523 00010000:00010000:2.0:1787739064.642718:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739064.643047:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739064.643112:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652493184 00000100:00100000:0.0:1787739064.643123:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652493184 00000100:00100000:0.0:1787739064.643212:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23524 00000100:00100000:0.0:1787739064.643223:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306392580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17645:x1874579652493184:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739064.643244:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739064.643266:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad8f6 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095137 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643332:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad8fd lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643346:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad8fd lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643357:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad8fd lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643367:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad904 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643375:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad904 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643381:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad904 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643489:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad90b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1521:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643499:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad90b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1521:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643507:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad90b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1521:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739064.643588:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1521:0x0]: returning size 4096 00000004:00200000:0.0:1787739064.643593:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:0.0:1787739064.643612:0:6524:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad90b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1521:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643624:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad8fd lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643631:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad8fd lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643637:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad8fd lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643644:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad8fd lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643651:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739064.643655:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad8fd lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643663:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad8fd lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643673:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07307064d00) 00010000:00010000:0.0:1787739064.643677:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739064.643681:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad8fd lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643687:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ad8fd lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643704:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad904 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643711:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad904 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643716:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad904 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643721:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609bc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739064.643724:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad904 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643729:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad904 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643736:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x101e480e (ffffa07307064c00) 00010000:00010000:0.0:1787739064.643738:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739064.643741:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad904 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643747:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad904 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643771:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad8f6 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095137 expref: 17 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643788:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad90b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1521:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095137 expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.643800:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1521:0x0].0x0 (ffffa07403a6f000) 00010000:00010000:0.0:1787739064.643803:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787739064.643805:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a400, rc 0) 00000100:00100000:0.0:1787739064.643866:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306392580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17645:x1874579652493184:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 645us (823us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739064.643874:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23524 00000100:00100000:3.0:1787739064.649391:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739064.649525:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652493312 00000100:00100000:2.0:1787739064.649534:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652493312 00000100:00100000:2.0:1787739064.649589:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23525 00000100:00100000:2.0:1787739064.649599:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306390f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17645:x1874579652493312:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739064.649619:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739064.649639:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad912 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xca:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09513e expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.649706:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ad919 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xca:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.649719:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ad919 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.649730:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ad919 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739064.649782:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xca:0x0]: returning size 4096 00000004:00200000:2.0:1787739064.649788:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739064.649821:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad912 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xca:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09513e expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.649839:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ad919 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xca:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09513e expref: 17 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.649853:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xca:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:2.0:1787739064.649856:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739064.649859:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c200, rc 0) 00000100:00100000:2.0:1787739064.649918:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306390f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17645:x1874579652493312:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 320us (535us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.649926:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23525 00000100:00100000:3.0:1787739064.663750:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739064.663879:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652493440 00000100:00100000:0.0:1787739064.663889:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652493440 00000100:00100000:0.0:1787739064.663952:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23526 00000100:00100000:0.0:1787739064.663963:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306392d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17645:x1874579652493440:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739064.663988:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739064.664017:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad920 lrc: 2/0,0 mode: --/CR res: [0x240000401:0xca:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095145 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.664085:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad927 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xca:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.664100:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad8ef lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad8e8 expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.664109:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad927 lrc: 4/1,0 mode: --/PR res: [0x240000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.664144:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad8ef lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ad8e8 expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739064.664159:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665709824:0@lo:104: 00000100:00100000:0.0:1787739064.664211:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739064.664264:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4db40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739064.664355:0:15907:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:3.0:1787739064.664367:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665709824 00000100:00100000:3.0:1787739064.664370:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665709824 00000100:00100000:3.0:1787739064.664401:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1182 00000100:00100000:3.0:1787739064.664408:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f85e050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:13793:x1874579665709824:12345-0@lo:104: 00010000:00010000:3.0:1787739064.664461:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad8e8 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad8ef expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.664523:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739064.664529:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f85e050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:13793:x1874579665709824:12345-0@lo:104: Request processed in 122us (317us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739064.664539:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1182 00010000:00010000:3.0:1787739064.664579:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739064.664586:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad8e8 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad8ef expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.664594:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633cc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739064.664599:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad8e8 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xca:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ad8ef expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739064.664608:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad8e8 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xca:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ad8ef expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.664634:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad8e8 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xca:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad8ef expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.664641:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739064.664662:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad8e8 lrc: 1/0,0 mode: --/PW res: [0x240000401:0xca:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad8ef expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.664669:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633cc00/0x7ee4453e4a1ad8e8 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xca:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad8ef expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.664686:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739064.664688:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739064.664690:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739064.664702:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073063903c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665709952:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739064.664729:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739064.664768:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665709952 00000100:00100000:3.0:1787739064.664772:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665709952 00010000:00010000:3.0:1787739064.665089:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad8ef lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ad8e8 expref: 10 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.665148:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1545 00000100:00100000:3.0:1787739064.665155:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073063930c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+10:3650:x1874579665709952:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739064.665165:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1787739064.665553:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665709824:0@lo:104: 00010000:00010000:3.0:1787739064.665572:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad927 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.673032:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad85c lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad863 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739064.673051:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad85c lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad863 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.673095:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad85c lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad863 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.673101:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739064.673133:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07441a1c600/0x7ee4453e4a1ad85c lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad863 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787739064.673153:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968405 is committed 00000001:00080000:0.0:1787739064.673166:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968404 is committed 00000020:00080000:0.0:1787739064.673173:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 372 batchid updated 00000020:00080000:0.0:1787739064.673186:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa0730623dc00 refcount 2 committed 0 result 0 batchid 373 00000020:00080000:0.0:1787739064.673189:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2d00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730623dcc0 00000020:00080000:0.0:1787739064.673193:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].366 00000020:00080000:0.0:1787739064.673195:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2c00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306940600 00000020:00080000:0.0:1787739064.673198:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].365 00010000:00010000:0.0:1787739064.673222:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad7ec lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x17f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.673230:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad7ec lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x17f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.673241:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad7ec lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x17f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.673248:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739064.673252:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad7ec lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x17f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.673259:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad7ec lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x17f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.673271:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17f:0x0].0x0 (ffffa0730608b700) 00010000:00010000:0.0:1787739064.673273:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739064.673277:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad7ec lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x17f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.673283:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa800/0x7ee4453e4a1ad7ec lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x17f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.673297:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad7f3 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.673302:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad7f3 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.673307:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad7f3 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.673312:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739064.673314:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad7f3 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.673323:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad7f3 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.673329:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc7:0x0].0x0 (ffffa0730608ba00) 00010000:00010000:0.0:1787739064.673331:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739064.673334:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad7f3 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.673339:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abc00/0x7ee4453e4a1ad7f3 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xc7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:0.0:1787739064.673360:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 373 committed batchid 372 00000004:00080000:0.0:1787739064.673486:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07410b4dcc0 with version 700 00000020:00080000:0.0:1787739064.673497:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 372 cancel update log [0x200000401:0x3:0x0].365: rc = 0 00000020:00080000:0.0:1787739064.673535:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 372 cancel update log [0x240000400:0x3:0x0].364: rc = 0 00000020:00080000:0.0:1787739064.673542:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 372 00000004:00080000:0.0:1787739064.673551:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 700 rpc_version 700 00000020:00100000:0.0:1787739064.673569:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739064.673580:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073060d2d00 x1874579665710336/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739064.673599:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d2d00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665710336:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739064.673673:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739064.673709:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d840 going to sleep for 11 seconds 00000100:00100000:0.0:1787739064.673728:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787739064.673738:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665710080:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:0.0:1787739064.673764:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787739064.673847:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ad8ef lrc: 0/0,0 mode: --/PW res: [0x240000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ad8e8 expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.673859:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xca:0x0].0x0 (ffffa07433235400) 00010000:00010000:0.0:1787739064.673864:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad927 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.673873:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad927 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.673881:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad927 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.673902:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739064.673936:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073063930c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+5:3650:x1874579665709952:12345-0@lo:103:ldlm_bl.0 Request processed in 8782us (9206us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739064.673945:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1545 00000100:00100000:0.0:1787739064.673966:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665710080 00000100:00100000:0.0:1787739064.673972:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665710080 00000100:00100000:0.0:1787739064.674020:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1350 00000100:00100000:0.0:1787739064.674028:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d1e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+7:3650:x1874579665710080:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:0.0:1787739064.674040:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739064.674053:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1ad863 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x61ea7fc2 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad85c expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.674064:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x61ea7fc2 (ffffa07403a6fd00) 00010000:00010000:0.0:1787739064.674073:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739064.674095:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d1e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+6:3650:x1874579665710080:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 68us (330us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739064.674101:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1350 00000100:00100000:0.0:1787739064.674143:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665710080:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:0.0:1787739064.674171:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073063903c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665709952:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739064.674200:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad927 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.674209:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad927 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739064.674268:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xca:0x0]: returning size 4096 00000004:00200000:0.0:1787739064.674274:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739064.674322:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ad920 lrc: 0/0,0 mode: --/CR res: [0x240000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095145 expref: 9 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.674339:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad927 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095145 expref: 8 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.674346:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xca:0x0].0x0 (ffffa07433235400) 00010000:00010000:0.0:1787739064.674349:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739064.674351:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609bc00, rc 0) 00000100:00100000:0.0:1787739064.674399:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306392d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17645:x1874579652493440:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 10439us (10659us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739064.674407:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23526 00000100:00100000:1.0:1787739064.675633:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665710336 00000100:00100000:1.0:1787739064.675642:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665710336 00000100:00100000:1.0:1787739064.675914:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3303 00000100:00100000:1.0:1787739064.675924:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:7467:x1874579665710336:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739064.676106:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665710336:12345-0@lo:1000:osp_up0-1.0 Request processed in 183us (2431us total) trans 4294983416 rc 0/0 00000100:00100000:1.0:1787739064.676113:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3303 00000100:00100000:3.0:1787739064.676186:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d2d00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665710336:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739064.679808:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739064.680510:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652493696 00000100:00100000:2.0:1787739064.680519:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652493696 00000100:00100000:2.0:1787739064.680575:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23527 00000100:00100000:2.0:1787739064.680585:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17645:x1874579652493696:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:2.0:1787739064.680661:0:6524:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739064.680687:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ad90b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1521:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095137 expref: 17 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.680705:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1521:0x0].0x0 (ffffa07403a6f000) 00010000:00010000:2.0:1787739064.680710:0:6524:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739064.680734:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad92e lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.680746:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad92e lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.680756:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad92e lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.680767:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ad935 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x101e480e bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.680779:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ad935 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.680784:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ad935 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.680853:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad93c lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1521:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.680860:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad93c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1521:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.680866:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad93c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1521:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.680881:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad943 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xca:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.680889:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ad919 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09513e expref: 16 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.680896:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad943 lrc: 4/0,1 mode: --/EX res: [0x200000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.680922:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ad919 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09513e expref: 16 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.680934:0:6524:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ad919 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09513e expref: 16 pid: 6525 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739064.680952:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f1a40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665710464:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739064.681033:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739064.692849:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f1a40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665710464:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739064.692878:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad943 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739064.696034:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739064.696123:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652493824 00000100:00100000:1.0:1787739064.696132:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652493824 00010000:00010000:1.0:1787739064.696173:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ad919 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09513e expref: 17 pid: 6525 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739064.696219:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1546 00000100:00100000:1.0:1787739064.696230:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba7050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2408:x1874579652493824:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739064.696246:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739064.696253:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ad919 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xca:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09513e expref: 17 pid: 6525 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.696266:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ad919 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xca:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09513e expref: 17 pid: 6525 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.696287:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ad919 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xca:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09513e expref: 17 pid: 6525 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.696303:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xca:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:1.0:1787739064.696308:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad943 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.696318:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad943 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.696326:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad943 lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.696565:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739064.696618:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba7050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:2408:x1874579652493824:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 388us (593us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.696626:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1546 00010000:00010000:3.0:1787739064.698275:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad943 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.698290:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad943 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.698588:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad94a lrc: 3/0,1 mode: --/EX res: [0x240000401:0xca:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.698596:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad94a lrc: 3/0,1 mode: --/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.698608:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad94a lrc: 3/0,1 mode: --/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.698630:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306392940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665710592:0@lo:101:mdt00_001.0 00000100:00100000:3.0:1787739064.698751:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739064.698896:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739064.698939:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665710592 00000100:00100000:2.0:1787739064.698945:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665710592 00000100:00100000:2.0:1787739064.699040:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3304 00000100:00100000:2.0:1787739064.699050:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306393c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6524:x1874579665710592:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739064.699065:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739064.699079:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad951 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xca:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad94a expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.699106:0:6529:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad927 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095145 expref: 7 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.699116:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad951 lrc: 4/0,0 mode: --/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad94a expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.699141:0:6529:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad927 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095145 expref: 7 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.699155:0:6529:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad927 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095145 expref: 7 pid: 13793 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739064.699172:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f1a40 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665710720:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739064.699236:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739064.702204:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f1a40 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665710720:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739064.702238:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad951 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad94a expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.702252:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xca:0x0].0x0 (ffffa07433235400) 00010000:00010000:2.0:1787739064.702256:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739064.702260:0:6529:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad951 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad94a expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.702270:0:6529:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad951 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad94a expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.702278:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d600, rc 0) 00000100:00100000:2.0:1787739064.702339:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306393c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6524:x1874579665710592:12345-0@lo:101:mdt00_001.0 Request processed in 3290us (3587us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.702348:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3304 00000100:00100000:3.0:1787739064.702444:0:6524:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968405 gen 2 00000004:00080000:3.0:1787739064.702450:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968400 committed transno 4294968405 00000020:00080000:3.0:1787739064.702456:0:6524:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa0730610e180 refcount 2 committed 0 result 0 batchid 382 00000020:00080000:3.0:1787739064.702460:0:6524:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07404f86000 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730610e780 00000020:00080000:3.0:1787739064.702463:0:6524:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].379 00000020:00080000:3.0:1787739064.702465:0:6524:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07404f86200 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07411bdaa00 00000020:00080000:3.0:1787739064.702468:0:6524:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].379 00000004:00080000:3.0:1787739064.702491:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968402 committed transno 4294968405 00000004:00080000:3.0:1787739064.702501:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968403 committed transno 4294968405 00000004:00080000:3.0:1787739064.702509:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968405 committed transno 4294968405 00000100:00100000:3.0:1787739064.702521:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306392940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665710592:0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787739064.702536:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad94a lrc: 4/0,1 mode: --/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad951 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.702547:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad94a lrc: 4/0,1 mode: --/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad951 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00080000:1.0:1787739064.702608:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 382 committed batchid 381 00000020:00080000:1.0:1787739064.702682:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 381 00000100:00100000:0.0:1787739064.704378:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739064.704522:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652493952 00000100:00100000:2.0:1787739064.704531:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652493952 00010000:00010000:2.0:1787739064.704572:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad927 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095145 expref: 8 pid: 13793 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739064.704616:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1547 00000100:00100000:2.0:1787739064.704627:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d12c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2409:x1874579652493952:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739064.704646:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739064.704662:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad927 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095145 expref: 8 pid: 13793 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.704677:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad927 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095145 expref: 8 pid: 13793 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.704700:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ad927 lrc: 0/0,0 mode: --/PR res: [0x240000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095145 expref: 8 pid: 13793 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.704716:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xca:0x0].0x0 (ffffa07433235400) 00010000:00010000:2.0:1787739064.704723:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad951 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad94a expref: 5 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.704733:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad951 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad94a expref: 5 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.704743:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad951 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad94a expref: 5 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.704784:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad951 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad94a expref: 5 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739064.704799:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f1a40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665710848:0@lo:105: 00000100:00100000:2.0:1787739064.704845:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739064.704886:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739064.704978:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665710848 00000100:00100000:1.0:1787739064.704983:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665710848 00000100:00100000:1.0:1787739064.705025:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1183 00000100:00100000:1.0:1787739064.705033:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665710848:12345-0@lo:105: 00010000:00010000:1.0:1787739064.705043:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad94a lrc: 5/0,1 mode: --/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad951 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.705053:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad94a lrc: 5/0,1 mode: --/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad951 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.705062:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad94a lrc: 5/0,1 mode: --/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad951 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.705070:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad94a lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad951 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.705077:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad94a lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ad951 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.705086:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad94a lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ad951 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.705103:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069ab800) 00000100:00100000:1.0:1787739064.705144:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665710848:12345-0@lo:105: Request processed in 112us (298us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.705152:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1183 00010000:00010000:3.0:1787739064.705208:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad94a lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad951 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.705218:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad94a lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad951 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.705884:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306392940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665710976:0@lo:403:mdt00_001.0 00000100:00100000:3.0:1787739064.705931:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739064.705968:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739064.706116:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665710976 00000100:00100000:1.0:1787739064.706122:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665710976 00000100:00100000:1.0:1787739064.706157:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3305 00000100:00100000:1.0:1787739064.706164:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306390b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6524:x1874579665710976:12345-0@lo:403:mdt00_001.0 00000100:00100000:1.0:1787739064.707770:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306390b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6524:x1874579665710976:12345-0@lo:403:mdt00_001.0 Request processed in 1600us (1832us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.707781:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3305 00000100:00100000:3.0:1787739064.708066:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306392940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665710976:0@lo:403:mdt00_001.0 00000040:00080000:3.0:1787739064.709110:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].381, 664 off582168 00000040:00080000:3.0:1787739064.709277:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].381, 744 off632216 00000004:00080000:3.0:1787739064.709283:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730610ec00 with version 726 00000100:00100000:2.0:1787739064.709450:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f1a40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665710848:0@lo:105: 00010000:00010000:2.0:1787739064.709468:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739064.709521:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d12c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2409:x1874579652493952:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 4893us (5147us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.709531:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1547 00000004:00080000:1.0:1787739064.709611:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 726 rpc_version 726 00000020:00100000:1.0:1787739064.709644:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739064.709719:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743562f480 x1874579665711232/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1448/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739064.709743:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562f480 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665711232:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739064.709918:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.709977:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739064.711541:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665711232 00000100:00100000:2.0:1787739064.711552:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665711232 00000100:00100000:3.0:1787739064.711757:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3306 00000100:00100000:3.0:1787739064.711767:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665711232:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739064.712061:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665711232:12345-0@lo:1000:osp_up1-0.0 Request processed in 296us (2142us total) trans 4294968406 rc 0/0 00000100:00100000:3.0:1787739064.712070:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3306 00000100:00100000:0.0:1787739064.712178:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562f480 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665711232:0@lo:1000:osp_up1-0.0 00000004:00200000:2.0:1787739064.712308:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1521:0x0]: returning size 4096 00010000:00010000:2.0:1787739064.712333:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad943 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712344:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad943 lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712352:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad943 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712359:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739064.712363:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad943 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712371:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad943 lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712386:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xca:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:2.0:1787739064.712389:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739064.712394:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad943 lrc: 2/0,0 mode: --/EX res: [0x200000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712400:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad943 lrc: 0/0,0 mode: --/EX res: [0x200000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712436:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad94a lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad951 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712446:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad94a lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad951 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712453:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad94a lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ad951 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712476:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739064.712485:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xca:0x0] 00010000:00010000:2.0:1787739064.712491:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad93c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1521:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712498:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad93c lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1521:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712504:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad93c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1521:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712510:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739064.712512:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad93c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1521:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712519:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad93c lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1521:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712527:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1521:0x0].0x0 (ffffa07403a6f000) 00010000:00010000:2.0:1787739064.712529:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739064.712533:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad93c lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1521:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712539:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad93c lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1521:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712554:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad92e lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712561:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad92e lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712566:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad92e lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712571:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad92e lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712575:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739064.712578:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad92e lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712584:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad92e lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712591:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:2.0:1787739064.712593:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739064.712596:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad92e lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712602:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad92e lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739064.712618:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000cf4f0680 reply state 000000000fd8b64c transno 4294983417 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ad935 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712629:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ad935 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.712636:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x101e480e (ffffa07307064c00) 00010000:00010000:2.0:1787739064.712639:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739064.712641:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787739064.712712:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17645:x1874579652493696:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 32126us (32909us total) trans 4294983417 rc 0/0 00000100:00100000:2.0:1787739064.712719:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23527 00010000:00010000:0.0:1787739064.713576:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739064.713585:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad94a lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ad951 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.713594:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739064.713600:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad94a lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ad951 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739064.713612:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad94a lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ad951 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.713647:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad94a lrc: 2/0,0 mode: --/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad951 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.713665:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739064.713688:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad94a lrc: 1/0,0 mode: --/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad951 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.713696:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ad94a lrc: 0/0,0 mode: --/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ad951 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.713713:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739064.713715:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739064.713717:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739064.714124:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665711360:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739064.714188:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739064.714329:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665711360 00000100:00100000:0.0:1787739064.714337:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665711360 00000100:00100000:0.0:1787739064.714389:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1351 00000100:00100000:0.0:1787739064.714398:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f1a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+7:3652:x1874579665711360:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739064.714435:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739064.714455:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ad951 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad94a expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.714470:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xca:0x0].0x0 (ffffa07433235400) 00010000:00010000:0.0:1787739064.714478:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739064.714514:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f1a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+6:3652:x1874579665711360:12345-0@lo:103:ldlm_bl.0 Request processed in 117us (325us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739064.714521:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1351 00000100:00100000:2.0:1787739064.714600:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665711360:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739064.738025:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739064.738123:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652494080 00000100:00100000:2.0:1787739064.738132:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652494080 00000100:00100000:2.0:1787739064.738202:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23528 00000100:00100000:2.0:1787739064.738213:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07411b76450 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17646:x1874579652494080:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739064.738235:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739064.738257:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad958 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09514c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.738320:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad95f lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.738335:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad95f lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.738346:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad95f lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.738360:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad966 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.738379:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad966 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.738386:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad966 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.738468:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad95f lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.738476:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad95f lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.738483:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad95f lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.738490:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad95f lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.738497:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739064.738501:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad95f lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.738508:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad95f lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.738521:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b000) 00010000:00010000:2.0:1787739064.738524:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739064.738528:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad95f lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.738535:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad95f lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.738552:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad966 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.738559:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad966 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.738565:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad966 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.738569:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739064.738572:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad966 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.738577:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad966 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.738584:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xe7d4651 (ffffa07403a6f300) 00010000:00010000:2.0:1787739064.738586:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739064.738589:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad966 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.738594:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad966 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.738627:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad958 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09514c expref: 8 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.738637:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b000) 00010000:00010000:2.0:1787739064.738640:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739064.738643:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1ad958 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09514c expref: 8 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.738664:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c400, rc 0) 00000100:00100000:2.0:1787739064.738740:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07411b76450 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17646:x1874579652494080:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 529us (734us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739064.738747:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23528 00000100:00100000:2.0:1787739064.747076:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739064.747225:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652494208 00000100:00100000:0.0:1787739064.747234:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652494208 00000100:00100000:0.0:1787739064.747292:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23529 00000100:00100000:0.0:1787739064.747305:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17646:x1874579652494208:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739064.747327:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739064.747349:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad96d lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095153 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.747465:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad974 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.747480:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad974 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.747492:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad974 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739064.747546:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:0.0:1787739064.747552:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739064.747593:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad96d lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095153 expref: 19 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.747616:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad974 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095153 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.747625:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9400) 00010000:00010000:0.0:1787739064.747629:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739064.747632:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a800, rc 0) 00000100:00100000:0.0:1787739064.747890:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17646:x1874579652494208:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 586us (820us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739064.747898:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23529 00000100:00100000:3.0:1787739064.766250:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739064.766350:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652494464 00000100:00100000:0.0:1787739064.766361:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652494464 00000100:00100000:0.0:1787739064.766473:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23530 00000100:00100000:0.0:1787739064.766485:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306392940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17646:x1874579652494464:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:0.0:1787739064.766543:0:6524:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739064.766571:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa600/0x7ee4453e4a1ad7c9 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0950ff expref: 7 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.766589:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b000) 00010000:00010000:0.0:1787739064.766599:0:6524:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739064.766670:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad97b lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.766683:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad97b lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.766695:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad97b lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.766713:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad982 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.766720:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad982 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.766727:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad982 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1787739064.770659:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].368, 2168 off559224 00000040:00080000:0.0:1787739064.770752:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].368, 2368 off610120 00000004:00080000:0.0:1787739064.770759:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa073068489c0 with version 701 00000004:00080000:1.0:1787739064.771671:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 701 rpc_version 701 00000020:00100000:1.0:1787739064.771714:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739064.771726:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743562e940 x1874579665711616/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 3976/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739064.771752:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562e940 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665711616:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739064.771802:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1787739064.771819:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665711744:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1787739064.771885:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.771938:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739064.771945:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739064.772040:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665711616 00000100:00100000:1.0:1787739064.772048:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665711616 00000100:00100000:1.0:1787739064.772093:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665711744 00000100:00100000:1.0:1787739064.772095:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665711744 00000100:00100000:1.0:1787739064.772107:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3307 00000100:00100000:1.0:1787739064.772114:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665711616:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739064.772403:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3308 00000100:00100000:3.0:1787739064.772444:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306391680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:3652:x1874579665711744:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1787739064.772468:0:6529:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1443966976 avail 1297727488 00000100:00100000:3.0:1787739064.772538:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306391680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:3652:x1874579665711744:12345-0@lo:41:osp-pre-0-1.0 Request processed in 127us (596us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739064.772546:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3308 00000100:00100000:1.0:1787739064.773542:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665711616:12345-0@lo:1000:osp_up0-1.0 Request processed in 1424us (1714us total) trans 4294983418 rc 0/0 00000100:00100000:1.0:1787739064.773552:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3307 00000100:00100000:1.0:1787739064.773655:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562e940 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665711616:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739064.775156:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665711744:0@lo:41:osp-pre-0-1.0 00010000:00010000:0.0:1787739064.780209:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad989 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x180:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.780229:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad989 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x180:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.780240:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad989 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x180:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.780262:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad990 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xc8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.780270:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad990 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.780276:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad990 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.780317:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ad997 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.780324:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ad997 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.780334:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ad997 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739064.780357:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0b40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665711872:0@lo:101:mdt00_001.0 00000100:00100000:0.0:1787739064.780436:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739064.780622:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074157c2780 going to sleep for 11 seconds 00000100:00100000:1.0:1787739064.782849:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665711872 00000100:00100000:1.0:1787739064.782863:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665711872 00000100:00100000:1.0:1787739064.782922:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3309 00000100:00100000:1.0:1787739064.782931:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6524:x1874579665711872:12345-0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739064.782955:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739064.783749:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad99e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xc8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad997 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.783782:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad99e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad997 expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.783796:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad99e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad997 expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.783805:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc8:0x0].0x0 (ffffa07433235500) 00010000:00010000:1.0:1787739064.783808:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739064.783812:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c000, rc 0) 00000100:00100000:1.0:1787739064.783888:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6524:x1874579665711872:12345-0@lo:101:mdt00_001.0 Request processed in 957us (3450us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.783895:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3309 00000100:00100000:2.0:1787739064.784061:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0b40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665711872:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739064.784080:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ad997 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad99e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.784090:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ad997 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ad99e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739064.784114:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000041293884 reply state 000000000019e064 transno 4294968407 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad990 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.784127:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad990 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.784134:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc8:0x0].0x0 (ffffa073068a9600) 00010000:00010000:2.0:1787739064.784136:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739064.784139:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787739064.784144:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ad997 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad99e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.784152:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ad997 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ad99e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739064.784167:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xc8:0x0] 00000004:00010000:2.0:1787739064.784173:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000041293884 reply state 000000000019e064 transno 4294968407 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad989 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x180:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.784182:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad989 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x180:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.784188:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x180:0x0].0x0 (ffffa073068a9200) 00010000:00010000:2.0:1787739064.784190:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739064.784192:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:2.0:1787739064.784196:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x180:0x0]: returning size 4096 00010000:00010000:2.0:1787739064.784206:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad97b lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.784212:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad97b lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.784218:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad97b lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.784224:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad97b lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.784230:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609aa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739064.784234:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad97b lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.784241:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad97b lrc: 5/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.784249:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa073068a9d00) 00010000:00010000:2.0:1787739064.784252:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739064.784254:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad97b lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.784261:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad97b lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739064.784276:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000041293884 reply state 000000000019e064 transno 4294968407 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad982 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.784284:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad982 lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.784289:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xe7d4651 (ffffa073068a9000) 00010000:00010000:2.0:1787739064.784291:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739064.784293:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 2 00000100:00100000:2.0:1787739064.784368:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306392940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17646:x1874579652494464:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 17884us (18127us total) trans 4294968407 rc 0/0 00000100:00100000:2.0:1787739064.784376:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23530 00000100:00100000:1.0:1787739064.812512:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739064.812605:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652494720 00000100:00100000:2.0:1787739064.812615:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652494720 00000100:00100000:2.0:1787739064.812687:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23531 00000100:00100000:2.0:1787739064.812697:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17647:x1874579652494720:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:2.0:1787739064.812895:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17647:x1874579652494720:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 199us (392us total) trans 0 rc -17/-17 00000100:00100000:2.0:1787739064.812903:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23531 00000100:00100000:3.0:1787739064.831248:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739064.831312:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652494848 00000100:00100000:2.0:1787739064.831324:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652494848 00000100:00100000:2.0:1787739064.831394:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23532 00000100:00100000:2.0:1787739064.831405:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306390000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17648:x1874579652494848:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739064.831444:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739064.831477:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad9a5 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09515a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.831543:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad9ac lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.831557:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad9ac lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.831568:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad9ac lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.831809:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad9b3 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.831820:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad9b3 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.831828:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad9b3 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.831897:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ad9ba lrc: 3/1,0 mode: --/PR res: [0x240000402:0x180:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.831906:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ad9ba lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x180:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.831913:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ad9ba lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x180:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739064.831996:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x180:0x0]: returning size 4096 00000004:00200000:2.0:1787739064.832001:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:2.0:1787739064.832030:0:13793:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ad9ba lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x180:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.832043:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad9ac lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.832050:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad9ac lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.832057:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad9ac lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.832064:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad9ac lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.832070:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739064.832074:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad9ac lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.832081:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad9ac lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.832093:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:2.0:1787739064.832096:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739064.832101:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad9ac lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.832107:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ad9ac lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.832123:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad9b3 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.832133:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad9b3 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.832138:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad9b3 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.832143:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739064.832146:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad9b3 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.832152:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad9b3 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.832159:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xe7d4651 (ffffa073068a9000) 00010000:00010000:2.0:1787739064.832161:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739064.832165:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad9b3 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.832170:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1ad9b3 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.832196:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ad9a5 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09515a expref: 10 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.832215:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ad9ba lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x180:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09515a expref: 9 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.832223:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x180:0x0].0x0 (ffffa073068a9200) 00010000:00010000:2.0:1787739064.832226:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787739064.832229:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633da00, rc 0) 00000100:00100000:2.0:1787739064.832297:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306390000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17648:x1874579652494848:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 893us (1057us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.832305:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23532 00000100:00100000:2.0:1787739064.836731:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739064.838096:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652494976 00000100:00100000:1.0:1787739064.838109:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652494976 00000100:00100000:1.0:1787739064.838169:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23533 00000100:00100000:1.0:1787739064.838179:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17648:x1874579652494976:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739064.838200:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739064.838220:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad9c1 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xc8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095161 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.838278:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad9c8 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xc8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.838291:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad9c8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.838303:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad9c8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739064.838351:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xc8:0x0]: returning size 4096 00000004:00200000:1.0:1787739064.838357:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739064.838392:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ad9c1 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095161 expref: 11 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.838409:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad9c8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095161 expref: 10 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.838435:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc8:0x0].0x0 (ffffa073068a9600) 00010000:00010000:1.0:1787739064.838439:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739064.838442:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d600, rc 0) 00000100:00100000:1.0:1787739064.838503:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17648:x1874579652494976:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 325us (1833us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739064.838510:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23533 00000100:00100000:1.0:1787739064.843280:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739064.843372:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652495104 00000100:00100000:0.0:1787739064.843382:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652495104 00000100:00100000:0.0:1787739064.843470:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23534 00000100:00100000:0.0:1787739064.843480:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17648:x1874579652495104:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739064.843501:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739064.843522:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad9cf lrc: 2/0,0 mode: --/CR res: [0x200000401:0xc8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095168 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.843580:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad9d6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.843596:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad99e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad997 expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.843605:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad9d6 lrc: 4/1,0 mode: --/PR res: [0x200000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.843644:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad99e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ad997 expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739064.843660:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0b40 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6525:1874579665712000:0@lo:104: 00000100:00100000:0.0:1787739064.843707:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739064.843752:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074157c2780 going to sleep for 11 seconds 00000100:00100000:0.0:1787739064.843769:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665712000 00000100:00100000:0.0:1787739064.843773:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665712000 00000100:00100000:0.0:1787739064.843804:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1184 00000100:00100000:0.0:1787739064.843812:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d0f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6525:x1874579665712000:12345-0@lo:104: 00010000:00010000:0.0:1787739064.843825:0:6506:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ad997 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad99e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.843878:0:6506:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787739064.843884:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d0f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6525:x1874579665712000:12345-0@lo:104: Request processed in 74us (178us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739064.843891:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1184 00000100:00100000:0.0:1787739064.843933:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0b40 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6525:1874579665712000:0@lo:104: 00010000:00010000:0.0:1787739064.843955:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad9d6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.845109:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739064.845120:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ad997 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ad99e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.845131:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739064.845137:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ad997 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xc8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ad99e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739064.845146:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ad997 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xc8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ad99e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.845184:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ad997 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xc8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad99e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.845191:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739064.845219:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ad997 lrc: 1/0,0 mode: --/PW res: [0x200000401:0xc8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad99e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.845226:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a000/0x7ee4453e4a1ad997 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xc8:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ad99e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.845241:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739064.845243:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739064.845244:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739064.845833:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787739064.845851:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665712128:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739064.845902:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739064.846014:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665712128 00000100:00100000:2.0:1787739064.846021:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665712128 00010000:00010000:2.0:1787739064.846045:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad99e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ad997 expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739064.846104:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1548 00000100:00100000:2.0:1787739064.846112:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306391a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3651:x1874579665712128:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739064.846125:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:0.0:1787739064.852662:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983418 is committed 00000001:00080000:0.0:1787739064.852692:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983417 is committed 00000020:00080000:0.0:1787739064.852701:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 382 batchid updated 00000020:00080000:0.0:1787739064.852719:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07306848780 refcount 2 committed 0 result 0 batchid 383 00000020:00080000:0.0:1787739064.852725:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2b00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa073068483c0 00000020:00080000:0.0:1787739064.852728:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].380 00000020:00080000:0.0:1787739064.852732:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2800 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306aac800 00000020:00080000:0.0:1787739064.852734:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].380 00010000:00010000:0.0:1787739064.852808:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ad99e lrc: 0/0,0 mode: --/PW res: [0x200000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ad997 expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.852827:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc8:0x0].0x0 (ffffa07433235500) 00010000:00010000:0.0:1787739064.852832:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad9d6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.852841:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad9d6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.852849:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad9d6 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.852877:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000020:00080000:2.0:1787739064.852890:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 383 committed batchid 382 00000100:00100000:0.0:1787739064.852950:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306391a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665712128:12345-0@lo:103:ldlm_bl.0 Request processed in 6835us (7044us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739064.852960:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1548 00010000:00010000:0.0:1787739064.852992:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad9d6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.853001:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad9d6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739064.853013:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306887300 with version 727 00000020:00080000:2.0:1787739064.853028:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 382 cancel update log [0x240000401:0x3:0x0].379: rc = 0 00000004:00200000:0.0:1787739064.853066:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xc8:0x0]: returning size 4096 00000020:00080000:2.0:1787739064.853070:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 382 cancel update log [0x200000400:0x3:0x0].379: rc = 0 00000004:00200000:0.0:1787739064.853072:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000020:00080000:2.0:1787739064.853083:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 382 00000004:00080000:2.0:1787739064.853091:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 727 rpc_version 727 00000020:00100000:2.0:1787739064.853118:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00010000:00010000:0.0:1787739064.853118:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ad9cf lrc: 0/0,0 mode: --/CR res: [0x200000401:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095168 expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739064.853129:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074080f03c0 x1874579665712384/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787739064.853139:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad9d6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095168 expref: 17 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.853150:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc8:0x0].0x0 (ffffa07433235500) 00000100:00100000:2.0:1787739064.853154:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f03c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665712384:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787739064.853155:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739064.853159:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609aa00, rc 0) 00000100:00100000:2.0:1787739064.853204:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739064.853210:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17648:x1874579652495104:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 9730us (9935us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739064.853218:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23534 00000100:00100000:2.0:1787739064.853240:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068870c0 going to sleep for 11 seconds 00010000:00010000:3.0:1787739064.853323:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad8d3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853332:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad8d3 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853344:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad8d3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853351:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739064.853355:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad8d3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853362:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad8d3 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853373:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x101e480e (ffffa07307064c00) 00010000:00010000:3.0:1787739064.853375:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739064.853379:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad8d3 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853385:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ad8d3 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853396:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad8da lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1521:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853402:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad8da lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1521:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853409:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad8da lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1521:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853446:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739064.853449:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad8da lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1521:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853460:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad8da lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1521:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853468:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1521:0x0].0x0 (ffffa07403a6f000) 00010000:00010000:3.0:1787739064.853470:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739064.853473:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad8da lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1521:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853478:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ad8da lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1521:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853494:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad8e1 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853501:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad8e1 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853506:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad8e1 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853510:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633de00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739064.853513:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad8e1 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853519:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad8e1 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853526:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xca:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739064.853528:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739064.853531:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad8e1 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853541:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ad8e1 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853558:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ad935 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853565:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ad935 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853570:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ad935 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853575:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739064.853578:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ad935 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853584:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ad935 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853592:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x101e480e (ffffa07307064c00) 00010000:00010000:3.0:1787739064.853594:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739064.853597:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ad935 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.853602:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ad935 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x101e480e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.853658:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f21c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665712128:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739064.853930:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665712384 00000100:00100000:0.0:1787739064.853937:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665712384 00000100:00100000:0.0:1787739064.853981:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3310 00000100:00100000:0.0:1787739064.853990:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665712384:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739064.854174:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665712384:12345-0@lo:1000:osp_up1-0.0 Request processed in 185us (970us total) trans 4294968408 rc 0/0 00000100:00100000:0.0:1787739064.854182:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3310 00000100:00100000:3.0:1787739064.855284:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f03c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665712384:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739064.861610:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739064.861727:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652495360 00000100:00100000:3.0:1787739064.861737:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652495360 00000100:00100000:3.0:1787739064.861799:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23535 00000100:00100000:3.0:1787739064.861811:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17648:x1874579652495360:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:3.0:1787739064.861865:0:6523:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739064.861891:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ad9ba lrc: 0/0,0 mode: --/PR res: [0x240000402:0x180:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09515a expref: 10 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.861910:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x180:0x0].0x0 (ffffa073068a9200) 00010000:00010000:3.0:1787739064.861915:0:6523:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739064.861942:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad9dd lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.861955:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad9dd lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.861965:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad9dd lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.861976:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad9e4 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.861983:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad9e4 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.861989:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad9e4 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.862079:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad9eb lrc: 3/0,1 mode: --/EX res: [0x240000402:0x180:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.862092:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad9eb lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x180:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.862100:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad9eb lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x180:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.862118:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad9f2 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.862128:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad9c8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095161 expref: 9 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.862135:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad9f2 lrc: 4/0,1 mode: --/EX res: [0x240000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.862168:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad9c8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095161 expref: 9 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.862181:0:6523:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad9c8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095161 expref: 9 pid: 6524 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.862195:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306393840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665712512:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739064.862275:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e840 going to sleep for 11 seconds 00000100:00100000:3.0:1787739064.866579:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306393840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665712512:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739064.866604:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad9f2 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739064.868224:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739064.868297:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652495488 00000100:00100000:2.0:1787739064.868305:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652495488 00010000:00010000:2.0:1787739064.868340:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad9c8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095161 expref: 10 pid: 6524 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739064.868378:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1549 00000100:00100000:2.0:1787739064.868388:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2406:x1874579652495488:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739064.868407:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739064.868436:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad9c8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095161 expref: 10 pid: 6524 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.868450:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad9c8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095161 expref: 10 pid: 6524 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.868472:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ad9c8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095161 expref: 10 pid: 6524 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.868486:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc8:0x0].0x0 (ffffa073068a9600) 00010000:00010000:2.0:1787739064.868491:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad9f2 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.868499:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad9f2 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.868505:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad9f2 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.868528:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739064.868576:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:2406:x1874579652495488:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 190us (358us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.868585:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1549 00010000:00010000:3.0:1787739064.868856:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad9f2 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.868866:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad9f2 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.868910:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ad9f9 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.868917:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ad9f9 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.868927:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ad9f9 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739064.868945:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306393840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665712640:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739064.868986:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739064.869041:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e840 going to sleep for 11 seconds 00000100:00100000:2.0:1787739064.869150:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665712640 00000100:00100000:2.0:1787739064.869155:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665712640 00000100:00100000:2.0:1787739064.869191:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3311 00000100:00100000:2.0:1787739064.869199:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306393480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+5:6523:x1874579665712640:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739064.869214:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739064.869230:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ada00 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ad9f9 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.869251:0:6529:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad9d6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095168 expref: 14 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.869261:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ada00 lrc: 4/0,0 mode: --/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad9f9 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.869281:0:6529:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad9d6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095168 expref: 14 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.869293:0:6529:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad9d6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095168 expref: 14 pid: 6525 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739064.869305:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f1e00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6529:1874579665712768:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739064.869742:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887600 going to sleep for 11 seconds 00000100:00100000:3.0:1787739064.878902:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f1e00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6529:1874579665712768:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739064.878945:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ada00 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad9f9 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.878959:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc8:0x0].0x0 (ffffa07433235500) 00010000:00010000:3.0:1787739064.878962:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739064.878966:0:6529:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ada00 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad9f9 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.878975:0:6529:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ada00 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad9f9 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739064.878982:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d800, rc 0) 00000100:00100000:3.0:1787739064.879067:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306393480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:6523:x1874579665712640:12345-0@lo:101:mdt00_000.0 Request processed in 9868us (10079us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739064.879076:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3311 00000100:00100000:0.0:1787739064.879153:0:6523:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983418 gen 2 00000004:00080000:0.0:1787739064.879159:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983414 committed transno 4294983418 00000020:00080000:0.0:1787739064.879165:0:6523:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07306848b40 refcount 2 committed 0 result 0 batchid 374 00000020:00080000:0.0:1787739064.879169:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2a00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07306848cc0 00000020:00080000:0.0:1787739064.879172:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].367 00000020:00080000:0.0:1787739064.879175:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2f00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073040e9e00 00000020:00080000:0.0:1787739064.879178:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].366 00000004:00080000:0.0:1787739064.879206:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983416 committed transno 4294983418 00000004:00080000:0.0:1787739064.879216:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983418 committed transno 4294983418 00000100:00100000:0.0:1787739064.879226:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306393840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665712640:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739064.879244:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ad9f9 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ada00 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.879254:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ad9f9 lrc: 4/0,1 mode: --/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ada00 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00080000:3.0:1787739064.879297:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 374 committed batchid 372 00000020:00080000:3.0:1787739064.879348:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 372 00000100:00100000:2.0:1787739064.882333:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739064.882512:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652495616 00000100:00100000:0.0:1787739064.882521:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652495616 00010000:00010000:0.0:1787739064.882559:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad9d6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095168 expref: 15 pid: 6525 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739064.882598:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1550 00000100:00100000:0.0:1787739064.882607:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f2d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2408:x1874579652495616:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739064.882621:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739064.882628:0:6510:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad9d6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095168 expref: 15 pid: 6525 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.882642:0:6510:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad9d6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095168 expref: 15 pid: 6525 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.882675:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ad9d6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc8:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095168 expref: 15 pid: 6525 timeout: 751 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.882692:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc8:0x0].0x0 (ffffa07433235500) 00010000:00010000:0.0:1787739064.882703:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ada00 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad9f9 expref: 5 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.882722:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ada00 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad9f9 expref: 5 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.882731:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ada00 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ad9f9 expref: 5 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.882764:0:6510:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ada00 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ad9f9 expref: 5 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739064.882788:0:6510:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d2d00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6510:1874579665712896:0@lo:105: 00000100:00100000:0.0:1787739064.882835:0:6510:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739064.882874:0:6510:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303937000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739064.883029:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665712896 00000100:00100000:2.0:1787739064.883034:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665712896 00000100:00100000:2.0:1787739064.883069:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1185 00000100:00100000:2.0:1787739064.883076:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d30c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6510:x1874579665712896:12345-0@lo:105: 00010000:00010000:2.0:1787739064.883088:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ad9f9 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ada00 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.883098:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ad9f9 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ada00 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.883112:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ad9f9 lrc: 5/0,1 mode: --/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ada00 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.883120:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ad9f9 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ada00 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.883127:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ad9f9 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ada00 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.883136:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ad9f9 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ada00 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.883156:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069ab200) 00000100:00100000:2.0:1787739064.883191:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d30c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6510:x1874579665712896:12345-0@lo:105: Request processed in 116us (355us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.883199:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1185 00010000:00010000:2.0:1787739064.883224:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ad9f9 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ada00 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.883233:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ad9f9 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ada00 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739064.885232:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665713024:0@lo:403:mdt00_000.0 00000100:00100000:2.0:1787739064.885303:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739064.885339:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739064.886899:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665713024 00000100:00100000:2.0:1787739064.886916:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665713024 00000100:00100000:2.0:1787739064.886995:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3312 00000100:00100000:2.0:1787739064.887015:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:6523:x1874579665713024:12345-0@lo:403:mdt00_000.0 00000100:00100000:0.0:1787739064.888506:0:6510:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d2d00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6510:1874579665712896:0@lo:105: 00010000:00010000:0.0:1787739064.888530:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739064.888591:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f2d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2408:x1874579652495616:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 5982us (6265us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739064.888602:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1550 00000100:00100000:2.0:1787739064.891595:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:6523:x1874579665713024:12345-0@lo:403:mdt00_000.0 Request processed in 4583us (6283us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.891610:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3312 00000100:00100000:1.0:1787739064.891803:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f3c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665713024:0@lo:403:mdt00_000.0 00000040:00080000:1.0:1787739064.892741:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].369, 664 off559888 00000040:00080000:1.0:1787739064.892917:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].369, 744 off610864 00000004:00080000:1.0:1787739064.892923:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07306241240 with version 702 00000004:00080000:2.0:1787739064.894280:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 702 rpc_version 702 00000020:00100000:2.0:1787739064.894320:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739064.894330:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074080f0780 x1874579665713280/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1448/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739064.894352:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f0780 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665713280:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739064.894456:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739064.894492:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068876c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739064.894817:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665713280 00000100:00100000:1.0:1787739064.894823:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665713280 00000100:00100000:1.0:1787739064.894869:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3313 00000100:00100000:1.0:1787739064.894877:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:7467:x1874579665713280:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739064.895201:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665713280:12345-0@lo:1000:osp_up0-1.0 Request processed in 323us (743us total) trans 4294983419 rc 0/0 00000100:00100000:1.0:1787739064.895209:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3313 00000100:00100000:2.0:1787739064.895549:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f0780 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665713280:0@lo:1000:osp_up0-1.0 00000004:00200000:1.0:1787739064.895632:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x180:0x0]: returning size 4096 00010000:00010000:1.0:1787739064.895887:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad9f2 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.895898:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad9f2 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.895906:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad9f2 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.895912:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739064.895916:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad9f2 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.895923:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad9f2 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.895938:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc8:0x0].0x0 (ffffa073068a9600) 00010000:00010000:1.0:1787739064.895941:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739064.895946:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad9f2 lrc: 2/0,0 mode: --/EX res: [0x240000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.895953:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ad9f2 lrc: 0/0,0 mode: --/EX res: [0x240000400:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.895969:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ad9f9 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ada00 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.895977:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ad9f9 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ada00 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.895983:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ad9f9 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ada00 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.896012:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739064.896021:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xc8:0x0] 00010000:00010000:1.0:1787739064.896028:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad9eb lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x180:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.896034:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad9eb lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x180:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.896039:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad9eb lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x180:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.896043:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739064.896045:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad9eb lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x180:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.896050:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad9eb lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x180:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.896060:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x180:0x0].0x0 (ffffa073068a9200) 00010000:00010000:1.0:1787739064.896063:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739064.896065:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad9eb lrc: 2/0,0 mode: --/EX res: [0x240000402:0x180:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.896070:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ad9eb lrc: 0/0,0 mode: --/EX res: [0x240000402:0x180:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.896085:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad9dd lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.896091:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad9dd lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.896097:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad9dd lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.896102:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad9dd lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.896108:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739064.896111:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad9dd lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.896117:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad9dd lrc: 5/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.896124:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b500) 00010000:00010000:1.0:1787739064.896126:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739064.896129:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad9dd lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.896138:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ad9dd lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739064.896154:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000004af3119 reply state 00000000693d2e12 transno 4294968409 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad9e4 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.896166:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad9e4 lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.896174:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xe7d4651 (ffffa073068a9000) 00010000:00010000:1.0:1787739064.896176:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739064.896179:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787739064.896247:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17648:x1874579652495360:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 34437us (34647us total) trans 4294968409 rc 0/0 00000100:00100000:1.0:1787739064.896854:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23535 00010000:00010000:2.0:1787739064.898524:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739064.898535:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ad9f9 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ada00 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.898545:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739064.898552:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ad9f9 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ada00 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739064.898562:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ad9f9 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ada00 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.898602:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ad9f9 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ada00 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.898614:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739064.898646:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ad9f9 lrc: 1/0,0 mode: --/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ada00 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.898663:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ad9f9 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ada00 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.898679:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739064.898688:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739064.898690:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739064.898698:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787739064.898717:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665713408:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739064.898771:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739064.898919:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665713408 00000100:00100000:2.0:1787739064.898926:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665713408 00000100:00100000:2.0:1787739064.898968:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1352 00000100:00100000:2.0:1787739064.898977:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f3c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+7:3650:x1874579665713408:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739064.898990:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739064.899017:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ada00 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ad9f9 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.899031:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc8:0x0].0x0 (ffffa07433235500) 00010000:00010000:2.0:1787739064.899039:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739064.899070:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f3c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+6:3650:x1874579665713408:12345-0@lo:103:ldlm_bl.0 Request processed in 95us (299us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.899076:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1352 00000100:00100000:2.0:1787739064.899112:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665713408:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739064.928980:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739064.929057:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652495744 00000100:00100000:1.0:1787739064.929066:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652495744 00000100:00100000:1.0:1787739064.929127:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23536 00000100:00100000:1.0:1787739064.929137:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17649:x1874579652495744:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739064.929157:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739064.929179:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ada07 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09516f expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.929241:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ada0e lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.929257:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ada0e lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.929273:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ada0e lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.929288:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ada15 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.929305:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ada15 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.929314:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ada15 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.929361:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ada0e lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.929370:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ada0e lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.929375:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ada0e lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.929382:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ada0e lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.929388:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739064.929392:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ada0e lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.929400:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ada0e lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.929434:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9400) 00010000:00010000:1.0:1787739064.929439:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739064.929444:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ada0e lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.929451:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ada0e lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.929461:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ada15 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.929466:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ada15 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.929472:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ada15 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.929477:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1de00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739064.929480:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ada15 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.929486:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ada15 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.929494:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x7ed215cf (ffffa07433235d00) 00010000:00010000:1.0:1787739064.929497:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739064.929499:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ada15 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.929504:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ada15 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.929534:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ada07 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09516f expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.929548:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9400) 00010000:00010000:1.0:1787739064.929551:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739064.929554:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ada07 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09516f expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.929563:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c200, rc 0) 00000100:00100000:1.0:1787739064.929622:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17649:x1874579652495744:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 487us (651us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739064.929630:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23536 00000100:00100000:1.0:1787739064.938138:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739064.938217:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652495872 00000100:00100000:0.0:1787739064.938227:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652495872 00000100:00100000:0.0:1787739064.938282:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23537 00000100:00100000:0.0:1787739064.938293:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17649:x1874579652495872:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739064.938328:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739064.938351:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ada1c lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095176 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.938479:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ada23 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.938497:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ada23 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.938508:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ada23 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739064.938556:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x5:0x0]: returning size 4096 00000004:00200000:0.0:1787739064.938561:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739064.938596:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ada1c lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095176 expref: 12 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.938611:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ada23 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095176 expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.938619:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064a00) 00010000:00010000:0.0:1787739064.938622:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739064.938626:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b600, rc 0) 00000100:00100000:0.0:1787739064.938954:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17649:x1874579652495872:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 662us (825us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739064.938963:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23537 00000100:00100000:3.0:1787739064.951940:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739064.952028:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652496128 00000100:00100000:1.0:1787739064.952039:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652496128 00000100:00100000:1.0:1787739064.952099:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23538 00000100:00100000:1.0:1787739064.952110:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306390780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17649:x1874579652496128:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:1.0:1787739064.952169:0:6523:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739064.952199:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1ad974 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095153 expref: 14 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.952219:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa073068a9400) 00010000:00010000:1.0:1787739064.952229:0:6523:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739064.952291:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ada2a lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.952315:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ada2a lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.952325:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ada2a lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.952339:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ada31 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.952346:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ada31 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739064.952352:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ada31 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1787739064.953229:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].382, 2168 off584336 00000040:00080000:1.0:1787739064.953307:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].382, 2368 off634584 00000004:00080000:1.0:1787739064.953313:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa073062410c0 with version 728 00000004:00080000:0.0:1787739064.953446:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 728 rpc_version 728 00000020:00100000:0.0:1787739064.953481:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739064.953493:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073060d2d00 x1874579665713664/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 3976/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739064.953517:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d2d00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665713664:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739064.953576:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739064.953618:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074332da180 going to sleep for 11 seconds 00000100:00100000:1.0:1787739064.953938:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665713664 00000100:00100000:1.0:1787739064.953944:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665713664 00000100:00100000:1.0:1787739064.954126:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3314 00000100:00100000:1.0:1787739064.954136:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665713664:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739064.954560:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665713664:12345-0@lo:1000:osp_up1-0.0 Request processed in 424us (982us total) trans 4294968410 rc 0/0 00000100:00100000:1.0:1787739064.954569:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3314 00000100:00100000:3.0:1787739064.954677:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d2d00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665713664:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787739064.956640:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ada38 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1523:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.956659:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ada38 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1523:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.956671:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ada38 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1523:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.956690:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ada3f lrc: 3/0,1 mode: --/PW res: [0x200000400:0xcb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.956697:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ada3f lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.956703:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ada3f lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.956737:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1ada46 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xcb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.956744:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1ada46 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.956753:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1ada46 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739064.956775:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d1a40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665713792:0@lo:101:mdt00_000.0 00000100:00100000:0.0:1787739064.956834:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739064.956884:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074332dad80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739064.956940:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665713792 00000100:00100000:2.0:1787739064.956947:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665713792 00000100:00100000:2.0:1787739064.956996:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3315 00000100:00100000:2.0:1787739064.957013:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6523:x1874579665713792:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739064.957033:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739064.957048:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ada4d lrc: 2/0,0 mode: --/PW res: [0x240000401:0xcb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ada46 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.957067:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ada4d lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ada46 expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.957078:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ada4d lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ada46 expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739064.957085:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xcb:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:2.0:1787739064.957088:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739064.957091:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d800, rc 0) 00000100:00100000:2.0:1787739064.957132:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:6523:x1874579665713792:12345-0@lo:101:mdt00_000.0 Request processed in 129us (298us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739064.957139:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3315 00000100:00100000:0.0:1787739064.957454:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d1a40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665713792:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739064.957470:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1ada46 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ada4d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.957482:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1ada46 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ada4d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739064.957501:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000213ad907 reply state 000000009ce7a329 transno 4294983420 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ada3f lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.957512:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ada3f lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.957519:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xcb:0x0].0x0 (ffffa07307064600) 00010000:00010000:0.0:1787739064.957521:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739064.957523:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739064.957529:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1ada46 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ada4d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.957536:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1ada46 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ada4d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739064.957546:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xcb:0x0] 00000004:00010000:0.0:1787739064.957552:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000213ad907 reply state 000000009ce7a329 transno 4294983420 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ada38 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1523:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.957559:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ada38 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1523:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.957565:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1523:0x0].0x0 (ffffa07307064900) 00010000:00010000:0.0:1787739064.957568:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739064.957569:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787739064.957574:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1523:0x0]: returning size 4096 00010000:00010000:0.0:1787739064.957585:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ada2a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.957590:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ada2a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.957595:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ada2a lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.957601:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ada2a lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.957607:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739064.957611:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ada2a lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.957618:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ada2a lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.957628:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235d00) 00010000:00010000:0.0:1787739064.957630:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739064.957634:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ada2a lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.957640:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ada2a lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739064.957666:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000213ad907 reply state 000000009ce7a329 transno 4294983420 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ada31 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.957675:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ada31 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739064.957686:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x7ed215cf (ffffa07433235b00) 00010000:00010000:0.0:1787739064.957689:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739064.957690:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 2 00000100:00100000:0.0:1787739064.957764:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306390780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17649:x1874579652496128:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 5654us (5830us total) trans 4294983420 rc 0/0 00000100:00100000:0.0:1787739064.957783:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23538 00000100:00100000:3.0:1787739064.997935:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739064.998000:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652496384 00000100:00100000:2.0:1787739064.998021:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652496384 00000100:00100000:2.0:1787739064.998107:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23539 00000100:00100000:2.0:1787739064.998117:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306391e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17650:x1874579652496384:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:2.0:1787739064.998306:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306391e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17650:x1874579652496384:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 190us (379us total) trans 0 rc -17/-17 00000100:00100000:2.0:1787739064.998316:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23539 00000100:00100000:2.0:1787739065.028646:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739065.028702:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652496512 00000100:00100000:1.0:1787739065.028712:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652496512 00000100:00100000:1.0:1787739065.028838:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23540 00000100:00100000:1.0:1787739065.028859:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17651:x1874579652496512:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739065.028880:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739065.028904:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ada54 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09517d expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.028966:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ada5b lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.028978:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ada5b lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.028991:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ada5b lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.029014:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada62 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.029022:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada62 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.029030:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada62 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.029107:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ada69 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1523:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.029116:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ada69 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1523:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.029123:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ada69 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1523:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787739065.029140:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000004:00200000:1.0:1787739065.029298:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1523:0x0]: returning size 4096 00000004:00200000:1.0:1787739065.029304:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:1.0:1787739065.029327:0:6524:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ada69 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1523:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.029340:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ada5b lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.029352:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ada5b lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.029481:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ada5b lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.029486:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ada5b lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.029491:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ca00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739065.029493:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ada5b lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.029624:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ada5b lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.029635:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235300) 00010000:00010000:1.0:1787739065.029637:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739065.029640:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ada5b lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.029644:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ada5b lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.029654:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada62 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.029657:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada62 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.029661:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada62 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.029665:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739065.029666:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada62 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.029670:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada62 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.029673:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x7ed215cf (ffffa07433235b00) 00010000:00010000:1.0:1787739065.029675:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739065.029676:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada62 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.029679:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada62 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.029698:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ada54 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09517d expref: 17 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.029712:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ada69 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1523:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09517d expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.029718:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1523:0x0].0x0 (ffffa07307064900) 00010000:00010000:1.0:1787739065.029720:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787739065.029723:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c200, rc 0) 00000100:00100000:1.0:1787739065.029782:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17651:x1874579652496512:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 935us (1143us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739065.029790:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23540 00000100:00100000:3.0:1787739065.033842:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739065.033911:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652496640 00000100:00100000:0.0:1787739065.033921:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652496640 00000100:00100000:0.0:1787739065.033988:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23541 00000100:00100000:0.0:1787739065.033999:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073063912c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17651:x1874579652496640:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739065.034035:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739065.034056:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ada70 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xcb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095184 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.034113:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ada77 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xcb:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.034127:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ada77 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.034137:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ada77 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739065.034183:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xcb:0x0]: returning size 4096 00000004:00200000:0.0:1787739065.034188:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739065.034222:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ada70 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095184 expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.034246:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ada77 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095184 expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.034256:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xcb:0x0].0x0 (ffffa07307064600) 00010000:00010000:0.0:1787739065.034259:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739065.034263:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a200, rc 0) 00000100:00100000:0.0:1787739065.036170:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073063912c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17651:x1874579652496640:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 2164us (2329us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.036182:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23541 00000100:00100000:3.0:1787739065.044760:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739065.045139:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652496768 00000100:00100000:1.0:1787739065.045149:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652496768 00000100:00100000:1.0:1787739065.045224:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23542 00000100:00100000:1.0:1787739065.045235:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073063921c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17651:x1874579652496768:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739065.045256:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739065.045279:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ada7e lrc: 2/0,0 mode: --/CR res: [0x240000401:0xcb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09518b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.045340:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada85 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xcb:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.045366:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ada4d lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ada46 expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.045375:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada85 lrc: 4/1,0 mode: --/PR res: [0x240000401:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.045444:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ada4d lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ada46 expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739065.045460:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562d2c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665713920:0@lo:104: 00000100:00100000:1.0:1787739065.046132:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1787739065.046133:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787739065.046197:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665713920 00000100:00100000:0.0:1787739065.046203:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665713920 00000100:00100000:1.0:1787739065.046237:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.046250:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1186 00000100:00100000:0.0:1787739065.046258:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:13793:x1874579665713920:12345-0@lo:104: 00010000:00010000:0.0:1787739065.046271:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1ada46 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ada4d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.046322:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787739065.046328:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:13793:x1874579665713920:12345-0@lo:104: Request processed in 71us (197us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.046334:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1186 00000100:00100000:1.0:1787739065.046339:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562d2c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665713920:0@lo:104: 00010000:00010000:1.0:1787739065.046358:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada85 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.046565:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739065.046572:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1ada46 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ada4d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.046581:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739065.046585:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1ada46 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xcb:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ada4d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739065.046593:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1ada46 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xcb:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1ada4d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.046622:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1ada46 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xcb:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ada4d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.046628:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739065.046660:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1ada46 lrc: 1/0,0 mode: --/PW res: [0x240000401:0xcb:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ada4d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.046671:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609b800/0x7ee4453e4a1ada46 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xcb:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ada4d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.046687:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739065.046689:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739065.046690:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739065.046753:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665714048:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739065.046787:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739065.046853:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665714048 00000100:00100000:3.0:1787739065.046857:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665714048 00010000:00010000:3.0:1787739065.046877:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ada4d lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ada46 expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739065.046905:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1551 00000100:00100000:3.0:1787739065.046912:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba4450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+9:3652:x1874579665714048:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739065.046922:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:1.0:1787739065.067779:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968410 is committed 00000001:00080000:1.0:1787739065.067806:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968409 is committed 00000020:00080000:1.0:1787739065.067816:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 374 batchid updated 00000020:00080000:1.0:1787739065.067826:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa073068483c0 refcount 2 committed 0 result 0 batchid 375 00000020:00080000:1.0:1787739065.067830:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2c00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa073068489c0 00000020:00080000:1.0:1787739065.067833:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].368 00000020:00080000:1.0:1787739065.067836:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2500 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07411bdaa00 00000020:00080000:1.0:1787739065.067838:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].368 00000020:00080000:1.0:1787739065.067852:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 373 batchid updated 00000020:00080000:1.0:1787739065.067867:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 375 committed batchid 374 00000004:00080000:1.0:1787739065.067976:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730623d0c0 with version 703 00000020:00080000:1.0:1787739065.067986:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 373 cancel update log [0x200000401:0x3:0x0].366: rc = 0 00000020:00080000:1.0:1787739065.068047:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 373 cancel update log [0x240000400:0x3:0x0].365: rc = 0 00000004:00080000:1.0:1787739065.068073:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730623dc00 with version 704 00000020:00080000:1.0:1787739065.068077:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 374 cancel update log [0x200000401:0x3:0x0].367: rc = 0 00000020:00080000:1.0:1787739065.068095:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 374 cancel update log [0x240000400:0x3:0x0].366: rc = 0 00000020:00080000:1.0:1787739065.068101:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 374 00010000:00010000:1.0:1787739065.068173:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ada4d lrc: 0/0,0 mode: --/PW res: [0x240000401:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ada46 expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.068190:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xcb:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:1.0:1787739065.068195:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada85 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.068204:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada85 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.068211:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada85 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.068235:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739065.068322:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba4450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+5:3652:x1874579665714048:12345-0@lo:103:ldlm_bl.0 Request processed in 21409us (21531us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739065.068332:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1551 00000100:00100000:1.0:1787739065.068451:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665714048:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739065.068477:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada85 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.068484:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada85 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739065.069024:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xcb:0x0]: returning size 4096 00000004:00200000:1.0:1787739065.069030:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739065.069078:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ada7e lrc: 0/0,0 mode: --/CR res: [0x240000401:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09518b expref: 11 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.069095:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada85 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09518b expref: 10 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.069104:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xcb:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:1.0:1787739065.069106:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739065.069109:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d000, rc 0) 00000100:00100000:1.0:1787739065.069166:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073063921c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17651:x1874579652496768:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 23934us (24417us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739065.069175:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23542 00010000:00010000:0.0:1787739065.069822:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad982 lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.069833:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad982 lrc: 2/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.069841:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad982 lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.069846:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739065.069850:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad982 lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.069857:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad982 lrc: 4/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.069869:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xe7d4651 (ffffa073068a9000) 00010000:00010000:0.0:1787739065.069872:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739065.069875:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad982 lrc: 2/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.069881:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1ad982 lrc: 0/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.069892:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad989 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x180:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.069898:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad989 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x180:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.069903:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad989 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x180:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.069907:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609be00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739065.069910:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad989 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x180:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.069931:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad989 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x180:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.069939:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x180:0x0].0x0 (ffffa073068a9200) 00010000:00010000:0.0:1787739065.069941:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739065.069944:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad989 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x180:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.069949:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609be00/0x7ee4453e4a1ad989 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x180:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.069963:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad990 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.069968:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad990 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.069974:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad990 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.069978:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739065.069981:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad990 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.069985:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad990 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.069991:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc8:0x0].0x0 (ffffa073068a9600) 00010000:00010000:0.0:1787739065.069993:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739065.069995:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad990 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.070012:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ad990 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xc8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.070030:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad9e4 lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.070036:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad9e4 lrc: 2/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.070042:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad9e4 lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.070047:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aae00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739065.070049:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad9e4 lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.070054:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad9e4 lrc: 4/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.070059:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xe7d4651 (ffffa073068a9000) 00010000:00010000:0.0:1787739065.070061:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739065.070063:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad9e4 lrc: 2/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.070068:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ad9e4 lrc: 0/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0xe7d4651 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739065.077968:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 703 rpc_version 703 00000020:00100000:0.0:1787739065.078034:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739065.078045:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073060d1a40 x1874579665714304/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739065.078071:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d1a40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665714304:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739065.078150:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739065.078196:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074332daa80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739065.078254:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665714304 00000100:00100000:1.0:1787739065.078261:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665714304 00000100:00100000:1.0:1787739065.078318:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3316 00000100:00100000:1.0:1787739065.078338:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:7467:x1874579665714304:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739065.078542:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665714304:12345-0@lo:1000:osp_up0-1.0 Request processed in 215us (391us total) trans 4294983421 rc 0/0 00000100:00100000:1.0:1787739065.078549:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3316 00000100:00100000:1.0:1787739065.078595:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d1a40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665714304:0@lo:1000:osp_up0-1.0 00000004:00080000:1.0:1787739065.078606:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 704 rpc_version 704 00000020:00100000:1.0:1787739065.078618:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739065.078622:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743562d2c0 x1874579665714560/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739065.078633:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562d2c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665714560:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739065.078671:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739065.078687:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730623d540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739065.078699:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665714560 00000100:00100000:1.0:1787739065.078702:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665714560 00000100:00100000:1.0:1787739065.078725:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3317 00000100:00100000:1.0:1787739065.078731:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665714560:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739065.078802:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665714560:12345-0@lo:1000:osp_up0-1.0 Request processed in 70us (130us total) trans 4294983422 rc 0/0 00000100:00100000:1.0:1787739065.078992:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3317 00000100:00100000:1.0:1787739065.079036:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562d2c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665714560:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739065.079081:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739065.079335:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652497024 00000100:00100000:0.0:1787739065.079340:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652497024 00000100:00100000:0.0:1787739065.079389:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23543 00000100:00100000:0.0:1787739065.079396:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17651:x1874579652497024:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:0.0:1787739065.079454:0:6524:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739065.079480:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ada69 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1523:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09517d expref: 17 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.079496:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1523:0x0].0x0 (ffffa07307064900) 00010000:00010000:0.0:1787739065.079723:0:6524:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739065.079748:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ada8c lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.079761:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ada8c lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.079771:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ada8c lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.079781:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ada93 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.079788:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ada93 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.079795:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ada93 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.079885:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ada9a lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1523:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.079894:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ada9a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1523:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.079901:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ada9a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1523:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.079918:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adaa1 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xcb:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.079927:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ada77 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095184 expref: 16 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.079935:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adaa1 lrc: 4/0,1 mode: --/EX res: [0x200000400:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.079960:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ada77 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095184 expref: 16 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.079975:0:6524:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ada77 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095184 expref: 16 pid: 6523 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739065.079986:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665714688:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739065.080139:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074332da480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.099776:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739065.099813:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652497152 00000100:00100000:0.0:1787739065.099822:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652497152 00000100:00100000:2.0:1787739065.102695:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665714688:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739065.102727:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adaa1 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.109533:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ada77 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095184 expref: 17 pid: 6523 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739065.109595:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1552 00000100:00100000:0.0:1787739065.109615:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d12c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652497152:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739065.109631:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739065.109639:0:6510:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ada77 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095184 expref: 17 pid: 6523 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.109668:0:6510:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ada77 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095184 expref: 17 pid: 6523 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.109690:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ada77 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095184 expref: 17 pid: 6523 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.109706:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xcb:0x0].0x0 (ffffa07307064600) 00010000:00010000:0.0:1787739065.109711:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adaa1 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.109720:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adaa1 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.109728:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adaa1 lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.109753:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739065.109809:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d12c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:2409:x1874579652497152:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 205us (10044us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.109817:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1552 00010000:00010000:0.0:1787739065.109854:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adaa1 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.109862:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adaa1 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.109903:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1adaa8 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.109912:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1adaa8 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.109921:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1adaa8 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739065.109939:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d3480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665714816:0@lo:101:mdt00_001.0 00000100:00100000:0.0:1787739065.109985:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739065.110023:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074332da840 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.110131:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665714816 00000100:00100000:0.0:1787739065.110138:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665714816 00000100:00100000:0.0:1787739065.110168:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3318 00000100:00100000:0.0:1787739065.110191:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6524:x1874579665714816:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739065.110206:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739065.110219:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adaaf lrc: 2/0,0 mode: --/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1adaa8 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.110238:0:6529:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada85 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09518b expref: 7 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.110248:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adaaf lrc: 4/0,0 mode: --/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1adaa8 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.111286:0:6529:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada85 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09518b expref: 7 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.111303:0:6529:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada85 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09518b expref: 7 pid: 13793 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739065.111315:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41e1c0 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665714944:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739065.111385:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074332da240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739065.117693:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41e1c0 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665714944:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739065.117730:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adaaf lrc: 3/0,0 mode: --/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adaa8 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.117743:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xcb:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:2.0:1787739065.117746:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739065.117750:0:6529:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adaaf lrc: 3/0,0 mode: --/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adaa8 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.117760:0:6529:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adaaf lrc: 3/0,0 mode: --/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adaa8 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.117767:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b200, rc 0) 00000100:00100000:2.0:1787739065.117823:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6524:x1874579665714816:12345-0@lo:101:mdt00_001.0 Request processed in 7633us (7837us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739065.117832:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3318 00000100:00100000:0.0:1787739065.117943:0:6524:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968410 gen 2 00000004:00080000:0.0:1787739065.117949:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968406 committed transno 4294968410 00000020:00080000:0.0:1787739065.117955:0:6524:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa0730610e780 refcount 2 committed 0 result 0 batchid 384 00000020:00080000:0.0:1787739065.117959:0:6524:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07404f86c00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730610ec00 00000020:00080000:0.0:1787739065.117963:0:6524:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].381 00000020:00080000:0.0:1787739065.117966:0:6524:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07404f86a00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304011600 00000020:00080000:0.0:1787739065.117968:0:6524:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].381 00000004:00080000:0.0:1787739065.117992:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968408 committed transno 4294968410 00000004:00080000:0.0:1787739065.118010:0:6524:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968410 committed transno 4294968410 00000100:00100000:0.0:1787739065.118024:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d3480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665714816:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739065.118041:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1adaa8 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adaaf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.118051:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1adaa8 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adaaf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739065.118093:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 384 committed batchid 382 00000020:00080000:2.0:1787739065.118142:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 382 00000100:00100000:0.0:1787739065.118751:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739065.124025:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652497280 00000100:00100000:2.0:1787739065.124039:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652497280 00010000:00010000:0.0:1787739065.128362:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada85 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09518b expref: 8 pid: 13793 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739065.128406:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1553 00000100:00100000:0.0:1787739065.128438:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2408:x1874579652497280:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739065.128453:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739065.128460:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada85 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09518b expref: 8 pid: 13793 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.128474:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada85 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09518b expref: 8 pid: 13793 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.128497:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ada85 lrc: 0/0,0 mode: --/PR res: [0x240000401:0xcb:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09518b expref: 8 pid: 13793 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.128511:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xcb:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:0.0:1787739065.128518:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adaaf lrc: 2/0,0 mode: --/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adaa8 expref: 5 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.128530:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adaaf lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1adaa8 expref: 5 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.128539:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adaaf lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1adaa8 expref: 5 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.128571:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adaaf lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adaa8 expref: 5 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739065.128590:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41e580 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665715072:0@lo:105: 00000100:00100000:0.0:1787739065.128661:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739065.128700:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074332da840 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.128721:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665715072 00000100:00100000:0.0:1787739065.128726:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665715072 00000100:00100000:0.0:1787739065.128756:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1187 00000100:00100000:0.0:1787739065.128763:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665715072:12345-0@lo:105: 00010000:00010000:0.0:1787739065.128774:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1adaa8 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adaaf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.128784:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1adaa8 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adaaf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.128793:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1adaa8 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adaaf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.128799:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1adaa8 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1adaaf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.128805:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1adaa8 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1adaaf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.128813:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1adaa8 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1adaaf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.128831:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609a600) 00000100:00100000:0.0:1787739065.128870:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665715072:12345-0@lo:105: Request processed in 108us (209us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.128878:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1187 00010000:00010000:0.0:1787739065.128906:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1adaa8 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adaaf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.128914:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1adaa8 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adaaf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739065.129574:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41f480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665715200:0@lo:403:mdt00_001.0 00000100:00100000:0.0:1787739065.129617:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739065.129642:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074332daf00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.129661:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665715200 00000100:00100000:0.0:1787739065.129666:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665715200 00000100:00100000:0.0:1787739065.129694:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3319 00000100:00100000:0.0:1787739065.129702:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6524:x1874579665715200:12345-0@lo:403:mdt00_001.0 00000100:00100000:2.0:1787739065.131508:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41e580 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665715072:0@lo:105: 00010000:00010000:2.0:1787739065.131531:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739065.131583:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652497280:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 3167us (12831us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739065.131594:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1553 00000100:00100000:0.0:1787739065.139358:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6524:x1874579665715200:12345-0@lo:403:mdt00_001.0 Request processed in 9650us (9733us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.139373:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3319 00000100:00100000:2.0:1787739065.139777:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41f480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665715200:0@lo:403:mdt00_001.0 00000040:00080000:2.0:1787739065.144099:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].383, 664 off585000 00000040:00080000:2.0:1787739065.144286:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].383, 744 off635328 00000004:00080000:2.0:1787739065.144294:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306887900 with version 729 00000004:00080000:3.0:1787739065.148138:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 729 rpc_version 729 00000020:00100000:3.0:1787739065.148179:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739065.148190:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306390f00 x1874579665715456/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1704/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739065.148211:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306390f00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665715456:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739065.148282:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739065.148329:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e180 going to sleep for 11 seconds 00000100:00100000:1.0:1787739065.151122:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665715456 00000100:00100000:1.0:1787739065.151137:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665715456 00000100:00100000:1.0:1787739065.151275:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3320 00000100:00100000:1.0:1787739065.151285:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306392d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665715456:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739065.151870:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306392d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665715456:12345-0@lo:1000:osp_up1-0.0 Request processed in 586us (3586us total) trans 4294968411 rc 0/0 00000100:00100000:1.0:1787739065.151879:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3320 00000100:00100000:3.0:1787739065.156885:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306390f00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665715456:0@lo:1000:osp_up1-0.0 00000004:00200000:2.0:1787739065.158064:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1523:0x0]: returning size 4096 00010000:00010000:2.0:1787739065.158092:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adaa1 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158103:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adaa1 lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158111:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adaa1 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158116:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739065.158120:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adaa1 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158127:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adaa1 lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158143:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xcb:0x0].0x0 (ffffa07307064600) 00010000:00010000:2.0:1787739065.158146:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739065.158150:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adaa1 lrc: 2/0,0 mode: --/EX res: [0x200000400:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158161:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adaa1 lrc: 0/0,0 mode: --/EX res: [0x200000400:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158345:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1adaa8 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adaaf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158355:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1adaa8 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1adaaf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158690:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1adaa8 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1adaaf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158706:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739065.158715:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xcb:0x0] 00010000:00010000:2.0:1787739065.158723:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ada9a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1523:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158730:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ada9a lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1523:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158736:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ada9a lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1523:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158742:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739065.158745:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ada9a lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1523:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158751:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ada9a lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1523:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158765:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1523:0x0].0x0 (ffffa07307064900) 00010000:00010000:2.0:1787739065.158767:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739065.158770:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ada9a lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1523:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158776:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1ada9a lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1523:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158792:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ada8c lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158798:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ada8c lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158803:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ada8c lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158809:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ada8c lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158814:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739065.158816:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ada8c lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158822:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ada8c lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158830:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07307064e00) 00010000:00010000:2.0:1787739065.158832:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739065.158835:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ada8c lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158846:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1ada8c lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739065.158860:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c314c693 reply state 00000000801d17b4 transno 4294983423 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ada93 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158872:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ada93 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.158879:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x7ed215cf (ffffa07433235b00) 00010000:00010000:2.0:1787739065.158882:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739065.158885:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787739065.158969:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17651:x1874579652497024:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 79569us (79885us total) trans 4294983423 rc 0/0 00000100:00100000:2.0:1787739065.158979:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23543 00010000:00010000:0.0:1787739065.159104:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739065.159111:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1adaa8 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1adaaf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.159119:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739065.159124:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1adaa8 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1adaaf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739065.159132:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1adaa8 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1adaaf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.159162:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1adaa8 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adaaf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.159168:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739065.159201:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1adaa8 lrc: 1/0,0 mode: --/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adaaf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.159209:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a600/0x7ee4453e4a1adaa8 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adaaf expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.159223:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739065.159224:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739065.159226:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739065.159462:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787739065.159478:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665715584:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739065.159521:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739065.159724:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665715584 00000100:00100000:0.0:1787739065.159730:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665715584 00000100:00100000:0.0:1787739065.159775:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1353 00000100:00100000:0.0:1787739065.159784:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562f840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+7:3652:x1874579665715584:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739065.159797:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739065.159814:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adaaf lrc: 0/0,0 mode: --/EX res: [0x240000401:0xcb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1adaa8 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.159825:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xcb:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:0.0:1787739065.159832:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739065.159862:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562f840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+6:3652:x1874579665715584:12345-0@lo:103:ldlm_bl.0 Request processed in 80us (341us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.159869:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1353 00000100:00100000:1.0:1787739065.159982:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41d680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665715584:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739065.203345:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739065.203545:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652497408 00000100:00100000:1.0:1787739065.203554:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652497408 00000100:00100000:1.0:1787739065.205941:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23544 00000100:00100000:1.0:1787739065.205959:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17652:x1874579652497408:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739065.205980:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739065.206011:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1adab6 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095192 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.206074:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adabd lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.206088:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adabd lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.206099:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adabd lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.206114:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1adac4 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x5a271237 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.206121:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1adac4 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.206127:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1adac4 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.206173:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adabd lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.206181:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adabd lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.206187:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adabd lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.206194:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adabd lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.206199:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739065.206202:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adabd lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.206208:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adabd lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.206220:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064a00) 00010000:00010000:1.0:1787739065.206222:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739065.206227:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adabd lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.206233:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adabd lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.206244:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1adac4 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.206249:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1adac4 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.206255:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1adac4 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.206259:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739065.206262:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1adac4 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.206267:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1adac4 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.206273:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x5a271237 (ffffa07305eae100) 00010000:00010000:1.0:1787739065.206275:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739065.206278:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1adac4 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.206283:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1adac4 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.206312:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1adab6 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095192 expref: 8 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.206322:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064a00) 00010000:00010000:1.0:1787739065.206325:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739065.206329:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1adab6 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095192 expref: 8 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.206338:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1dc00, rc 0) 00000100:00100000:1.0:1787739065.206402:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17652:x1874579652497408:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 448us (3065us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739065.206429:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23544 00000100:00100000:1.0:1787739065.218505:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739065.218618:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652497536 00000100:00100000:2.0:1787739065.218628:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652497536 00000100:00100000:2.0:1787739065.218696:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23545 00000100:00100000:2.0:1787739065.218706:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17652:x1874579652497536:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739065.218730:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739065.218751:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1adacb lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095199 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.218892:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1adad2 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.218908:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1adad2 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.218923:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1adad2 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739065.218969:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:2.0:1787739065.218974:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739065.219019:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1adacb lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095199 expref: 19 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.219038:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1adad2 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095199 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.219047:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:2.0:1787739065.219050:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739065.219053:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c200, rc 0) 00000100:00100000:2.0:1787739065.219152:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17652:x1874579652497536:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 447us (658us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739065.219160:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23545 00000100:00100000:3.0:1787739065.255855:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739065.255930:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652497792 00000100:00100000:0.0:1787739065.255939:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652497792 00000100:00100000:0.0:1787739065.255994:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23546 00000100:00100000:0.0:1787739065.256013:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073063930c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17652:x1874579652497792:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:0.0:1787739065.256068:0:6524:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739065.256093:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1ada23 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095176 expref: 7 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.256109:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064a00) 00010000:00010000:0.0:1787739065.256119:0:6524:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739065.256178:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adad9 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.256191:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adad9 lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.256201:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adad9 lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.256213:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1adae0 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x5a271237 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.256220:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1adae0 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.256225:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1adae0 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00080000:02000000:0.0:1787739065.256475:0:6524:0:(osd_handler.c:2218:osd_trans_start()) *** cfs_fail_loc=19a, val=0*** 00000004:00080000:0.0:1787739065.256491:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa074332daf00 with version 0 00000020:00080000:0.0:1787739065.256496:0:6524:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa074332da840 refcount 3 committed 0 result -5 batchid 377 00000020:00080000:0.0:1787739065.256500:0:6524:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2700 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result -5 sub_th ffffa074332daf00 00000020:00080000:0.0:1787739065.256503:0:6524:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2400 obd lustre-MDT0001-osd committed 1 started 0 stopped 1 result 0 sub_th ffffa07411bda200 00010000:00010000:0.0:1787739065.256526:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adad9 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.256533:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adad9 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.256539:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adad9 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.256545:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adad9 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.256550:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739065.256554:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adad9 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.256561:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adad9 lrc: 5/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.256569:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064000) 00010000:00010000:0.0:1787739065.256571:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739065.256575:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adad9 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.256581:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adad9 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.256593:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1adae0 lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.256602:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1adae0 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.256607:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1adae0 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.256611:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739065.256614:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1adae0 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.256619:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1adae0 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.256625:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x5a271237 (ffffa07307064b00) 00010000:00010000:0.0:1787739065.256627:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739065.256629:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1adae0 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.256634:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1adae0 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739065.256699:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073063930c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17652:x1874579652497792:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 695us (853us total) trans 0 rc -5/-5 00000100:00100000:0.0:1787739065.256706:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23546 00000020:00080000:3.0:1787739065.257129:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 374 00000100:00100000:2.0:1787739065.285513:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739065.293746:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652498048 00000100:00100000:3.0:1787739065.293759:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652498048 00000100:00100000:3.0:1787739065.293858:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23547 00000100:00100000:3.0:1787739065.293869:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17654:x1874579652498048:12345-192.168.203.15@tcp:36:mkdir.0 00010000:00010000:3.0:1787739065.293985:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1adae7 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.294010:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1adae7 lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.294021:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1adae7 lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.294034:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1adaee lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x5a271237 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.294041:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1adaee lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.294046:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1adaee lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.294583:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1adaf5 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x183:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.294592:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1adaf5 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x183:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.294599:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1adaf5 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x183:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739065.294611:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000028c809b7 reply state 000000005b3ed1ed transno 4294968412 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1adaf5 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x183:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.294626:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1adaf5 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x183:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.294633:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x183:0x0].0x0 (ffffa0730608b100) 00010000:00010000:3.0:1787739065.294635:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739065.294639:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:3.0:1787739065.294643:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x183:0x0]: returning size 4096 00010000:00010000:3.0:1787739065.294712:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1adae7 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.294719:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1adae7 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.294725:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1adae7 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.294732:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1adae7 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.294737:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739065.294741:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1adae7 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.294747:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1adae7 lrc: 5/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.294757:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b900) 00010000:00010000:3.0:1787739065.294759:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739065.294762:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1adae7 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.294772:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1adae7 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739065.294788:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000028c809b7 reply state 000000005b3ed1ed transno 4294968412 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1adaee lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.294796:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1adaee lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.294801:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x5a271237 (ffffa0730608b600) 00010000:00010000:3.0:1787739065.294804:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739065.294805:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000100:00100000:3.0:1787739065.295135:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17654:x1874579652498048:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 1267us (9627us total) trans 4294968412 rc 0/0 00000100:00100000:3.0:1787739065.295144:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23547 00000100:00100000:0.0:1787739065.343535:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739065.343572:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652498176 00000100:00100000:0.0:1787739065.343581:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652498176 00000100:00100000:0.0:1787739065.343632:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23548 00000100:00100000:0.0:1787739065.343642:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17657:x1874579652498176:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739065.343667:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739065.343689:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1adafc lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0951a0 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.343754:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adb03 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.343767:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adb03 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.343778:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adb03 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.343789:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1adb0a lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x5a271237 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.343795:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1adb0a lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.343801:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1adb0a lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.343872:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adb11 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x183:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.343880:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adb11 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x183:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.343887:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adb11 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x183:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739065.343925:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x183:0x0]: returning size 4096 00000004:00200000:0.0:1787739065.343929:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787739065.343946:0:6525:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adb11 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x183:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.343958:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adb03 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.343964:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adb03 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.343970:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adb03 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.343975:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adb03 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.343981:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739065.343985:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adb03 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.343991:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adb03 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.344015:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07307064b00) 00010000:00010000:0.0:1787739065.344018:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739065.344021:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adb03 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.344028:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adb03 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.344041:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1adb0a lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.344050:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1adb0a lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.344055:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1adb0a lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.344060:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739065.344063:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1adb0a lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.344069:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1adb0a lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.344077:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x5a271237 (ffffa0730608b600) 00010000:00010000:0.0:1787739065.344079:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739065.344082:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1adb0a lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.344088:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1adb0a lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.344112:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1adafc lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951a0 expref: 10 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.344130:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adb11 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x183:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951a0 expref: 9 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.344137:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x183:0x0].0x0 (ffffa0730608b100) 00010000:00010000:0.0:1787739065.344139:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787739065.344142:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a200, rc 0) 00000100:00100000:0.0:1787739065.344214:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17657:x1874579652498176:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 574us (688us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.344221:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23548 00000100:00100000:2.0:1787739065.370719:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739065.370772:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652498432 00000100:00100000:1.0:1787739065.370782:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652498432 00000100:00100000:1.0:1787739065.370839:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23549 00000100:00100000:1.0:1787739065.370849:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17657:x1874579652498432:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:1.0:1787739065.370902:0:7944:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739065.370927:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adb11 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x183:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951a0 expref: 9 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.370944:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x183:0x0].0x0 (ffffa0730608b100) 00010000:00010000:1.0:1787739065.370948:0:7944:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739065.370974:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adb18 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.370986:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adb18 lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.370997:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adb18 lrc: 3/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.371019:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1adb1f lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x5a271237 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.371027:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1adb1f lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.371033:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1adb1f lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.371112:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1adb26 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x183:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.371120:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1adb26 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x183:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.371126:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1adb26 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x183:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739065.371390:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x183:0x0]: returning size 4096 00010000:00010000:1.0:1787739065.371404:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1adb26 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x183:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.371484:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1adb26 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x183:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.371492:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1adb26 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x183:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.371499:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739065.371503:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1adb26 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x183:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.371510:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1adb26 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x183:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.371520:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x183:0x0].0x0 (ffffa0730608b100) 00010000:00010000:1.0:1787739065.371523:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739065.371526:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1adb26 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x183:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.371537:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1adb26 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x183:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.371554:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adb18 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.371560:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adb18 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.371566:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adb18 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.371572:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adb18 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.371577:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739065.371580:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adb18 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.371586:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adb18 lrc: 5/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.371593:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305eae100) 00010000:00010000:1.0:1787739065.371595:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739065.371598:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adb18 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.371604:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adb18 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739065.371618:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d959d449 reply state 00000000190bd8d6 transno 4294968413 ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1adb1f lrc: 3/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.371629:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1adb1f lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.371636:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x5a271237 (ffffa0730608b600) 00010000:00010000:1.0:1787739065.371638:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739065.371641:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787739065.371819:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17657:x1874579652498432:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 970us (1107us total) trans 4294968413 rc 0/0 00000100:00100000:1.0:1787739065.371827:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23549 00000100:00100000:2.0:1787739065.406452:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739065.406569:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652498560 00000100:00100000:3.0:1787739065.406579:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652498560 00000100:00100000:0.0:1787739065.406704:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23550 00000100:00100000:0.0:1787739065.406718:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17664:x1874579652498560:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739065.406737:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739065.406756:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1adb2d lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0951a7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.406815:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1adb34 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.406828:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1adb34 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.406838:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1adb34 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.406852:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1adb3b lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x60da72aa bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.406859:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1adb3b lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.406865:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1adb3b lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.406899:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1adb34 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.406911:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1adb34 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.406916:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1adb34 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.406922:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1adb34 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.406927:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739065.406931:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1adb34 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.406938:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1adb34 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.406949:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:0.0:1787739065.406951:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739065.406956:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1adb34 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.406962:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1adb34 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.406974:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1adb3b lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.406981:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1adb3b lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.406987:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1adb3b lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.406993:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739065.406995:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1adb3b lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.407009:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1adb3b lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.407016:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x60da72aa (ffffa07307064b00) 00010000:00010000:0.0:1787739065.407019:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739065.407021:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1adb3b lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.407028:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1adb3b lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.407056:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1adb2d lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951a7 expref: 19 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.407066:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:0.0:1787739065.407069:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739065.407072:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1adb2d lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951a7 expref: 19 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.407081:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b600, rc 0) 00000100:00100000:0.0:1787739065.407141:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17664:x1874579652498560:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 427us (735us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739065.407149:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23550 00000100:00100000:0.0:1787739065.414389:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739065.419695:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652498688 00000100:00100000:2.0:1787739065.419709:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652498688 00000100:00100000:2.0:1787739065.419779:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23551 00000100:00100000:2.0:1787739065.419790:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17664:x1874579652498688:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739065.419813:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739065.419835:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1adb42 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0951ae expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.419900:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adb49 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.419914:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adb49 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.419925:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adb49 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739065.419972:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x5:0x0]: returning size 4096 00000004:00200000:2.0:1787739065.419977:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739065.420022:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633cc00/0x7ee4453e4a1adb42 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951ae expref: 12 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.420040:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adb49 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951ae expref: 11 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.420048:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:2.0:1787739065.420051:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739065.420054:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633ce00, rc 0) 00000100:00100000:2.0:1787739065.420126:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17664:x1874579652498688:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 338us (5745us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739065.420134:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23551 00000100:00100000:2.0:1787739065.438980:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739065.439116:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652498944 00000100:00100000:1.0:1787739065.439126:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652498944 00000100:00100000:1.0:1787739065.439192:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23552 00000100:00100000:1.0:1787739065.439202:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17664:x1874579652498944:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:1.0:1787739065.439261:0:7944:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739065.439287:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1adad2 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095199 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.439307:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:1.0:1787739065.439315:0:7944:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739065.439373:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adb50 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.439385:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adb50 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.439394:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adb50 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.439435:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1adb57 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x60da72aa bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.439442:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1adb57 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.439448:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1adb57 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1787739065.440268:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].384, 2168 off587168 00000040:00080000:1.0:1787739065.440340:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].384, 2368 off637696 00000004:00080000:1.0:1787739065.440346:0:7944:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0743c0efb40 with version 730 00000004:00080000:3.0:1787739065.440479:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 730 rpc_version 730 00000020:00100000:3.0:1787739065.440511:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739065.440520:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073063903c0 x1874579665715840/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 3976/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739065.440541:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073063903c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665715840:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739065.440603:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739065.440672:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739065.440881:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665715840 00000100:00100000:1.0:1787739065.440888:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665715840 00000100:00100000:1.0:1787739065.440969:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3321 00000100:00100000:1.0:1787739065.440978:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306392580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665715840:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739065.441362:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306392580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665715840:12345-0@lo:1000:osp_up1-0.0 Request processed in 384us (758us total) trans 4294968414 rc 0/0 00000100:00100000:1.0:1787739065.441370:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3321 00000100:00100000:3.0:1787739065.441478:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073063903c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665715840:0@lo:1000:osp_up1-0.0 00010000:00010000:1.0:1787739065.446562:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1adb5e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1525:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.446582:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1adb5e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1525:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.446594:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1adb5e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1525:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.446613:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1adb65 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.446621:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1adb65 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.446626:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1adb65 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.446885:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1adb6c lrc: 3/0,1 mode: --/PW res: [0x240000401:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.446894:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1adb6c lrc: 3/0,1 mode: --/PW res: [0x240000401:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.446903:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1adb6c lrc: 3/0,1 mode: --/PW res: [0x240000401:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739065.446925:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562da40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665715968:0@lo:101:mdt00_003.0 00000100:00100000:1.0:1787739065.446986:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739065.447048:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074154fce40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739065.447156:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665715968 00000100:00100000:3.0:1787739065.447163:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665715968 00000100:00100000:3.0:1787739065.447214:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3322 00000100:00100000:3.0:1787739065.447223:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:7944:x1874579665715968:12345-0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739065.447243:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739065.447258:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1adb73 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1adb6c expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.447279:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1adb73 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1adb6c expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.447291:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1adb73 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adb6c expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.447298:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xcc:0x0].0x0 (ffffa0730608b900) 00010000:00010000:3.0:1787739065.447301:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739065.447304:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab600, rc 0) 00000100:00100000:3.0:1787739065.447347:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:7944:x1874579665715968:12345-0@lo:101:mdt00_003.0 Request processed in 126us (362us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739065.447354:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3322 00000100:00100000:1.0:1787739065.447524:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562da40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665715968:0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739065.447546:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1adb6c lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adb73 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.447556:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1adb6c lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adb73 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739065.447575:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008b13ff4f reply state 00000000b7cdab99 transno 4294983424 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1adb65 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.447587:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1adb65 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.447593:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xcc:0x0].0x0 (ffffa07305eaed00) 00010000:00010000:1.0:1787739065.447596:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739065.447598:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787739065.447604:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1adb6c lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adb73 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.447611:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1adb6c lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adb73 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739065.447622:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xcc:0x0] 00000004:00010000:1.0:1787739065.447638:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008b13ff4f reply state 00000000b7cdab99 transno 4294983424 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1adb5e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1525:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.447653:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1adb5e lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1525:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.447659:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1525:0x0].0x0 (ffffa07305eae500) 00010000:00010000:1.0:1787739065.447662:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739065.447664:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:1.0:1787739065.447668:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1525:0x0]: returning size 4096 00010000:00010000:1.0:1787739065.447677:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adb50 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.447684:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adb50 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.447690:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adb50 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.447697:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adb50 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.447707:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739065.447711:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adb50 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.447718:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adb50 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.447727:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae100) 00010000:00010000:1.0:1787739065.447729:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739065.447732:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adb50 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.447738:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adb50 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739065.447752:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008b13ff4f reply state 00000000b7cdab99 transno 4294983424 ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1adb57 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.447760:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1adb57 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.447766:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x60da72aa (ffffa07305eaec00) 00010000:00010000:1.0:1787739065.447768:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739065.447770:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 2 00000100:00100000:1.0:1787739065.447878:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+19:17664:x1874579652498944:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 8677us (8907us total) trans 4294983424 rc 0/0 00000100:00100000:1.0:1787739065.447886:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23552 00000100:00100000:3.0:1787739065.483851:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739065.483946:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652499200 00000100:00100000:1.0:1787739065.483957:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652499200 00000100:00100000:1.0:1787739065.484029:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23553 00000100:00100000:1.0:1787739065.484040:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306391a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+19:17665:x1874579652499200:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:1.0:1787739065.484235:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306391a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+19:17665:x1874579652499200:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 195us (392us total) trans 0 rc -17/-17 00000100:00100000:1.0:1787739065.484244:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23553 00000100:00100000:3.0:1787739065.527380:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739065.527491:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652499328 00000100:00100000:1.0:1787739065.527500:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652499328 00000100:00100000:1.0:1787739065.527555:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23554 00000100:00100000:1.0:1787739065.527564:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306393840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+19:17666:x1874579652499328:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739065.527587:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739065.527610:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adb7a lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0951b5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.527682:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1adb81 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.527695:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1adb81 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.527706:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1adb81 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.527718:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1adb88 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x60da72aa bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.527725:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1adb88 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.527730:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1adb88 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.527799:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1adb8f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1525:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.527809:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1adb8f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1525:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.527816:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1adb8f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1525:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739065.527900:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1525:0x0]: returning size 4096 00000004:00200000:1.0:1787739065.527905:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:1.0:1787739065.527924:0:13793:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1adb8f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1525:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.527946:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1adb81 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.527952:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1adb81 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.527958:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1adb81 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.527964:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1adb81 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.527970:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739065.527974:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1adb81 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.527981:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1adb81 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.528197:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae100) 00010000:00010000:1.0:1787739065.528201:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739065.528206:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1adb81 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.528213:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1adb81 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.528225:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1adb88 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.528231:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1adb88 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.528237:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1adb88 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.528242:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739065.528245:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1adb88 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.528250:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1adb88 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.528257:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x60da72aa (ffffa07305eaec00) 00010000:00010000:1.0:1787739065.528260:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739065.528263:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1adb88 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.528269:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1adb88 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.528299:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1adb7a lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951b5 expref: 21 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.528314:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1adb8f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1525:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951b5 expref: 20 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.528321:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1525:0x0].0x0 (ffffa07305eae500) 00010000:00010000:1.0:1787739065.528323:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787739065.528325:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c200, rc 0) 00000100:00100000:1.0:1787739065.528386:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306393840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+20:17666:x1874579652499328:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 823us (1015us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739065.528394:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23554 00000100:00100000:3.0:1787739065.537866:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739065.537946:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652499456 00000100:00100000:1.0:1787739065.537955:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652499456 00000100:00100000:1.0:1787739065.538028:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23555 00000100:00100000:1.0:1787739065.538038:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f75680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+20:17666:x1874579652499456:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739065.538059:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739065.538080:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1adb96 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0951bc expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.538137:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1adb9d lrc: 3/1,0 mode: --/PR res: [0x200000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.538151:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1adb9d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.538162:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1adb9d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739065.538211:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xcc:0x0]: returning size 4096 00000004:00200000:1.0:1787739065.538216:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739065.538250:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1adb96 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951bc expref: 22 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.538268:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1adb9d lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951bc expref: 21 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.538276:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xcc:0x0].0x0 (ffffa07305eaed00) 00010000:00010000:1.0:1787739065.538278:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739065.538281:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1ca00, rc 0) 00000100:00100000:1.0:1787739065.538346:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f75680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:17666:x1874579652499456:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 299us (477us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739065.538354:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23555 00000100:00100000:0.0:1787739065.553072:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739065.553259:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652499584 00000100:00100000:1.0:1787739065.553269:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652499584 00000100:00100000:1.0:1787739065.553330:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23556 00000100:00100000:1.0:1787739065.553350:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17666:x1874579652499584:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739065.553372:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739065.553392:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1adba4 lrc: 2/0,0 mode: --/CR res: [0x240000401:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0951c3 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.553468:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adbab lrc: 3/1,0 mode: --/PR res: [0x240000401:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.553483:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1adb73 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adb6c expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.553499:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adbab lrc: 4/1,0 mode: --/PR res: [0x240000401:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.553530:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1adb73 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1adb6c expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739065.553546:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562da40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665716096:0@lo:104: 00000100:00100000:1.0:1787739065.553589:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739065.553636:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074154fce40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739065.554572:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665716096 00000100:00100000:3.0:1787739065.554580:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665716096 00000100:00100000:3.0:1787739065.554627:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1188 00000100:00100000:3.0:1787739065.554635:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562cf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6524:x1874579665716096:12345-0@lo:104: 00010000:00010000:3.0:1787739065.554656:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1adb6c lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1adb73 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.554735:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739065.554742:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562cf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6524:x1874579665716096:12345-0@lo:104: Request processed in 109us (1152us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739065.554749:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1188 00010000:00010000:3.0:1787739065.554781:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739065.554787:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1adb6c lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1adb73 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.554796:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739065.554801:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1adb6c lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xcc:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1adb73 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739065.554810:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1adb6c lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xcc:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1adb73 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.554836:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1adb6c lrc: 2/0,0 mode: --/PW res: [0x240000401:0xcc:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1adb73 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.554842:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739065.554872:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1adb6c lrc: 1/0,0 mode: --/PW res: [0x240000401:0xcc:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1adb73 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.554881:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07441a1c000/0x7ee4453e4a1adb6c lrc: 0/0,0 mode: --/PW res: [0x240000401:0xcc:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1adb73 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.554901:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739065.554903:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739065.554905:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739065.554917:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665716224:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739065.555219:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739065.555308:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665716224 00000100:00100000:0.0:1787739065.555312:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665716224 00010000:00010000:0.0:1787739065.555345:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1adb73 lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1adb6c expref: 8 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739065.555377:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1554 00000100:00100000:0.0:1787739065.555384:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+8:3650:x1874579665716224:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739065.555397:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787739065.557800:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562da40 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665716096:0@lo:104: 00010000:00010000:1.0:1787739065.557833:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adbab lrc: 3/1,0 mode: --/PR res: [0x240000401:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787739065.573549:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968414 is committed 00000001:00080000:0.0:1787739065.573575:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968413 is committed 00000020:00080000:0.0:1787739065.573587:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 375 batchid updated 00010000:00010000:0.0:1787739065.573616:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1adaee lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.573627:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1adaee lrc: 2/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.573636:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1adaee lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.573642:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aaa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739065.573647:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1adaee lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.573655:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1adaee lrc: 4/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.573671:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x5a271237 (ffffa0730608b600) 00010000:00010000:0.0:1787739065.573674:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739065.573679:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1adaee lrc: 2/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.573686:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1adaee lrc: 0/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.573699:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1adaf5 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x183:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.573705:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1adaf5 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x183:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.573711:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1adaf5 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x183:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.573716:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739065.573719:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1adaf5 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x183:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.573725:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1adaf5 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x183:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.573731:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x183:0x0].0x0 (ffffa0730608b100) 00010000:00010000:0.0:1787739065.573734:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739065.573736:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1adaf5 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x183:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.573742:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1adaf5 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x183:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.573761:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1adb1f lrc: 3/0,1 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.573771:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1adb1f lrc: 2/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.573778:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1adb1f lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.573782:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ce00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739065.573785:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1adb1f lrc: 3/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.573791:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1adb1f lrc: 4/0,0 mode: TXN/TXN res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.573797:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x5a271237 (ffffa0730608b600) 00010000:00010000:0.0:1787739065.573799:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739065.573802:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1adb1f lrc: 2/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.573807:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1adb1f lrc: 0/0,0 mode: --/TXN res: [0x240000401:0x5:0x0].0x5a271237 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:0.0:1787739065.573821:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 375 00000004:00080000:0.0:1787739065.573875:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa074332daf00 with version 705 00000020:00080000:0.0:1787739065.573884:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 375 cancel update log [0x200000401:0x3:0x0].368: rc = 0 00000020:00080000:0.0:1787739065.573946:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 375 cancel update log [0x240000400:0x3:0x0].368: rc = 0 00000020:00080000:0.0:1787739065.573952:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 375 00000004:00080000:0.0:1787739065.573958:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 705 rpc_version 705 00000020:00100000:0.0:1787739065.573986:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739065.573995:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743c41fc00 x1874579665716480/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739065.574031:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41fc00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665716480:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739065.574092:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739065.574118:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074332da480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.574136:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665716480 00000100:00100000:0.0:1787739065.574141:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665716480 00000100:00100000:0.0:1787739065.574185:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3323 00000100:00100000:0.0:1787739065.574193:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665716480:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739065.574307:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665716480:12345-0@lo:1000:osp_up0-1.0 Request processed in 114us (214us total) trans 4294983425 rc 0/0 00000100:00100000:0.0:1787739065.574315:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3323 00000100:00100000:0.0:1787739065.574356:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41fc00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665716480:0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1787739065.574404:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1adb73 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1adb6c expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.574437:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xcc:0x0].0x0 (ffffa0730608b900) 00010000:00010000:0.0:1787739065.574442:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adbab lrc: 3/1,0 mode: --/PR res: [0x240000401:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.574451:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adbab lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.574458:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adbab lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.574476:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739065.574504:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+5:3650:x1874579665716224:12345-0@lo:103:ldlm_bl.0 Request processed in 19122us (19286us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.574515:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1554 00010000:00010000:0.0:1787739065.574535:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adbab lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.574543:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adbab lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739065.574588:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xcc:0x0]: returning size 4096 00000004:00200000:0.0:1787739065.574593:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739065.574630:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1adba4 lrc: 0/0,0 mode: --/CR res: [0x240000401:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951c3 expref: 9 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.574644:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adbab lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951c3 expref: 8 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.574651:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xcc:0x0].0x0 (ffffa0730608b900) 00010000:00010000:0.0:1787739065.574654:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739065.574656:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fda00, rc 0) 00000100:00100000:0.0:1787739065.574692:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17666:x1874579652499584:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 21346us (21628us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.574699:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23556 00000100:00100000:3.0:1787739065.575712:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665716224:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739065.584778:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739065.584891:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652499840 00000100:00100000:2.0:1787739065.584900:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652499840 00000100:00100000:2.0:1787739065.584957:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23557 00000100:00100000:2.0:1787739065.584967:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:17666:x1874579652499840:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:2.0:1787739065.585166:0:13793:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739065.585194:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1adb8f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1525:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951b5 expref: 21 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.585211:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1525:0x0].0x0 (ffffa07305eae500) 00010000:00010000:2.0:1787739065.585216:0:13793:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739065.585241:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1adbb2 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.585253:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1adbb2 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.585263:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1adbb2 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.585273:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1adbb9 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x60da72aa bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.585280:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1adbb9 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.585286:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1adbb9 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.585359:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1adbc0 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1525:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.585367:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1adbc0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1525:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.585373:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1adbc0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1525:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.585390:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adbc7 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.585399:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1adb9d lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951bc expref: 20 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.585409:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adbc7 lrc: 4/0,1 mode: --/EX res: [0x200000400:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.585473:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1adb9d lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951bc expref: 20 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.585487:0:13793:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1adb9d lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951bc expref: 20 pid: 7944 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739065.585499:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f2d00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665716608:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739065.585574:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739065.588502:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f2d00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665716608:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739065.588529:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adbc7 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739065.590128:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739065.593484:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652499968 00000100:00100000:2.0:1787739065.593498:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652499968 00010000:00010000:2.0:1787739065.593544:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1adb9d lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951bc expref: 21 pid: 7944 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739065.593577:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1555 00000100:00100000:2.0:1787739065.593589:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:2409:x1874579652499968:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739065.593604:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739065.593611:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1adb9d lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951bc expref: 21 pid: 7944 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.593624:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1adb9d lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951bc expref: 21 pid: 7944 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.593646:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1adb9d lrc: 0/0,0 mode: --/PR res: [0x200000400:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951bc expref: 21 pid: 7944 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.593661:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xcc:0x0].0x0 (ffffa07305eaed00) 00010000:00010000:2.0:1787739065.593666:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adbc7 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.593676:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adbc7 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.593684:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adbc7 lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.593717:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739065.593779:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+20:2409:x1874579652499968:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 192us (3658us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739065.593787:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1555 00010000:00010000:1.0:1787739065.594116:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adbc7 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.594125:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adbc7 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.594171:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd000/0x7ee4453e4a1adbce lrc: 3/0,1 mode: --/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.594178:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd000/0x7ee4453e4a1adbce lrc: 3/0,1 mode: --/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.594187:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd000/0x7ee4453e4a1adbce lrc: 3/0,1 mode: --/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739065.594209:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562e940 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665716736:0@lo:101:mdt00_004.0 00000100:00100000:1.0:1787739065.594260:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739065.594296:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074154fce40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739065.594309:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665716736 00000100:00100000:2.0:1787739065.594314:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665716736 00000100:00100000:2.0:1787739065.594352:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3324 00000100:00100000:2.0:1787739065.594359:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304245a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:13793:x1874579665716736:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739065.594376:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739065.594390:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1adbd5 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1adbce expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.594445:0:6528:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adbab lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951c3 expref: 7 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.594455:0:6528:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1adbd5 lrc: 4/0,0 mode: --/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1adbce expref: 6 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.594477:0:6528:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adbab lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951c3 expref: 7 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.594489:0:6528:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adbab lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951c3 expref: 7 pid: 6524 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739065.594499:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f2d00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001_UUID:6528:1874579665716864:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739065.594539:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739065.604352:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739065.604455:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652500096 00000100:00100000:0.0:1787739065.604462:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652500096 00000100:00100000:1.0:1787739065.604470:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f2d00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001_UUID:6528:1874579665716864:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739065.604499:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adbab lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951c3 expref: 8 pid: 6524 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.604501:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1adbd5 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adbce expref: 6 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.604513:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xcc:0x0].0x0 (ffffa0730608b900) 00010000:00010000:1.0:1787739065.604515:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739065.604519:0:6528:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1adbd5 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adbce expref: 6 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.604528:0:6528:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1adbd5 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adbce expref: 6 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739065.604531:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1556 00000100:00100000:0.0:1787739065.604540:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f770c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2408:x1874579652500096:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739065.604548:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633da00, rc 0) 00010000:00010000:0.0:1787739065.604555:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739065.604560:0:6510:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adbab lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951c3 expref: 8 pid: 6524 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.604570:0:6510:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adbab lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951c3 expref: 8 pid: 6524 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.604590:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adbab lrc: 0/0,0 mode: --/PR res: [0x240000401:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951c3 expref: 8 pid: 6524 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739065.604598:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304245a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:13793:x1874579665716736:12345-0@lo:101:mdt00_004.0 Request processed in 10239us (10338us total) trans 0 rc 0/0 00010000:00010000:0.0:1787739065.604602:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xcc:0x0].0x0 (ffffa0730608b900) 00000100:00100000:1.0:1787739065.604606:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3324 00010000:00010000:0.0:1787739065.604606:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1adbd5 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adbce expref: 6 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.604614:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1adbd5 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1adbce expref: 6 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.604622:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1adbd5 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1adbce expref: 5 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739065.604650:0:13793:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968414 gen 2 00010000:00010000:0.0:1787739065.604651:0:6510:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1adbd5 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adbce expref: 5 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739065.604654:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968411 committed transno 4294968414 00000100:00100000:0.0:1787739065.604663:0:6510:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41d680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6510:1874579665716992:0@lo:105: 00000004:00080000:1.0:1787739065.604669:0:13793:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968414 committed transno 4294968414 00000100:00100000:1.0:1787739065.604684:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562e940 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665716736:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739065.604696:0:6510:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787739065.604704:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd000/0x7ee4453e4a1adbce lrc: 4/0,1 mode: --/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adbd5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.604718:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd000/0x7ee4453e4a1adbce lrc: 4/0,1 mode: --/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adbd5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739065.604723:0:6510:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074332da480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.604741:0:6507:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787739065.604751:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665716992 00000100:00100000:0.0:1787739065.604754:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665716992 00000100:00100000:0.0:1787739065.604790:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1189 00000100:00100000:0.0:1787739065.604797:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306b98450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6510:x1874579665716992:12345-0@lo:105: 00010000:00010000:0.0:1787739065.604806:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd000/0x7ee4453e4a1adbce lrc: 5/0,1 mode: --/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adbd5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.604815:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd000/0x7ee4453e4a1adbce lrc: 5/0,1 mode: --/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adbd5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.604823:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd000/0x7ee4453e4a1adbce lrc: 5/0,1 mode: --/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adbd5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.604830:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd000/0x7ee4453e4a1adbce lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1adbd5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.604838:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd000/0x7ee4453e4a1adbce lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1adbd5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.604845:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd000/0x7ee4453e4a1adbce lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1adbd5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.605121:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073063fd000) 00000100:00100000:0.0:1787739065.605161:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306b98450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6510:x1874579665716992:12345-0@lo:105: Request processed in 365us (463us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.605168:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1189 00010000:00010000:1.0:1787739065.605235:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd000/0x7ee4453e4a1adbce lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adbd5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739065.605245:0:6510:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41d680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6510:1874579665716992:0@lo:105: 00010000:00010000:1.0:1787739065.605245:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd000/0x7ee4453e4a1adbce lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adbd5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.605254:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739065.605296:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f770c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652500096:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 757us (953us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739065.605303:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1556 00000100:00100000:1.0:1787739065.605932:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042461c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665717120:0@lo:403:mdt00_004.0 00000100:00100000:1.0:1787739065.605974:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739065.606011:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074154fce40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739065.606230:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665717120 00000100:00100000:3.0:1787739065.606237:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665717120 00000100:00100000:3.0:1787739065.606274:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3325 00000100:00100000:3.0:1787739065.606281:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304247480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:13793:x1874579665717120:12345-0@lo:403:mdt00_004.0 00000100:00100000:0.0:1787739065.617164:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304247480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:13793:x1874579665717120:12345-0@lo:403:mdt00_004.0 Request processed in 10874us (11179us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.617182:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3325 00000100:00100000:1.0:1787739065.619201:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042461c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665717120:0@lo:403:mdt00_004.0 00000040:00080000:1.0:1787739065.624061:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].385, 952 off588120 00000040:00080000:1.0:1787739065.624408:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].385, 1032 off638728 00000004:00080000:1.0:1787739065.624438:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa074154fce40 with version 731 00000004:00080000:1.0:1787739065.624460:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 731 rpc_version 731 00000020:00100000:1.0:1787739065.624495:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739065.624504:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073042461c0 x1874579665717376/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1992/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739065.624526:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042461c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665717376:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739065.624839:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739065.624880:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234000 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.625092:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665717376 00000100:00100000:0.0:1787739065.625100:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665717376 00000100:00100000:0.0:1787739065.625160:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3326 00000100:00100000:0.0:1787739065.625170:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410b08c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665717376:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739065.625409:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410b08c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665717376:12345-0@lo:1000:osp_up1-0.0 Request processed in 240us (570us total) trans 4294968415 rc 0/0 00000100:00100000:0.0:1787739065.625438:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3326 00000100:00100000:3.0:1787739065.625612:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042461c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665717376:0@lo:1000:osp_up1-0.0 00000004:00200000:0.0:1787739065.627041:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1525:0x0]: returning size 4096 00010000:00010000:0.0:1787739065.627073:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adbc7 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627083:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adbc7 lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627091:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adbc7 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627097:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739065.627102:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adbc7 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627110:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adbc7 lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627128:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xcc:0x0].0x0 (ffffa07305eaed00) 00010000:00010000:0.0:1787739065.627132:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739065.627137:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adbc7 lrc: 2/0,0 mode: --/EX res: [0x200000400:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627149:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adbc7 lrc: 0/0,0 mode: --/EX res: [0x200000400:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627167:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd000/0x7ee4453e4a1adbce lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adbd5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627176:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd000/0x7ee4453e4a1adbce lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1adbd5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627184:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd000/0x7ee4453e4a1adbce lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1adbd5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627216:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739065.627225:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xcc:0x0] 00010000:00010000:0.0:1787739065.627232:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1adbc0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1525:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627239:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1adbc0 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1525:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627244:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1adbc0 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1525:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627248:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739065.627251:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1adbc0 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1525:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627256:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1adbc0 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1525:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627268:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1525:0x0].0x0 (ffffa07305eae500) 00010000:00010000:0.0:1787739065.627271:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739065.627274:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1adbc0 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1525:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627280:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1adbc0 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1525:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627297:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1adbb2 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627303:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1adbb2 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627308:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1adbb2 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627314:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1adbb2 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627319:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739065.627322:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1adbb2 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627328:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1adbb2 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627335:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:0.0:1787739065.627337:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739065.627339:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1adbb2 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627349:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1adbb2 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739065.627364:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000aa09ff59 reply state 000000000fd8b64c transno 4294983426 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1adbb9 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627377:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1adbb9 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.627384:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x60da72aa (ffffa07305eaec00) 00010000:00010000:0.0:1787739065.627387:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739065.627389:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1787739065.627484:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:17666:x1874579652499840:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 42517us (42724us total) trans 4294983426 rc 0/0 00000100:00100000:0.0:1787739065.627492:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23557 00010000:00010000:3.0:1787739065.627565:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739065.627571:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd000/0x7ee4453e4a1adbce lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1adbd5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.627578:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fd000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739065.627582:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd000/0x7ee4453e4a1adbce lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1adbd5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739065.627589:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd000/0x7ee4453e4a1adbce lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1adbd5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.627618:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd000/0x7ee4453e4a1adbce lrc: 2/0,0 mode: --/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adbd5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.627625:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739065.627653:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd000/0x7ee4453e4a1adbce lrc: 1/0,0 mode: --/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adbd5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.627660:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd000/0x7ee4453e4a1adbce lrc: 0/0,0 mode: --/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adbd5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.627675:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739065.627677:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739065.627678:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739065.627686:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787739065.627698:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665717504:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739065.627745:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739065.627954:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665717504 00000100:00100000:0.0:1787739065.627961:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665717504 00000100:00100000:0.0:1787739065.628017:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1354 00000100:00100000:0.0:1787739065.628026:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba5050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+7:3650:x1874579665717504:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739065.628037:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739065.628053:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1adbd5 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xcc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1adbce expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.628064:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xcc:0x0].0x0 (ffffa0730608b900) 00010000:00010000:0.0:1787739065.628070:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739065.628099:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba5050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+6:3650:x1874579665717504:12345-0@lo:103:ldlm_bl.0 Request processed in 75us (355us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.628106:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1354 00000100:00100000:3.0:1787739065.628257:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f752c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665717504:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739065.666967:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739065.675626:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739065.676097:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652500224 00000100:00100000:1.0:1787739065.676107:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652500224 00000100:00100000:1.0:1787739065.676167:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23558 00000100:00100000:1.0:1787739065.676176:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f752c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:17667:x1874579652500224:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739065.676197:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739065.676227:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fde00/0x7ee4453e4a1adbdc lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0951ca expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.676286:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc000/0x7ee4453e4a1adbe3 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.676299:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc000/0x7ee4453e4a1adbe3 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.676310:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc000/0x7ee4453e4a1adbe3 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.676323:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fdc00/0x7ee4453e4a1adbea lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.676330:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fdc00/0x7ee4453e4a1adbea lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.676461:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fdc00/0x7ee4453e4a1adbea lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.676507:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc000/0x7ee4453e4a1adbe3 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.676515:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc000/0x7ee4453e4a1adbe3 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.676521:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc000/0x7ee4453e4a1adbe3 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.676597:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc000/0x7ee4453e4a1adbe3 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.676604:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fc000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739065.676607:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc000/0x7ee4453e4a1adbe3 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.676614:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc000/0x7ee4453e4a1adbe3 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.676626:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eaef00) 00010000:00010000:1.0:1787739065.676630:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739065.676752:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc000/0x7ee4453e4a1adbe3 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.676761:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc000/0x7ee4453e4a1adbe3 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.676772:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fdc00/0x7ee4453e4a1adbea lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.676779:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fdc00/0x7ee4453e4a1adbea lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.676784:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fdc00/0x7ee4453e4a1adbea lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.676789:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fdc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739065.676793:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fdc00/0x7ee4453e4a1adbea lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.676802:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fdc00/0x7ee4453e4a1adbea lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.676810:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x1903f68f (ffffa07305eae000) 00010000:00010000:1.0:1787739065.676812:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739065.676815:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fdc00/0x7ee4453e4a1adbea lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.676819:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fdc00/0x7ee4453e4a1adbea lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.676850:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fde00/0x7ee4453e4a1adbdc lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951ca expref: 22 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.676870:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eaef00) 00010000:00010000:1.0:1787739065.676872:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739065.676876:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fde00/0x7ee4453e4a1adbdc lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951ca expref: 22 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.676886:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fde00, rc 0) 00000100:00100000:1.0:1787739065.676959:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f752c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:17667:x1874579652500224:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 783us (1341us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739065.676966:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23558 00000100:00100000:1.0:1787739065.687092:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739065.687148:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652500352 00000100:00100000:1.0:1787739065.687157:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652500352 00000100:00100000:1.0:1787739065.687212:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23559 00000100:00100000:1.0:1787739065.687222:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304246940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:17667:x1874579652500352:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739065.687244:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739065.687267:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1adbf1 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0951d1 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.687334:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1adbf8 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.688749:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1adbf8 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.688773:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1adbf8 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.688795:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1adbff lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.688805:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1adbff lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.688813:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1adbff lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.688852:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1adbf8 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.688860:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1adbf8 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.688867:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1adbf8 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.688873:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1adbf8 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.688880:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fd600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739065.688885:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1adbf8 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.688892:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1adbf8 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.688904:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae400) 00010000:00010000:1.0:1787739065.688907:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739065.688911:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1adbf8 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.688918:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1adbf8 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.690248:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1adbff lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.690263:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1adbff lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.690270:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1adbff lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.690277:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fcc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739065.690281:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1adbff lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.690288:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1adbff lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.690307:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x1903f68f (ffffa07305eae300) 00010000:00010000:1.0:1787739065.690310:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739065.690314:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1adbff lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.690321:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1adbff lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.690451:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1adbf1 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951d1 expref: 22 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.690464:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae400) 00010000:00010000:1.0:1787739065.690466:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739065.690470:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1adbf1 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951d1 expref: 22 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.690483:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fc800, rc 0) 00000100:00100000:1.0:1787739065.691022:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304246940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:17667:x1874579652500352:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 3797us (3937us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739065.691034:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23559 00000100:00100000:2.0:1787739065.705356:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739065.705455:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652500480 00000100:00100000:0.0:1787739065.705464:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652500480 00000100:00100000:0.0:1787739065.705523:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23560 00000100:00100000:0.0:1787739065.705534:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:17667:x1874579652500480:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739065.705553:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739065.705575:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adc06 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0951d8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.705634:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1adc0d lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.705647:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1adc0d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.705658:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1adc0d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739065.705702:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:0.0:1787739065.705707:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739065.705741:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adc06 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951d8 expref: 23 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.705758:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1adc0d lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951d8 expref: 22 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.705767:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305ca2900) 00010000:00010000:0.0:1787739065.705770:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739065.705773:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a000, rc 0) 00000100:00100000:0.0:1787739065.705824:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+22:17667:x1874579652500480:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 292us (476us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.705834:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23560 00000100:00100000:3.0:1787739065.714098:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739065.714578:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652500736 00000100:00100000:1.0:1787739065.714587:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652500736 00000100:00100000:1.0:1787739065.714645:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23561 00000100:00100000:1.0:1787739065.714656:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f76d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17667:x1874579652500736:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:1.0:1787739065.714762:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304246d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665717632:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739065.714843:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739065.714901:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234840 going to sleep for 11 seconds 00000100:00100000:3.0:1787739065.717029:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665717632 00000100:00100000:3.0:1787739065.717042:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665717632 00000100:00100000:3.0:1787739065.717100:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3327 00000100:00100000:3.0:1787739065.717108:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042443c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665717632:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739065.717218:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042443c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665717632:12345-0@lo:1000:mdt00_000.0 Request processed in 110us (2373us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739065.717225:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3327 00000100:00100000:1.0:1787739065.720303:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304246d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665717632:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739065.720376:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304246d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665717760:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739065.720445:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739065.720482:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234840 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.721241:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665717760 00000100:00100000:0.0:1787739065.721251:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665717760 00000100:00100000:0.0:1787739065.721297:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3328 00000100:00100000:0.0:1787739065.721305:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304247c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665717760:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739065.721440:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304247c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665717760:12345-0@lo:1000:mdt00_000.0 Request processed in 135us (994us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739065.721448:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3328 00000100:00100000:1.0:1787739065.722003:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304246d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665717760:0@lo:1000:mdt00_000.0 00010000:00010000:1.0:1787739065.722132:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fcc00/0x7ee4453e4a1adc14 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.722143:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fcc00/0x7ee4453e4a1adc14 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.722154:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fcc00/0x7ee4453e4a1adc14 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739065.722170:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304246d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665717888:0@lo:101:mdt00_000.0 00000100:00100000:1.0:1787739065.722204:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739065.722246:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.722300:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665717888 00000100:00100000:0.0:1787739065.722305:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665717888 00000100:00100000:3.0:1787739065.722451:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3329 00000100:00100000:3.0:1787739065.722461:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304245e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665717888:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739065.722473:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739065.722489:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1adc1b lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1adc14 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.722518:0:6529:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1adc0d lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951d8 expref: 21 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.722528:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1adc1b lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1adc14 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.722552:0:6529:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1adc0d lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951d8 expref: 21 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.722566:0:6529:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1adc0d lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951d8 expref: 21 pid: 13793 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739065.722578:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f743c0 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6529:1874579665718016:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739065.722639:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e000 going to sleep for 11 seconds 00000100:00100000:3.0:1787739065.728202:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f743c0 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000_UUID:6529:1874579665718016:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739065.728267:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1adc1b lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adc14 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.728282:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305ca2900) 00010000:00010000:3.0:1787739065.728286:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739065.728292:0:6529:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1adc1b lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adc14 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.728305:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739065.728444:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304245e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665717888:12345-0@lo:101:mdt00_000.0 Request processed in 5954us (6208us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739065.728457:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3329 00000100:00100000:1.0:1787739065.729082:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304246d00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665717888:0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739065.729118:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fcc00/0x7ee4453e4a1adc14 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adc1b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.729130:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fcc00/0x7ee4453e4a1adc14 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adc1b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739065.729166:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739065.729263:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652500864 00000100:00100000:0.0:1787739065.729272:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652500864 00010000:00010000:0.0:1787739065.729309:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1adc0d lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951d8 expref: 22 pid: 13793 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739065.729358:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1557 00000100:00100000:0.0:1787739065.729368:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f743c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+22:2408:x1874579652500864:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739065.729382:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739065.729389:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1adc0d lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951d8 expref: 22 pid: 13793 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.729402:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1adc0d lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951d8 expref: 22 pid: 13793 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.729446:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1adc0d lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951d8 expref: 22 pid: 13793 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.729466:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305ca2900) 00010000:00010000:0.0:1787739065.729472:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1adc1b lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adc14 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.729483:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1adc1b lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1adc14 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.729492:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1adc1b lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1adc14 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.729520:0:6509:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1adc1b lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adc14 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739065.729533:0:6509:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665718144:0@lo:105: 00000100:00100000:0.0:1787739065.729584:0:6509:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739065.729623:0:6509:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074332da540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739065.729724:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665718144 00000100:00100000:1.0:1787739065.729729:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665718144 00000100:00100000:1.0:1787739065.729768:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1190 00000100:00100000:1.0:1787739065.729776:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41cf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665718144:12345-0@lo:105: 00010000:00010000:1.0:1787739065.729788:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fcc00/0x7ee4453e4a1adc14 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adc1b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.729797:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fcc00/0x7ee4453e4a1adc14 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adc1b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.729807:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fcc00/0x7ee4453e4a1adc14 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adc1b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.729814:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fcc00/0x7ee4453e4a1adc14 lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1adc1b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.729821:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fcc00/0x7ee4453e4a1adc14 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1adc1b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.729829:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fcc00/0x7ee4453e4a1adc14 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1adc1b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.729852:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073063fcc00) 00000100:00100000:1.0:1787739065.729901:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41cf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665718144:12345-0@lo:105: Request processed in 126us (316us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739065.729909:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1190 00010000:00010000:3.0:1787739065.730061:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fcc00/0x7ee4453e4a1adc14 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adc1b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.730076:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fcc00/0x7ee4453e4a1adc14 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adc1b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739065.730108:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739065.730164:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1adc22 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.730172:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1adc22 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.730183:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1adc22 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739065.730206:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f76940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665718272:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739065.730254:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739065.730348:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e000 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.732216:0:6509:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665718144:0@lo:105: 00010000:00010000:0.0:1787739065.732243:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739065.732307:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f743c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:2408:x1874579652500864:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 2939us (3142us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.732318:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1557 00000100:00100000:1.0:1787739065.737458:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665718272 00000100:00100000:1.0:1787739065.737478:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665718272 00000100:00100000:1.0:1787739065.737566:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3330 00000100:00100000:1.0:1787739065.737579:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f75e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6523:x1874579665718272:12345-0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739065.737611:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739065.737712:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fde00/0x7ee4453e4a1adc29 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1adc22 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.737757:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fde00/0x7ee4453e4a1adc29 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1adc22 expref: 11 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.737840:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fde00/0x7ee4453e4a1adc29 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adc22 expref: 11 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.737850:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x1903f68f (ffffa07305eae300) 00010000:00010000:1.0:1787739065.737853:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739065.737857:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fde00, rc 0) 00000100:00100000:1.0:1787739065.737978:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f75e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665718272:12345-0@lo:101:mdt00_000.0 Request processed in 402us (7722us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739065.737986:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3330 00000100:00100000:2.0:1787739065.738092:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f76940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665718272:0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739065.738125:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1adc22 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adc29 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.738141:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1adc22 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adc29 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739065.738165:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:2.0:1787739065.738221:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f12c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665718400:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739065.738281:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739065.738324:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.738454:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665718400 00000100:00100000:0.0:1787739065.738461:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665718400 00000100:00100000:0.0:1787739065.738512:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3331 00000100:00100000:0.0:1787739065.738521:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665718400:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739065.738597:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665718400:12345-0@lo:1000:mdt00_000.0 Request processed in 76us (315us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.738604:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3331 00000100:00100000:2.0:1787739065.738772:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f12c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665718400:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739065.738810:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f12c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665718528:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739065.738833:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739065.738871:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.738910:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665718528 00000100:00100000:0.0:1787739065.738914:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665718528 00000100:00100000:0.0:1787739065.738944:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3332 00000100:00100000:0.0:1787739065.738950:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665718528:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739065.739024:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665718528:12345-0@lo:1000:mdt00_000.0 Request processed in 74us (191us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739065.739031:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3332 00000100:00100000:0.0:1787739065.739054:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f12c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665718528:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739065.739135:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665718656:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739065.739161:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739065.739175:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074332da480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.739185:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665718656 00000100:00100000:0.0:1787739065.739189:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665718656 00000100:00100000:0.0:1787739065.739204:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3333 00000100:00100000:0.0:1787739065.739210:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665718656:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739065.739256:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665718656:12345-0@lo:1000:mdt00_000.0 Request processed in 45us (94us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739065.739263:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3333 00000100:00100000:0.0:1787739065.739286:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665718656:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739065.739328:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665718784:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739065.739351:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739065.739364:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07441aca9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.739373:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665718784 00000100:00100000:0.0:1787739065.739377:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665718784 00000100:00100000:0.0:1787739065.739393:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3334 00000100:00100000:0.0:1787739065.739398:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665718784:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739065.739466:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665718784:12345-0@lo:1000:mdt00_000.0 Request processed in 67us (115us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.739473:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3334 00000100:00100000:0.0:1787739065.739494:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665718784:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739065.739528:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665718912:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739065.739549:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739065.739561:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07441aca9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.739570:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665718912 00000100:00100000:0.0:1787739065.739573:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665718912 00000100:00100000:0.0:1787739065.739589:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3335 00000100:00100000:0.0:1787739065.739594:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665718912:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739065.739643:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665718912:12345-0@lo:1000:mdt00_000.0 Request processed in 48us (93us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.739649:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3335 00000100:00100000:0.0:1787739065.739671:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665718912:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739065.739701:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665719040:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739065.739722:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739065.739735:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07441aca9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.739744:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665719040 00000100:00100000:0.0:1787739065.739747:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665719040 00000100:00100000:0.0:1787739065.739762:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3336 00000100:00100000:0.0:1787739065.739768:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665719040:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739065.739808:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665719040:12345-0@lo:1000:mdt00_000.0 Request processed in 39us (84us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.739814:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3336 00000100:00100000:0.0:1787739065.739834:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665719040:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739065.739861:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665719168:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739065.739880:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739065.739892:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07441aca9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.739902:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665719168 00000100:00100000:0.0:1787739065.739905:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665719168 00000100:00100000:0.0:1787739065.739919:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3337 00000100:00100000:0.0:1787739065.739924:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665719168:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739065.739963:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665719168:12345-0@lo:1000:mdt00_000.0 Request processed in 38us (81us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.739968:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3337 00000100:00100000:0.0:1787739065.739990:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665719168:0@lo:1000:mdt00_000.0 00000040:00080000:0.0:1787739065.740721:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].370, 2168 off562056 00000040:00080000:0.0:1787739065.740792:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].370, 2368 off613232 00000004:00080000:0.0:1787739065.740799:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07441aca480 with version 706 00000004:00080000:0.0:1787739065.740815:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 706 rpc_version 706 00000020:00100000:0.0:1787739065.740839:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739065.740848:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743c41c000 x1874579665719424/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739065.740865:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665719424:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739065.740905:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739065.740931:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07406c263c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739065.741306:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665719424 00000100:00100000:2.0:1787739065.741312:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665719424 00000100:00100000:2.0:1787739065.741354:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3338 00000100:00100000:2.0:1787739065.741362:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:7467:x1874579665719424:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739065.742397:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:7467:x1874579665719424:12345-0@lo:1000:osp_up0-1.0 Request processed in 1033us (1488us total) trans 4294983427 rc 0/0 00000100:00100000:2.0:1787739065.742407:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3338 00000100:00100000:0.0:1787739065.744649:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41c000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665719424:0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1787739065.745025:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fdc00/0x7ee4453e4a1adc30 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x184:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.745044:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fdc00/0x7ee4453e4a1adc30 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x184:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.745056:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fdc00/0x7ee4453e4a1adc30 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x184:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.745153:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc000/0x7ee4453e4a1adc37 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xca:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.745162:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc000/0x7ee4453e4a1adc37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.745169:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc000/0x7ee4453e4a1adc37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.745209:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fce00/0x7ee4453e4a1adc3e lrc: 3/0,1 mode: --/PW res: [0x200000401:0xca:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.745217:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fce00/0x7ee4453e4a1adc3e lrc: 3/0,1 mode: --/PW res: [0x200000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.745226:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fce00/0x7ee4453e4a1adc3e lrc: 3/0,1 mode: --/PW res: [0x200000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739065.745245:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304244780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665719552:0@lo:101:mdt00_000.0 00000100:00100000:1.0:1787739065.745304:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739065.745376:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.747665:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665719552 00000100:00100000:0.0:1787739065.747678:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665719552 00000100:00100000:0.0:1787739065.747786:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3339 00000100:00100000:0.0:1787739065.747797:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042452c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:6523:x1874579665719552:12345-0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739065.747832:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739065.747854:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adc45 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xca:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1adc3e expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.747894:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adc45 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1adc3e expref: 13 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.747910:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adc45 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adc3e expref: 13 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.747920:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xca:0x0].0x0 (ffffa07305ca2500) 00010000:00010000:0.0:1787739065.747923:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739065.747926:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609ac00, rc 0) 00000100:00100000:0.0:1787739065.748015:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042452c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:6523:x1874579665719552:12345-0@lo:101:mdt00_000.0 Request processed in 221us (2711us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.748022:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3339 00000100:00100000:1.0:1787739065.748097:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304244780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665719552:0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739065.748116:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fce00/0x7ee4453e4a1adc3e lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adc45 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.748127:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fce00/0x7ee4453e4a1adc3e lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adc45 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739065.748151:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005f24ebaf reply state 000000000af941ab transno 4294968416 ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc000/0x7ee4453e4a1adc37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.748163:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc000/0x7ee4453e4a1adc37 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.748169:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xca:0x0].0x0 (ffffa07305eae000) 00010000:00010000:1.0:1787739065.748172:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739065.748174:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787739065.748180:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fce00/0x7ee4453e4a1adc3e lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adc45 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.748188:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fce00/0x7ee4453e4a1adc3e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adc45 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739065.748200:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xca:0x0] 00000004:00010000:1.0:1787739065.748208:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005f24ebaf reply state 000000000af941ab transno 4294968416 ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fdc00/0x7ee4453e4a1adc30 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x184:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.748216:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fdc00/0x7ee4453e4a1adc30 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x184:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.748227:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x184:0x0].0x0 (ffffa07305eaef00) 00010000:00010000:1.0:1787739065.748229:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739065.748231:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:1.0:1787739065.748236:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x184:0x0]: returning size 4096 00000004:00080000:1.0:1787739065.748245:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739065.748253:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fcc00/0x7ee4453e4a1adc14 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adc1b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.748261:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fcc00/0x7ee4453e4a1adc14 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1adc1b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.748268:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fcc00/0x7ee4453e4a1adc14 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1adc1b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.748287:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739065.748293:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739065.748298:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1adc22 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adc29 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.748310:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1adc22 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adc29 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739065.748378:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f76d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17667:x1874579652500736:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 33723us (34289us total) trans 4294968416 rc 0/0 00000100:00100000:1.0:1787739065.748386:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23561 00010000:00010000:0.0:1787739065.753903:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739065.753916:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fcc00/0x7ee4453e4a1adc14 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1adc1b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.753929:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fcc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739065.753936:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fcc00/0x7ee4453e4a1adc14 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1adc1b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739065.753946:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fcc00/0x7ee4453e4a1adc14 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1adc1b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.754019:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fcc00/0x7ee4453e4a1adc14 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adc1b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.754026:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1787739065.754084:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665719680:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739065.754155:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787739065.754276:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fcc00/0x7ee4453e4a1adc14 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adc1b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.754286:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fcc00/0x7ee4453e4a1adc14 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adc1b expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.754302:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739065.754304:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739065.754306:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739065.755479:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665719680 00000100:00100000:3.0:1787739065.755499:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665719680 00000100:00100000:3.0:1787739065.755706:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1355 00000100:00100000:3.0:1787739065.755720:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304244780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+13:3652:x1874579665719680:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739065.755742:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739065.755784:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1adc1b lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1adc14 expref: 13 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.755816:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305ca2900) 00010000:00010000:3.0:1787739065.755835:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739065.762458:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304244780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+12:3652:x1874579665719680:12345-0@lo:103:ldlm_bl.0 Request processed in 6695us (8255us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739065.762480:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1355 00000100:00100000:1.0:1787739065.762632:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562f840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665719680:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739065.788268:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739065.788339:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652501120 00000100:00100000:1.0:1787739065.788349:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652501120 00000100:00100000:1.0:1787739065.788403:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23562 00000100:00100000:1.0:1787739065.788447:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:17668:x1874579652501120:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:1.0:1787739065.788596:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:17668:x1874579652501120:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 184us (337us total) trans 0 rc -17/-17 00000100:00100000:1.0:1787739065.788604:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23562 00000100:00100000:0.0:1787739065.826225:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739065.826339:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652501248 00000100:00100000:2.0:1787739065.826350:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652501248 00000100:00100000:2.0:1787739065.826428:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23563 00000100:00100000:2.0:1787739065.826438:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:17669:x1874579652501248:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739065.826457:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739065.826479:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1adc4c lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0951df expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.826540:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1adc53 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.826554:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1adc53 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.826570:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1adc53 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.826582:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc5a lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.826592:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fde00/0x7ee4453e4a1adc29 lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adc22 expref: 11 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.826600:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc5a lrc: 4/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.826631:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fde00/0x7ee4453e4a1adc29 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1adc22 expref: 11 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739065.826654:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f0000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665719808:0@lo:104: 00000100:00100000:2.0:1787739065.826703:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739065.826745:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068870c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739065.826763:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665719808 00000100:00100000:2.0:1787739065.826767:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665719808 00000100:00100000:2.0:1787739065.826796:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1191 00000100:00100000:2.0:1787739065.826802:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f2d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:13793:x1874579665719808:12345-0@lo:104: 00010000:00010000:2.0:1787739065.826813:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1adc22 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1adc29 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.826857:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739065.826863:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f2d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:13793:x1874579665719808:12345-0@lo:104: Request processed in 62us (160us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739065.826874:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1191 00010000:00010000:2.0:1787739065.826895:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739065.826900:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1adc22 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1adc29 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.826908:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739065.826913:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1adc22 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1adc29 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739065.826921:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1adc22 lrc: 4/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1adc29 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.827200:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1adc22 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1adc29 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.827207:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1787739065.827250:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665719936:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739065.827281:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787739065.827322:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1adc22 lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1adc29 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.827339:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1adc22 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1adc29 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.827355:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739065.827356:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739065.827358:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739065.827386:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f0000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665719808:0@lo:104: 00010000:00010000:2.0:1787739065.827401:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc5a lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739065.828185:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665719936 00000100:00100000:3.0:1787739065.828192:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665719936 00010000:00010000:3.0:1787739065.828216:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fde00/0x7ee4453e4a1adc29 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1adc22 expref: 12 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739065.828241:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1558 00000100:00100000:3.0:1787739065.828248:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+12:3653:x1874579665719936:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739065.828260:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:3.0:1787739065.846893:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983427 is committed 00000001:00080000:3.0:1787739065.846920:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983426 is committed 00000020:00080000:3.0:1787739065.846932:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa0743c0efd80 refcount 2 committed 0 result 0 batchid 387 00000020:00080000:3.0:1787739065.846938:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2c00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa0743c0efb40 00000020:00080000:3.0:1787739065.846942:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].384 00000020:00080000:3.0:1787739065.846945:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00d00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304276000 00000020:00080000:3.0:1787739065.846948:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].384 00000020:00080000:3.0:1787739065.846965:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07306887480 refcount 2 committed 0 result 0 batchid 386 00000020:00080000:3.0:1787739065.846968:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c693700 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07306887900 00000020:00080000:3.0:1787739065.846971:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].383 00000020:00080000:3.0:1787739065.846973:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c693800 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa074354b4800 00000020:00080000:3.0:1787739065.846975:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].383 00000020:00080000:3.0:1787739065.846980:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 384 batchid updated 00000020:00080000:3.0:1787739065.846987:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa073062416c0 refcount 2 committed 0 result 0 batchid 385 00000020:00080000:3.0:1787739065.846990:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00100 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa073062410c0 00000020:00080000:3.0:1787739065.846992:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].382 00000020:00080000:3.0:1787739065.846995:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00300 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304277e00 00000020:00080000:3.0:1787739065.847009:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].382 00000020:00080000:3.0:1787739065.847017:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 383 batchid updated 00010000:00010000:3.0:1787739065.847043:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ada31 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847054:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ada31 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847062:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ada31 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847069:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1de00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739065.847074:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ada31 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847083:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ada31 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847101:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x7ed215cf (ffffa07433235b00) 00010000:00010000:3.0:1787739065.847104:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739065.847109:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ada31 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847116:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ada31 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847134:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ada38 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1523:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847141:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ada38 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1523:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847147:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ada38 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1523:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847152:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609aa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739065.847155:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ada38 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1523:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847160:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ada38 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1523:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847167:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1523:0x0].0x0 (ffffa07307064900) 00010000:00010000:3.0:1787739065.847170:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739065.847173:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ada38 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1523:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847177:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ada38 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1523:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847193:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ada3f lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847199:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ada3f lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847204:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ada3f lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847209:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ba00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739065.847211:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ada3f lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847217:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ada3f lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847235:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xcb:0x0].0x0 (ffffa07307064600) 00010000:00010000:3.0:1787739065.847237:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739065.847240:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ada3f lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847246:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ada3f lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xcb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847263:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ada93 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847270:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ada93 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847275:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ada93 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847281:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609bc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739065.847284:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ada93 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847290:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ada93 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847301:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x7ed215cf (ffffa07433235b00) 00010000:00010000:3.0:1787739065.847304:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739065.847306:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ada93 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847312:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1ada93 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x7ed215cf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847325:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1adb57 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847331:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1adb57 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847337:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1adb57 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847342:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1da00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739065.847344:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1adb57 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847359:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1adb57 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847366:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x60da72aa (ffffa07305eaec00) 00010000:00010000:3.0:1787739065.847368:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739065.847371:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1adb57 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847376:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1da00/0x7ee4453e4a1adb57 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847384:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1adb5e lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1525:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847390:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1adb5e lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1525:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847396:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1adb5e lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1525:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847400:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739065.847403:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1adb5e lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1525:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847409:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1adb5e lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1525:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847433:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1525:0x0].0x0 (ffffa07305eae500) 00010000:00010000:3.0:1787739065.847436:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739065.847439:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1adb5e lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1525:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847445:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1adb5e lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1525:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847457:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1adb65 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847462:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1adb65 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847469:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1adb65 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847475:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739065.847478:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1adb65 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847483:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1adb65 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847490:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xcc:0x0].0x0 (ffffa07305eaed00) 00010000:00010000:3.0:1787739065.847492:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739065.847494:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1adb65 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847500:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1adb65 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847512:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1adbb9 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847517:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1adbb9 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847522:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1adbb9 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847528:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739065.847641:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1adbb9 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847708:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1adbb9 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847721:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x60da72aa (ffffa07305eaec00) 00010000:00010000:3.0:1787739065.847723:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739065.847726:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1adbb9 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847732:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1adbb9 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x60da72aa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847798:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fde00/0x7ee4453e4a1adc29 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1adc22 expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847808:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x1903f68f (ffffa07305eae300) 00010000:00010000:3.0:1787739065.847812:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc5a lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847822:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc5a lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847829:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc5a lrc: 4/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.847865:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739065.847955:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+6:3653:x1874579665719936:12345-0@lo:103:ldlm_bl.0 Request processed in 19705us (20671us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739065.847965:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1558 00000100:00100000:3.0:1787739065.848033:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665719936:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739065.848161:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc5a lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.848170:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc5a lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739065.848288:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562f0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665720064:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739065.848329:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739065.848359:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739065.848477:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665720064 00000100:00100000:3.0:1787739065.848488:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665720064 00000100:00100000:3.0:1787739065.848598:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3340 00000100:00100000:3.0:1787739065.848609:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:13793:x1874579665720064:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739065.848763:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:13793:x1874579665720064:12345-0@lo:1000:mdt00_004.0 Request processed in 156us (433us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739065.848771:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3340 00000100:00100000:0.0:1787739065.848900:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562f0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665720064:0@lo:1000:mdt00_004.0 00010000:00010000:0.0:1787739065.848946:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1adc61 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x184:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.848957:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1adc61 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x184:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.848967:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1adc61 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x184:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739065.848987:0:13793:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1adc61 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x184:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.849009:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1adc53 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.849015:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1adc53 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.849022:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1adc53 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.849028:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1adc53 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.849034:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739065.849037:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1adc53 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.849043:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1adc53 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.849052:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403a6f000) 00010000:00010000:0.0:1787739065.849054:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739065.849058:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1adc53 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.849132:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1adc53 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.849146:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc5a lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.849151:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc5a lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.849157:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc5a lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.849161:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739065.849164:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc5a lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.849170:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc5a lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.849178:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x1903f68f (ffffa07305eae300) 00010000:00010000:0.0:1787739065.849180:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739065.849183:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc5a lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.849188:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc5a lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.849235:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1adc4c lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951df expref: 15 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.849257:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1adc61 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x184:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951df expref: 14 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.849263:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x184:0x0].0x0 (ffffa07305ca2400) 00010000:00010000:0.0:1787739065.849266:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739065.849268:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a800, rc 0) 00000100:00100000:0.0:1787739065.849316:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17669:x1874579652501248:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 22880us (23101us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.849324:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23563 00000020:00080000:2.0:1787739065.849728:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 387 committed batchid 384 00000004:00080000:2.0:1787739065.849828:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa073068870c0 with version 732 00000020:00080000:2.0:1787739065.850725:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 383 cancel update log [0x240000401:0x3:0x0].380: rc = 0 00000020:00080000:2.0:1787739065.850785:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 383 cancel update log [0x200000400:0x3:0x0].380: rc = 0 00000004:00080000:2.0:1787739065.850816:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306887600 with version 733 00000004:00080000:0.0:1787739065.854502:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 732 rpc_version 732 00000020:00100000:0.0:1787739065.854618:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739065.854631:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743562f0c0 x1874579665720320/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739065.854719:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562f0c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665720320:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739065.855069:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739065.855151:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739065.855262:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665720320 00000100:00100000:3.0:1787739065.855271:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665720320 00000100:00100000:3.0:1787739065.855365:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3341 00000100:00100000:3.0:1787739065.855376:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665720320:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739065.855635:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665720320:12345-0@lo:1000:osp_up1-0.0 Request processed in 259us (776us total) trans 4294968417 rc 0/0 00000100:00100000:3.0:1787739065.855653:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3341 00000100:00100000:3.0:1787739065.856527:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562f0c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665720320:0@lo:1000:osp_up1-0.0 00000004:00080000:3.0:1787739065.856616:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 733 rpc_version 733 00000100:00100000:0.0:1787739065.856695:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739065.857284:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652501376 00000100:00100000:1.0:1787739065.857291:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652501376 00000100:00100000:1.0:1787739065.857342:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23564 00000100:00100000:1.0:1787739065.857349:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17669:x1874579652501376:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739065.857372:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739065.857394:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd800/0x7ee4453e4a1adc68 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x184:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0951e6 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.857480:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd400/0x7ee4453e4a1adc6f lrc: 3/1,0 mode: --/PR res: [0x240000402:0x184:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:2.0:1787739065.857709:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 384 cancel update log [0x240000401:0x3:0x0].381: rc = 0 00010000:00010000:1.0:1787739065.857737:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd400/0x7ee4453e4a1adc6f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x184:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.857750:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd400/0x7ee4453e4a1adc6f lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x184:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739065.857845:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x184:0x0]: returning size 4096 00000004:00200000:1.0:1787739065.857850:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739065.857897:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd800/0x7ee4453e4a1adc68 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x184:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951e6 expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787739065.857912:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00010000:00010000:1.0:1787739065.857918:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd400/0x7ee4453e4a1adc6f lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x184:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951e6 expref: 10 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739065.857923:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303f74780 x1874579665720576/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787739065.857927:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x184:0x0].0x0 (ffffa07305eaef00) 00010000:00010000:1.0:1787739065.857930:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739065.857933:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fd400, rc 0) 00000100:00100000:3.0:1787739065.857945:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665720576:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739065.857990:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17669:x1874579652501376:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 641us (1298us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739065.858007:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23564 00000020:00080000:2.0:1787739065.859020:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 384 cancel update log [0x200000400:0x3:0x0].381: rc = 0 00000020:00080000:2.0:1787739065.859035:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 384 00000100:00100000:3.0:1787739065.859248:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739065.859342:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ee40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.859610:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665720576 00000100:00100000:0.0:1787739065.859618:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665720576 00000100:00100000:2.0:1787739065.859948:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3342 00000100:00100000:2.0:1787739065.859958:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665720576:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739065.860155:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665720576:12345-0@lo:1000:osp_up1-0.0 Request processed in 196us (2156us total) trans 4294968418 rc 0/0 00000100:00100000:2.0:1787739065.860163:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3342 00000100:00100000:3.0:1787739065.860241:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74780 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665720576:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739065.868164:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739065.868472:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652501504 00000100:00100000:0.0:1787739065.868482:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652501504 00000100:00100000:2.0:1787739065.868557:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23565 00000100:00100000:2.0:1787739065.868569:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304247840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17669:x1874579652501504:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739065.868589:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739065.868609:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1adc76 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xca:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0951ed expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.868743:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1adc7d lrc: 3/1,0 mode: --/PR res: [0x240000400:0xca:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.868757:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1adc7d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.868773:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1adc7d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739065.868825:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xca:0x0]: returning size 4096 00000004:00200000:2.0:1787739065.868830:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739065.868865:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c400/0x7ee4453e4a1adc76 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xca:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951ed expref: 12 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.868882:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1adc7d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xca:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951ed expref: 11 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.868891:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xca:0x0].0x0 (ffffa07305eae000) 00010000:00010000:2.0:1787739065.868894:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739065.868897:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633de00, rc 0) 00000100:00100000:2.0:1787739065.868953:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304247840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17669:x1874579652501504:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 387us (800us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739065.868960:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23565 00000100:00100000:0.0:1787739065.876366:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739065.876461:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652501632 00000100:00100000:2.0:1787739065.876470:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652501632 00000100:00100000:2.0:1787739065.876518:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23566 00000100:00100000:2.0:1787739065.876527:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17669:x1874579652501632:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:2.0:1787739065.876545:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739065.876566:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1adc84 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xca:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0951f4 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.876642:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc8b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xca:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.876664:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adc45 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adc3e expref: 5 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.876673:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc8b lrc: 4/1,0 mode: --/PR res: [0x200000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.876707:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adc45 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1adc3e expref: 5 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739065.876723:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f0000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665720704:0@lo:104: 00000100:00100000:2.0:1787739065.876771:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739065.876812:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068879c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739065.876947:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665720704 00000100:00100000:1.0:1787739065.876951:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665720704 00000100:00100000:1.0:1787739065.876985:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1192 00000100:00100000:1.0:1787739065.876993:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f12c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:13793:x1874579665720704:12345-0@lo:104: 00010000:00010000:1.0:1787739065.877019:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fce00/0x7ee4453e4a1adc3e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1adc45 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.877087:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787739065.877093:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f12c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:13793:x1874579665720704:12345-0@lo:104: Request processed in 102us (322us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739065.877101:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1192 00010000:00010000:1.0:1787739065.877121:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739065.877126:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fce00/0x7ee4453e4a1adc3e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1adc45 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.877135:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fce00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739065.877140:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fce00/0x7ee4453e4a1adc3e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xca:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1adc45 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739065.877148:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fce00/0x7ee4453e4a1adc3e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xca:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1adc45 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.877171:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fce00/0x7ee4453e4a1adc3e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xca:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1adc45 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.877178:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739065.877196:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fce00/0x7ee4453e4a1adc3e lrc: 1/0,0 mode: --/PW res: [0x200000401:0xca:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1adc45 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.877204:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fce00/0x7ee4453e4a1adc3e lrc: 0/0,0 mode: --/PW res: [0x200000401:0xca:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1adc45 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.877217:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739065.877219:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739065.877233:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739065.877245:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304244000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665720832:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739065.877272:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739065.877311:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665720832 00000100:00100000:1.0:1787739065.877315:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665720832 00010000:00010000:1.0:1787739065.877331:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adc45 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1adc3e expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739065.877351:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1559 00000100:00100000:1.0:1787739065.877358:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304244b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+6:3650:x1874579665720832:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739065.877368:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1787739065.878962:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f0000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665720704:0@lo:104: 00010000:00010000:2.0:1787739065.878991:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc8b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00080000:1.0:1787739065.879977:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 387 batchid updated 00010000:00010000:0.0:1787739065.880104:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adc45 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1adc3e expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.880122:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xca:0x0].0x0 (ffffa07305ca2500) 00010000:00010000:0.0:1787739065.880128:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc8b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.880138:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc8b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.880145:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc8b lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.880179:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739065.880307:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc8b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.880315:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc8b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739065.880370:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xca:0x0]: returning size 4096 00000004:00200000:1.0:1787739065.880376:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739065.880443:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1adc84 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xca:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951f4 expref: 16 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.880460:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc8b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951f4 expref: 15 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.880467:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xca:0x0].0x0 (ffffa07305ca2500) 00010000:00010000:1.0:1787739065.880470:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739065.880473:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d200, rc 0) 00000100:00100000:1.0:1787739065.880518:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17669:x1874579652501632:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 3992us (4163us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739065.880527:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23566 00000100:00100000:0.0:1787739065.881714:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304244b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+5:3650:x1874579665720832:12345-0@lo:103:ldlm_bl.0 Request processed in 4349us (4433us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.881727:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1559 00000020:00080000:2.0:1787739065.881851:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 387 00000004:00080000:2.0:1787739065.881910:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa073068879c0 with version 734 00000020:00080000:2.0:1787739065.881928:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 385 cancel update log [0x240000401:0x3:0x0].382: rc = 0 00000020:00080000:2.0:1787739065.882012:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 385 cancel update log [0x200000400:0x3:0x0].382: rc = 0 00000004:00080000:2.0:1787739065.882036:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306887f00 with version 735 00000020:00080000:2.0:1787739065.882040:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 386 cancel update log [0x240000401:0x3:0x0].383: rc = 0 00000020:00080000:2.0:1787739065.882152:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 386 cancel update log [0x200000400:0x3:0x0].383: rc = 0 00000004:00080000:2.0:1787739065.882189:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306887480 with version 736 00000020:00080000:2.0:1787739065.882192:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 387 cancel update log [0x240000401:0x3:0x0].384: rc = 0 00000020:00080000:2.0:1787739065.882217:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 387 cancel update log [0x200000400:0x3:0x0].384: rc = 0 00000020:00080000:2.0:1787739065.882290:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 387 00000100:00100000:1.0:1787739065.882376:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304244000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665720832:0@lo:103:ldlm_bl.0 00000004:00080000:0.0:1787739065.882456:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 734 rpc_version 734 00000020:00100000:0.0:1787739065.882487:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739065.882496:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743562f840 x1874579665721088/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739065.882518:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562f840 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665721088:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739065.882573:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739065.882603:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af600 going to sleep for 11 seconds 00000100:00100000:3.0:1787739065.882616:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665721088 00000100:00100000:3.0:1787739065.882623:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665721088 00000100:00100000:3.0:1787739065.882672:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3343 00000100:00100000:3.0:1787739065.882681:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665721088:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739065.882938:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665721088:12345-0@lo:1000:osp_up1-0.0 Request processed in 257us (363us total) trans 4294968419 rc 0/0 00000100:00100000:3.0:1787739065.882946:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3343 00000100:00100000:0.0:1787739065.883120:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562f840 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665721088:0@lo:1000:osp_up1-0.0 00000004:00080000:0.0:1787739065.883130:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 735 rpc_version 735 00000020:00100000:0.0:1787739065.883140:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739065.883145:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073060d3c00 x1874579665721344/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739065.883157:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d3c00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665721344:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739065.883181:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739065.883209:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af600 going to sleep for 11 seconds 00000100:00100000:3.0:1787739065.883392:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665721344 00000100:00100000:3.0:1787739065.883396:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665721344 00000100:00100000:3.0:1787739065.883478:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3344 00000100:00100000:3.0:1787739065.883484:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665721344:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739065.883562:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665721344:12345-0@lo:1000:osp_up1-0.0 Request processed in 77us (379us total) trans 4294968420 rc 0/0 00000100:00100000:3.0:1787739065.883568:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3344 00000100:00100000:1.0:1787739065.883658:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d3c00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665721344:0@lo:1000:osp_up1-0.0 00000004:00080000:1.0:1787739065.883666:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 736 rpc_version 736 00000020:00100000:1.0:1787739065.883676:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739065.883681:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304244000 x1874579665721600/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739065.883694:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304244000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665721600:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739065.883717:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739065.883738:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0743c0ef300 going to sleep for 11 seconds 00000100:00100000:3.0:1787739065.883818:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665721600 00000100:00100000:3.0:1787739065.883822:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665721600 00000100:00100000:3.0:1787739065.883848:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3345 00000100:00100000:3.0:1787739065.883854:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042470c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665721600:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739065.883921:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042470c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+11:7740:x1874579665721600:12345-0@lo:1000:osp_up1-0.0 Request processed in 67us (203us total) trans 4294968421 rc 0/0 00000100:00100000:3.0:1787739065.883928:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3345 00000100:00100000:3.0:1787739065.883951:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304244000 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665721600:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739065.885271:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739065.885375:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652501760 00000100:00100000:3.0:1787739065.885383:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652501760 00000100:00100000:3.0:1787739065.885459:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1356 00000100:00100000:3.0:1787739065.885468:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:2408:x1874579652501760:12345-192.168.203.15@tcp:103:rmdir.0 00010000:00010000:3.0:1787739065.885482:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739065.885506:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a800/0x7ee4453e4a1adc61 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x184:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951df expref: 15 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.885522:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x184:0x0].0x0 (ffffa07305ca2400) 00010000:00010000:3.0:1787739065.885532:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739065.885576:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2408:x1874579652501760:12345-192.168.203.15@tcp:103:rmdir.0 Request processed in 109us (310us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739065.885583:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1356 00000100:00100000:3.0:1787739065.886739:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739065.886813:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652502016 00000100:00100000:0.0:1787739065.886821:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652502016 00000100:00100000:0.0:1787739065.886866:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23567 00000100:00100000:0.0:1787739065.886875:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f76580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17669:x1874579652502016:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:0.0:1787739065.886927:0:6523:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739065.886948:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd400/0x7ee4453e4a1adc6f lrc: 0/0,0 mode: --/PR res: [0x240000402:0x184:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951e6 expref: 11 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.886964:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x184:0x0].0x0 (ffffa07305eaef00) 00010000:00010000:0.0:1787739065.886968:0:6523:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739065.887029:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1adc92 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.887037:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1adc92 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.887047:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1adc92 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739065.887077:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d3840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665721728:0@lo:101:mdt00_000.0 00000100:00100000:0.0:1787739065.887122:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739065.887151:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739065.887349:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665721728 00000100:00100000:3.0:1787739065.887354:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665721728 00000100:00100000:3.0:1787739065.887388:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3346 00000100:00100000:3.0:1787739065.887396:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+5:6523:x1874579665721728:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739065.887406:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739065.887448:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1adc99 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1adc92 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.887471:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1adc99 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1adc92 expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.887484:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1adc99 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adc92 expref: 6 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.887491:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2d00) 00010000:00010000:3.0:1787739065.887494:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739065.887497:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739065.887550:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:6523:x1874579665721728:12345-0@lo:101:mdt00_000.0 Request processed in 155us (427us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739065.887557:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3346 00000100:00100000:2.0:1787739065.887633:0:6523:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983427 gen 2 00000004:00080000:2.0:1787739065.887641:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983419 committed transno 4294983427 00000020:00080000:2.0:1787739065.887647:0:6523:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07306241540 refcount 2 committed 0 result 0 batchid 376 00000020:00080000:2.0:1787739065.887650:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00a00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07306241240 00000020:00080000:2.0:1787739065.887654:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].369 00000020:00080000:2.0:1787739065.887656:0:6523:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00900 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073027da800 00000020:00080000:2.0:1787739065.887659:0:6523:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].369 00000004:00080000:2.0:1787739065.887688:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983421 committed transno 4294983427 00000004:00080000:2.0:1787739065.887699:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983422 committed transno 4294983427 00000004:00080000:2.0:1787739065.887708:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983425 committed transno 4294983427 00000004:00080000:2.0:1787739065.887717:0:6523:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983427 committed transno 4294983427 00000100:00100000:2.0:1787739065.887733:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d3840 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665721728:0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739065.887749:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1adc92 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adc99 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.887760:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1adc92 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adc99 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739065.887774:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739065.887796:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1adca0 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.887803:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1adca0 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.887812:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1adca0 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739065.887828:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f0780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665721856:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1787739065.887852:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739065.887871:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074332daf00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739065.887886:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665721856 00000100:00100000:2.0:1787739065.887889:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665721856 00000100:00100000:2.0:1787739065.887913:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3347 00000100:00100000:2.0:1787739065.887919:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:6523:x1874579665721856:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739065.887927:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739065.887938:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1adca7 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1adca0 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.887952:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1adca7 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1adca0 expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.887963:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1adca7 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adca0 expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.887970:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x1903f68f (ffffa07403a6f100) 00010000:00010000:2.0:1787739065.887972:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739065.887974:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c600, rc 0) 00000100:00100000:2.0:1787739065.888012:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665721856:12345-0@lo:101:mdt00_000.0 Request processed in 92us (159us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739065.888018:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3347 00000100:00100000:2.0:1787739065.888044:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f0780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665721856:0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739065.888053:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1adca0 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adca7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.888076:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1adca0 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adca7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739065.888088:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:2.0:1787739065.888113:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f0780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665721984:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739065.888134:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739065.888146:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074332da240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739065.888157:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665721984 00000100:00100000:2.0:1787739065.888160:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665721984 00000100:00100000:2.0:1787739065.888176:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3348 00000100:00100000:2.0:1787739065.888181:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665721984:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739065.888292:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665721984:12345-0@lo:1000:mdt00_000.0 Request processed in 109us (156us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739065.888298:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3348 00000100:00100000:2.0:1787739065.888321:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f0780 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665721984:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739065.888350:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41fc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665722112:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739065.888371:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739065.888383:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07441aca780 going to sleep for 11 seconds 00000100:00100000:2.0:1787739065.888393:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665722112 00000100:00100000:2.0:1787739065.888396:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665722112 00000100:00100000:2.0:1787739065.888439:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3349 00000100:00100000:2.0:1787739065.888445:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665722112:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739065.888512:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665722112:12345-0@lo:1000:mdt00_000.0 Request processed in 67us (140us total) trans 0 rc 1/1 00000100:00100000:2.0:1787739065.888519:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3349 00000100:00100000:2.0:1787739065.888541:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41fc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665722112:0@lo:1000:mdt00_000.0 00010000:00010000:2.0:1787739065.888575:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1adcae lrc: 3/0,1 mode: --/EX res: [0x240000402:0x184:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.888583:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1adcae lrc: 3/0,1 mode: --/EX res: [0x240000402:0x184:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.888590:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1adcae lrc: 3/0,1 mode: --/EX res: [0x240000402:0x184:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739065.888601:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41fc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665722240:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1787739065.888621:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739065.888633:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074332daf00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739065.888643:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665722240 00000100:00100000:2.0:1787739065.888646:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665722240 00000100:00100000:2.0:1787739065.888665:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3350 00000100:00100000:2.0:1787739065.888670:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6523:x1874579665722240:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739065.888677:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739065.888690:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1adcb5 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x184:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1adcae expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.888704:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1adcb5 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x184:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1adcae expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.888713:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1adcb5 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x184:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adcae expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.888721:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x184:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:2.0:1787739065.888725:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739065.888728:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d000, rc 0) 00000100:00100000:2.0:1787739065.888753:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665722240:12345-0@lo:101:mdt00_000.0 Request processed in 83us (131us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739065.888759:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3350 00000100:00100000:2.0:1787739065.888781:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41fc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665722240:0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739065.888792:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1adcae lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x184:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adcb5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.888800:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1adcae lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x184:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adcb5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.888819:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1adcbc lrc: 3/0,1 mode: --/EX res: [0x240000402:0x184:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.888827:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1adcbc lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x184:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.888835:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1adcbc lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x184:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.888850:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1adcc3 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xca:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.888861:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1adc7d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951ed expref: 10 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.888869:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1adcc3 lrc: 4/0,1 mode: --/EX res: [0x240000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.888891:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1adc7d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951ed expref: 10 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.888910:0:6523:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1adc7d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951ed expref: 10 pid: 6524 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739065.888921:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41fc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665722368:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739065.888974:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074332daf00 going to sleep for 11 seconds 00000020:00080000:3.0:1787739065.889134:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 377 committed batchid 375 00000020:00080000:3.0:1787739065.889189:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 375 00000100:00100000:3.0:1787739065.905757:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739065.905787:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652502144 00000100:00100000:3.0:1787739065.905795:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652502144 00010000:00010000:3.0:1787739065.905832:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1adc7d lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951ed expref: 11 pid: 6524 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739065.905871:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1560 00000100:00100000:3.0:1787739065.905881:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2409:x1874579652502144:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739065.905897:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739065.905903:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1adc7d lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xca:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951ed expref: 11 pid: 6524 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.905916:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1adc7d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xca:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951ed expref: 11 pid: 6524 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.905937:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xca:0x0].0x0 (ffffa07305eae000) 00010000:00010000:3.0:1787739065.905942:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1adcc3 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.905951:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1adcc3 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.905958:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1adcc3 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.905975:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739065.906030:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2409:x1874579652502144:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 151us (283us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739065.906038:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1560 00010000:00010000:2.0:1787739065.906459:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633de00/0x7ee4453e4a1adc7d lrc: 0/0,0 mode: --/PR res: [0x240000400:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951ed expref: 10 pid: 6524 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739065.906475:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41fc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665722368:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739065.906495:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1adcc3 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.906504:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1adcc3 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.906510:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1adcc3 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.906551:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1adcca lrc: 3/0,1 mode: --/EX res: [0x200000401:0xca:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.906558:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1adcca lrc: 3/0,1 mode: --/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.906567:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1adcca lrc: 3/0,1 mode: --/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739065.906584:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41fc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665722496:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1787739065.906624:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739065.906651:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074332daf00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.906818:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665722496 00000100:00100000:0.0:1787739065.906823:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665722496 00000100:00100000:0.0:1787739065.906865:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3351 00000100:00100000:0.0:1787739065.906873:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157eb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:6523:x1874579665722496:12345-0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739065.906888:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739065.906903:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adcd1 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xca:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1adcca expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.906925:0:6528:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc8b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951f4 expref: 13 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.906934:0:6528:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adcd1 lrc: 4/0,0 mode: --/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1adcca expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.906956:0:6528:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc8b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951f4 expref: 13 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.906969:0:6528:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc8b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951f4 expref: 13 pid: 13793 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739065.906979:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6528:1874579665722624:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739065.907038:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.914394:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000_UUID:6528:1874579665722624:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739065.914461:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adcd1 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adcca expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.914475:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xca:0x0].0x0 (ffffa07305ca2500) 00010000:00010000:0.0:1787739065.914479:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739065.914483:0:6528:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adcd1 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adcca expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.914492:0:6528:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adcd1 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adcca expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.914505:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609ac00, rc 0) 00000100:00100000:0.0:1787739065.914568:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157eb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665722496:12345-0@lo:101:mdt00_000.0 Request processed in 7695us (7942us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.914576:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3351 00000100:00100000:2.0:1787739065.914691:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41fc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665722496:0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739065.914711:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1adcca lrc: 4/0,1 mode: --/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adcd1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.914722:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1adcca lrc: 4/0,1 mode: --/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adcd1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739065.914789:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739065.914861:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652502272 00000100:00100000:0.0:1787739065.914868:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652502272 00010000:00010000:0.0:1787739065.914897:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc8b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951f4 expref: 14 pid: 13793 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739065.914930:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1561 00000100:00100000:0.0:1787739065.914940:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba7450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:2409:x1874579652502272:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739065.914955:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739065.914961:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc8b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951f4 expref: 14 pid: 13793 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.914973:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc8b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951f4 expref: 14 pid: 13793 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.914993:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1adc8b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xca:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951f4 expref: 14 pid: 13793 timeout: 752 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.915019:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xca:0x0].0x0 (ffffa07305ca2500) 00010000:00010000:0.0:1787739065.915024:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adcd1 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adcca expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.915034:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adcd1 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1adcca expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.915045:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adcd1 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1adcca expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.915142:0:6509:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adcd1 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adcca expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739065.915160:0:6509:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665722752:0@lo:105: 00000100:00100000:0.0:1787739065.915198:0:6509:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739065.915325:0:6509:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739065.915612:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665722752 00000100:00100000:2.0:1787739065.915618:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665722752 00000100:00100000:0.0:1787739065.915762:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1193 00000100:00100000:0.0:1787739065.915770:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d0f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665722752:12345-0@lo:105: 00010000:00010000:0.0:1787739065.915783:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1adcca lrc: 5/0,1 mode: --/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adcd1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.915792:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1adcca lrc: 5/0,1 mode: --/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adcd1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.915800:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1adcca lrc: 5/0,1 mode: --/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adcd1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.915807:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1adcca lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1adcd1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.915814:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1adcca lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1adcd1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.915826:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1adcca lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1adcd1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.915882:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633cc00) 00000100:00100000:0.0:1787739065.915936:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d0f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665722752:12345-0@lo:105: Request processed in 167us (737us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.915944:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1193 00000100:00100000:3.0:1787739065.916232:0:6509:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665722752:0@lo:105: 00010000:00010000:3.0:1787739065.916249:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739065.916298:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba7450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:2409:x1874579652502272:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 1358us (2083us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739065.916306:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1561 00010000:00010000:2.0:1787739065.916635:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1adcca lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adcd1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.916832:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1adcca lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adcd1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739065.917924:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157ebc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665722880:0@lo:403:mdt00_000.0 00000100:00100000:2.0:1787739065.917994:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739065.918085:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074332daf00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.918287:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665722880 00000100:00100000:0.0:1787739065.918293:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665722880 00000100:00100000:0.0:1787739065.918382:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3352 00000100:00100000:0.0:1787739065.918391:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157e9e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665722880:12345-0@lo:403:mdt00_000.0 00000100:00100000:0.0:1787739065.920490:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157e9e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6523:x1874579665722880:12345-0@lo:403:mdt00_000.0 Request processed in 2092us (2487us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.920504:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3352 00000100:00100000:1.0:1787739065.920778:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074157ebc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665722880:0@lo:403:mdt00_000.0 00000040:00080000:1.0:1787739065.921927:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].371, 664 off562720 00000040:00080000:1.0:1787739065.922210:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].371, 744 off613976 00000004:00080000:1.0:1787739065.922217:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0743c0efd80 with version 707 00000004:00080000:0.0:1787739065.922365:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 707 rpc_version 707 00000020:00100000:0.0:1787739065.922398:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739065.922408:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073060d30c0 x1874579665723136/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1808/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739065.922506:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d30c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665723136:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739065.922569:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739065.922647:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739065.922688:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665723136 00000100:00100000:2.0:1787739065.922695:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665723136 00000100:00100000:2.0:1787739065.922775:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3353 00000100:00100000:2.0:1787739065.922784:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665723136:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739065.923294:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665723136:12345-0@lo:1000:osp_up0-1.0 Request processed in 441us (654us total) trans 4294983428 rc 0/0 00000100:00100000:2.0:1787739065.923303:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3353 00000100:00100000:1.0:1787739065.923520:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d30c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665723136:0@lo:1000:osp_up0-1.0 00000004:00200000:3.0:1787739065.923654:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x184:0x0]: returning size 4096 00010000:00010000:3.0:1787739065.923678:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1adcc3 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.923688:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1adcc3 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.923695:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1adcc3 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.923702:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739065.923706:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1adcc3 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.923714:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1adcc3 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.923730:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xca:0x0].0x0 (ffffa07305eae000) 00010000:00010000:3.0:1787739065.923733:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739065.923739:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1adcc3 lrc: 2/0,0 mode: --/EX res: [0x240000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.923745:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c000/0x7ee4453e4a1adcc3 lrc: 0/0,0 mode: --/EX res: [0x240000400:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.923759:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1adcca lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adcd1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.923767:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1adcca lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1adcd1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.923775:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1adcca lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1adcd1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.923827:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739065.923837:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xca:0x0] 00010000:00010000:3.0:1787739065.923845:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1adcbc lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x184:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.923852:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1adcbc lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x184:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.923858:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1adcbc lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x184:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.923864:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739065.923867:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1adcbc lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x184:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.923873:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1adcbc lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x184:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.923881:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x184:0x0].0x0 (ffffa07305eaef00) 00010000:00010000:3.0:1787739065.923884:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739065.923886:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1adcbc lrc: 2/0,0 mode: --/EX res: [0x240000402:0x184:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.923893:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1adcbc lrc: 0/0,0 mode: --/EX res: [0x240000402:0x184:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.923904:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1adcae lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x184:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adcb5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.923912:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1adcae lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x184:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1adcb5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.923919:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1adcae lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x184:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1adcb5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.923946:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787739065.923952:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1adcca lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1adcd1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.923959:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633cc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739065.923964:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1adcca lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1adcd1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739065.923972:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1adcca lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1adcd1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.924006:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1adcca lrc: 2/0,0 mode: --/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adcd1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.924014:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1787739065.924055:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665723264:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739065.924108:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739065.924239:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665723264 00000100:00100000:2.0:1787739065.924244:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665723264 00000100:00100000:2.0:1787739065.924283:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1357 00000100:00100000:2.0:1787739065.924291:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304246580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+10:3651:x1874579665723264:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739065.924302:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739065.924320:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adcd1 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1adcca expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739065.924332:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xca:0x0].0x0 (ffffa07305ca2500) 00010000:00010000:2.0:1787739065.924340:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739065.924469:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1adcca lrc: 1/0,0 mode: --/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adcd1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.924478:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633cc00/0x7ee4453e4a1adcca lrc: 0/0,0 mode: --/EX res: [0x200000401:0xca:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adcd1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.924492:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739065.924495:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1adcae lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x184:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1adcb5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.924501:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633da00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739065.924504:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1adcae lrc: 5/0,0 mode: EX/EX res: [0x240000402:0x184:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1adcb5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739065.924510:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1adcae lrc: 5/0,0 mode: EX/EX res: [0x240000402:0x184:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1adcb5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.924526:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1adcae lrc: 4/0,0 mode: --/EX res: [0x240000402:0x184:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adcb5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.924537:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739065.924552:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1adcae lrc: 3/0,0 mode: --/EX res: [0x240000402:0x184:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adcb5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.924559:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739065.924561:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739065.924563:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739065.924574:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1787739065.924582:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665723392:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739065.924607:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787739065.924706:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:3.0:1787739065.924712:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633da00/0x7ee4453e4a1adcae lrc: 0/0,0 mode: --/EX res: [0x240000402:0x184:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adcb5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739065.924732:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739065.924737:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1adc92 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adc99 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.924745:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1adc92 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1adc99 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.924752:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1adc92 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1adc99 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.924764:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739065.924767:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739065.924772:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1adca0 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adca7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.924784:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1adca0 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adca7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739065.924843:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f76580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17669:x1874579652502016:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 37967us (38110us total) trans 4294968422 rc 0/0 00000100:00100000:3.0:1787739065.924851:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23567 00010000:00010000:0.0:1787739065.925009:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739065.925015:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1adc92 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1adc99 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.925022:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609be00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739065.925026:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1adc92 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1adc99 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739065.925034:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1adc92 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1adc99 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.925053:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1adc92 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adc99 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.925059:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739065.925086:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1adc92 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adc99 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.925098:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609be00/0x7ee4453e4a1adc92 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adc99 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.925110:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739065.925111:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739065.925113:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739065.925346:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665723520:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739065.925375:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739065.925437:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665723392 00000100:00100000:3.0:1787739065.925443:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665723392 00000100:00100000:3.0:1787739065.925472:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665723520 00000100:00100000:3.0:1787739065.925475:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665723520 00000100:00100000:3.0:1787739065.925489:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1358 00000100:00100000:3.0:1787739065.925495:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3653:x1874579665723392:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739065.925504:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739065.925518:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1adcb5 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x184:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1adcae expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.925531:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x184:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739065.925538:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739065.925568:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+10:3653:x1874579665723392:12345-0@lo:103:ldlm_bl.0 Request processed in 73us (960us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739065.925574:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1358 00000100:00100000:3.0:1787739065.925587:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1359 00000100:00100000:3.0:1787739065.925593:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f743c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+9:3653:x1874579665723520:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739065.925603:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739065.925614:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1adc99 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1adc92 expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739065.925624:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2d00) 00010000:00010000:3.0:1787739065.925629:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739065.925651:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f743c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+8:3653:x1874579665723520:12345-0@lo:103:ldlm_bl.0 Request processed in 59us (275us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739065.925657:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1359 00000100:00100000:3.0:1787739065.925690:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665723520:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739065.925700:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d2d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665723392:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739065.925836:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304246580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+7:3651:x1874579665723264:12345-0@lo:103:ldlm_bl.0 Request processed in 1545us (1727us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739065.925844:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1357 00010000:00010000:2.0:1787739065.925868:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739065.925930:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665723264:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739065.967089:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739065.969501:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652502400 00000100:00100000:0.0:1787739065.969516:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652502400 00000100:00100000:0.0:1787739065.969580:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23568 00000100:00100000:0.0:1787739065.969591:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17670:x1874579652502400:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739065.969613:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739065.969637:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1adcd8 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0951fb expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.969697:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1adcdf lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.969710:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1adcdf lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.969721:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1adcdf lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.969733:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1adce6 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.969740:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1adce6 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.969751:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1adce6 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.969787:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1adcdf lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.969794:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1adcdf lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.969801:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1adcdf lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.969808:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1adcdf lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.969814:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739065.969818:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1adcdf lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.969825:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1adcdf lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.969836:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305ca2a00) 00010000:00010000:0.0:1787739065.969839:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739065.969843:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1adcdf lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.969849:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1adcdf lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.969861:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1adce6 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.969868:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1adce6 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.969873:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1adce6 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.969890:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739065.969893:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1adce6 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.969898:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1adce6 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.969905:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xa20ac6a2 (ffffa07305ca2d00) 00010000:00010000:0.0:1787739065.969908:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739065.969910:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1adce6 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.969916:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a600/0x7ee4453e4a1adce6 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.969942:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1adcd8 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951fb expref: 14 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.969952:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305ca2a00) 00010000:00010000:0.0:1787739065.969954:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739065.969958:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1adcd8 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951fb expref: 14 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.969968:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609be00, rc 0) 00000100:00100000:0.0:1787739065.970046:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17670:x1874579652502400:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 457us (2973us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739065.970054:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23568 00000100:00100000:0.0:1787739065.976599:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739065.976669:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652502528 00000100:00100000:1.0:1787739065.976678:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652502528 00000100:00100000:1.0:1787739065.976741:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23569 00000100:00100000:1.0:1787739065.976751:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17670:x1874579652502528:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739065.976773:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739065.976797:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fce00/0x7ee4453e4a1adced lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095202 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.976859:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1adcf4 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.976873:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1adcf4 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.976885:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1adcf4 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.976897:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd800/0x7ee4453e4a1adcfb lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.976904:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd800/0x7ee4453e4a1adcfb lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.976910:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd800/0x7ee4453e4a1adcfb lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.976951:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1adcf4 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.976957:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1adcf4 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.976963:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1adcf4 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.976970:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1adcf4 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.976977:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fd400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739065.976981:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1adcf4 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.976988:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1adcf4 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.977009:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eaee00) 00010000:00010000:1.0:1787739065.977012:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739065.977016:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1adcf4 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.977023:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1adcf4 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.977037:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd800/0x7ee4453e4a1adcfb lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.977042:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd800/0x7ee4453e4a1adcfb lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.977048:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd800/0x7ee4453e4a1adcfb lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.977052:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fd800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739065.977055:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd800/0x7ee4453e4a1adcfb lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.977061:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd800/0x7ee4453e4a1adcfb lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.977072:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xa20ac6a2 (ffffa07305eae100) 00010000:00010000:1.0:1787739065.977075:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739065.977078:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd800/0x7ee4453e4a1adcfb lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.977088:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd800/0x7ee4453e4a1adcfb lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.977118:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fce00/0x7ee4453e4a1adced lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095202 expref: 14 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.977128:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eaee00) 00010000:00010000:1.0:1787739065.977130:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739065.977133:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fce00/0x7ee4453e4a1adced lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095202 expref: 14 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.977143:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fce00, rc 0) 00000100:00100000:3.0:1787739065.977276:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17670:x1874579652502528:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 525us (684us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739065.977283:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23569 00000100:00100000:3.0:1787739065.983079:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739065.984662:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652502656 00000100:00100000:1.0:1787739065.984676:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652502656 00000100:00100000:1.0:1787739065.984732:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23570 00000100:00100000:1.0:1787739065.984743:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f77840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17670:x1874579652502656:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739065.984762:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739065.984787:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fde00/0x7ee4453e4a1add02 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095209 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.984844:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1add09 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.984862:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1add09 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.984873:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1add09 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739065.984929:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:1.0:1787739065.984935:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739065.984975:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fde00/0x7ee4453e4a1add02 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095209 expref: 15 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.985006:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1add09 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095209 expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739065.985016:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae400) 00010000:00010000:1.0:1787739065.985019:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739065.985023:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fcc00, rc 0) 00000100:00100000:1.0:1787739065.985082:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f77840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17670:x1874579652502656:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 340us (2011us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739065.985089:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23570 00000100:00100000:0.0:1787739065.992574:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739065.992615:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652502912 00000100:00100000:0.0:1787739065.992623:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652502912 00000100:00100000:0.0:1787739065.992676:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23571 00000100:00100000:0.0:1787739065.992685:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17670:x1874579652502912:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:0.0:1787739065.992743:0:6524:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739065.992771:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1add09 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095209 expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.992786:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eae400) 00010000:00010000:0.0:1787739065.992796:0:6524:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739065.992856:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1add10 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.992868:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1add10 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.992878:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1add10 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.992889:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1add17 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.992896:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1add17 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.992907:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1add17 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1787739065.993895:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].387, 2168 off591992 00000040:00080000:0.0:1787739065.993976:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].386, 2368 off641096 00000004:00080000:0.0:1787739065.993983:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa074052afd80 with version 737 00000004:00080000:0.0:1787739065.994028:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 737 rpc_version 737 00000020:00100000:0.0:1787739065.994060:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739065.994075:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073060d3480 x1874579665723776/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 3976/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739065.994100:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d3480 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665723776:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739065.994164:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739065.994208:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af600 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.994232:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665723776 00000100:00100000:0.0:1787739065.994237:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665723776 00000100:00100000:0.0:1787739065.994276:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3354 00000100:00100000:0.0:1787739065.994284:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+11:7740:x1874579665723776:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739065.994665:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+12:7740:x1874579665723776:12345-0@lo:1000:osp_up1-0.0 Request processed in 381us (500us total) trans 4294968423 rc 0/0 00000100:00100000:0.0:1787739065.994674:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3354 00000100:00100000:0.0:1787739065.994730:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d3480 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665723776:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787739065.994821:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1add1e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1528:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.994835:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1add1e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1528:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.994848:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1add1e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1528:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.994863:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1add25 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xcd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.994869:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1add25 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.994876:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1add25 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.994899:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a800/0x7ee4453e4a1add2c lrc: 3/0,1 mode: --/PW res: [0x240000401:0xcd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.994906:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a800/0x7ee4453e4a1add2c lrc: 3/0,1 mode: --/PW res: [0x240000401:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.994915:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a800/0x7ee4453e4a1add2c lrc: 3/0,1 mode: --/PW res: [0x240000401:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739065.994928:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665723904:0@lo:101:mdt00_001.0 00000100:00100000:0.0:1787739065.994954:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739065.994971:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a459000 going to sleep for 11 seconds 00000100:00100000:0.0:1787739065.994983:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665723904 00000100:00100000:0.0:1787739065.994987:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665723904 00000100:00100000:0.0:1787739065.995026:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3355 00000100:00100000:0.0:1787739065.995032:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+12:6524:x1874579665723904:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739065.995043:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739065.995057:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1add33 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xcd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1add2c expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.995076:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1add33 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1add2c expref: 13 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.995088:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1add33 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1add2c expref: 13 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.995097:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xcd:0x0].0x0 (ffffa07305ca2100) 00010000:00010000:0.0:1787739065.995100:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739065.995103:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a000, rc 0) 00000100:00100000:0.0:1787739065.995130:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+13:6524:x1874579665723904:12345-0@lo:101:mdt00_001.0 Request processed in 98us (175us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739065.995136:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3355 00000100:00100000:0.0:1787739065.995159:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665723904:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739065.995169:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a800/0x7ee4453e4a1add2c lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1add33 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.995179:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a800/0x7ee4453e4a1add2c lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1add33 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739065.995197:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000051952ffa reply state 00000000d80254b9 transno 4294983429 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1add25 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.995208:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1add25 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.995214:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xcd:0x0].0x0 (ffffa07305ca2e00) 00010000:00010000:0.0:1787739065.995217:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739065.995220:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739065.995226:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a800/0x7ee4453e4a1add2c lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1add33 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.995234:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a800/0x7ee4453e4a1add2c lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1add33 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739065.995243:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xcd:0x0] 00000004:00010000:0.0:1787739065.995247:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000051952ffa reply state 00000000d80254b9 transno 4294983429 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1add1e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1528:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.995254:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1add1e lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1528:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.995261:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1528:0x0].0x0 (ffffa07305ca2600) 00010000:00010000:0.0:1787739065.995263:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739065.995264:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:0.0:1787739065.995268:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1528:0x0]: returning size 4096 00010000:00010000:0.0:1787739065.995275:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1add10 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.995280:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1add10 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.995285:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1add10 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.995291:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1add10 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.995296:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739065.995300:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1add10 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.995312:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1add10 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.995321:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305ca2c00) 00010000:00010000:0.0:1787739065.995323:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739065.995326:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1add10 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.995333:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1add10 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739065.995347:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000051952ffa reply state 00000000d80254b9 transno 4294983429 ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1add17 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.995355:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1add17 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739065.995360:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xa20ac6a2 (ffffa07305ca2800) 00010000:00010000:0.0:1787739065.995363:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739065.995364:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 2 00000100:00100000:0.0:1787739065.995438:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17670:x1874579652502912:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 2752us (2872us total) trans 4294983429 rc 0/0 00000100:00100000:0.0:1787739065.995445:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23571 00000100:00100000:2.0:1787739066.025437:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739066.025532:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652503168 00000100:00100000:3.0:1787739066.025543:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652503168 00000100:00100000:3.0:1787739066.025597:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23572 00000100:00100000:3.0:1787739066.025607:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157ead00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17671:x1874579652503168:12345-192.168.203.15@tcp:36:mkdir.0 00000100:00100000:3.0:1787739066.025791:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157ead00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17671:x1874579652503168:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 184us (386us total) trans 0 rc -17/-17 00000100:00100000:3.0:1787739066.025799:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23572 00000100:00100000:0.0:1787739066.049938:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739066.050036:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652503296 00000100:00100000:3.0:1787739066.050045:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652503296 00000100:00100000:3.0:1787739066.050103:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23573 00000100:00100000:3.0:1787739066.050112:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17672:x1874579652503296:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739066.050132:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739066.050154:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1add3a lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095210 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.050210:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1add41 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.050225:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1add41 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.050238:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1add41 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.050247:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1add48 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.050255:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1add48 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.050262:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1add48 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.050798:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1add4f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1528:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.050810:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1add4f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1528:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.050818:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1add4f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1528:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739066.050910:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1528:0x0]: returning size 4096 00000004:00200000:2.0:1787739066.050915:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:2.0:1787739066.050935:0:6525:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1add4f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1528:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.050948:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1add41 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.050956:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1add41 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.050962:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1add41 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.050968:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1add41 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.050974:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739066.050978:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1add41 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.050986:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1add41 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.051007:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2d00) 00010000:00010000:2.0:1787739066.051010:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739066.051014:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1add41 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.051020:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1add41 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.051032:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1add48 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.051037:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1add48 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.051042:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1add48 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.051047:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aba00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739066.051050:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1add48 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.051056:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1add48 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.051063:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xa20ac6a2 (ffffa07305ca2800) 00010000:00010000:2.0:1787739066.051065:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739066.051067:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1add48 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.051073:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1add48 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.051099:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1add3a lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095210 expref: 17 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.051116:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1add4f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1528:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095210 expref: 16 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.051124:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1528:0x0].0x0 (ffffa07305ca2600) 00010000:00010000:2.0:1787739066.051127:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787739066.051129:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c200, rc 0) 00000100:00100000:2.0:1787739066.051228:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17672:x1874579652503296:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 1115us (1298us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739066.051236:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23573 00000100:00100000:2.0:1787739066.059723:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739066.059871:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652503424 00000100:00100000:0.0:1787739066.059881:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652503424 00000100:00100000:0.0:1787739066.059941:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23574 00000100:00100000:0.0:1787739066.059951:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157ea940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17672:x1874579652503424:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:0.0:1787739066.059981:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739066.060011:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1add56 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xcd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095217 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.060073:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1add5d lrc: 3/1,0 mode: --/PR res: [0x200000400:0xcd:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.060088:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1add5d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.060099:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1add5d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739066.060151:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xcd:0x0]: returning size 4096 00000004:00200000:0.0:1787739066.060156:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739066.060193:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1add56 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095217 expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.060209:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1add5d lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095217 expref: 17 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.060218:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xcd:0x0].0x0 (ffffa07305ca2e00) 00010000:00010000:0.0:1787739066.060221:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739066.060224:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609be00, rc 0) 00000100:00100000:0.0:1787739066.060291:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157ea940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17672:x1874579652503424:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 341us (579us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739066.060298:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23574 00000100:00100000:2.0:1787739066.066107:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739066.066190:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652503552 00000100:00100000:3.0:1787739066.066200:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652503552 00000100:00100000:3.0:1787739066.066271:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23575 00000100:00100000:3.0:1787739066.066281:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157e9a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17672:x1874579652503552:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:3.0:1787739066.066315:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739066.066334:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1add64 lrc: 2/0,0 mode: --/CR res: [0x240000401:0xcd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09521e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.066404:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1add6b lrc: 3/1,0 mode: --/PR res: [0x240000401:0xcd:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.066458:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1add33 lrc: 2/0,0 mode: PW/PW res: [0x240000401:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1add2c expref: 12 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.066469:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1add6b lrc: 4/1,0 mode: --/PR res: [0x240000401:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.066502:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1add33 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1add2c expref: 12 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739066.066517:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f76940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665724032:0@lo:104: 00000100:00100000:3.0:1787739066.066566:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739066.066849:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665724032 00000100:00100000:2.0:1787739066.066855:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665724032 00000100:00100000:2.0:1787739066.066894:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1194 00000100:00100000:2.0:1787739066.066901:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6523:x1874579665724032:12345-0@lo:104: 00010000:00010000:2.0:1787739066.066913:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a800/0x7ee4453e4a1add2c lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1add33 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.070336:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739066.070353:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6523:x1874579665724032:12345-0@lo:104: Request processed in 3448us (3781us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739066.070374:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1194 00010000:00010000:1.0:1787739066.073553:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739066.073568:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a800/0x7ee4453e4a1add2c lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1add33 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.073581:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739066.073588:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a800/0x7ee4453e4a1add2c lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xcd:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1add33 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739066.073602:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a800/0x7ee4453e4a1add2c lrc: 4/0,0 mode: PW/PW res: [0x240000401:0xcd:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1add33 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.073647:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a800/0x7ee4453e4a1add2c lrc: 2/0,0 mode: --/PW res: [0x240000401:0xcd:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1add33 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.073654:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739066.073683:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a800/0x7ee4453e4a1add2c lrc: 1/0,0 mode: --/PW res: [0x240000401:0xcd:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1add33 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.073690:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730609a800/0x7ee4453e4a1add2c lrc: 0/0,0 mode: --/PW res: [0x240000401:0xcd:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1add33 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.073709:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739066.073711:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739066.073713:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739066.073721:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787739066.073742:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304244f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665724160:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739066.073817:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739066.073882:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665724160 00000100:00100000:1.0:1787739066.073889:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665724160 00010000:00010000:1.0:1787739066.073919:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1add33 lrc: 5/0,0 mode: PW/PW res: [0x240000401:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1add2c expref: 13 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739066.073950:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1562 00000100:00100000:1.0:1787739066.073959:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304245680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+13:3651:x1874579665724160:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739066.073973:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1787739066.075380:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739066.075462:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f76940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665724032:0@lo:104: 00010000:00010000:2.0:1787739066.075486:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1add6b lrc: 3/1,0 mode: --/PR res: [0x240000401:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.080172:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1adca0 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1adca7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739066.080195:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1adca0 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1adca7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.080246:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1adca0 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adca7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.080254:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1787739066.080330:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304244780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665724288:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:2.0:1787739066.080409:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787739066.080502:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730633d800/0x7ee4453e4a1adca0 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adca7 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787739066.080523:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968423 is committed 00000001:00080000:1.0:1787739066.080538:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968422 is committed 00000020:00080000:1.0:1787739066.080546:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 377 batchid updated 00000020:00080000:1.0:1787739066.080565:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07441aca9c0 refcount 2 committed 0 result 0 batchid 378 00000020:00080000:1.0:1787739066.080569:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2f00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07441aca480 00000020:00080000:1.0:1787739066.080573:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].370 00000020:00080000:1.0:1787739066.080575:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2000 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07305407e00 00000020:00080000:1.0:1787739066.080578:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].370 00010000:00010000:1.0:1787739066.080602:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fdc00/0x7ee4453e4a1adc30 lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x184:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.080610:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fdc00/0x7ee4453e4a1adc30 lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x184:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.080617:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fdc00/0x7ee4453e4a1adc30 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x184:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.080627:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fdc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739066.080631:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fdc00/0x7ee4453e4a1adc30 lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x184:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.080638:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fdc00/0x7ee4453e4a1adc30 lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x184:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.080651:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x184:0x0].0x0 (ffffa07305eaef00) 00010000:00010000:1.0:1787739066.080654:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739066.080658:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fdc00/0x7ee4453e4a1adc30 lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x184:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.080665:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fdc00/0x7ee4453e4a1adc30 lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x184:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.080679:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc000/0x7ee4453e4a1adc37 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.080685:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc000/0x7ee4453e4a1adc37 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.080689:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc000/0x7ee4453e4a1adc37 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.080694:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fc000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739066.080697:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc000/0x7ee4453e4a1adc37 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.080702:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc000/0x7ee4453e4a1adc37 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.080712:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xca:0x0].0x0 (ffffa07305eae000) 00010000:00010000:1.0:1787739066.080714:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739066.080717:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc000/0x7ee4453e4a1adc37 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.080722:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc000/0x7ee4453e4a1adc37 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000020:00080000:1.0:1787739066.080744:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 378 committed batchid 377 00000004:00080000:1.0:1787739066.080855:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07307054000 with version 708 00000020:00080000:1.0:1787739066.080868:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 376 cancel update log [0x200000401:0x3:0x0].369: rc = 0 00000020:00080000:1.0:1787739066.080912:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 376 cancel update log [0x240000400:0x3:0x0].369: rc = 0 00000020:00080000:1.0:1787739066.080929:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 377 00000004:00080000:1.0:1787739066.080939:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 708 rpc_version 708 00000020:00100000:1.0:1787739066.080962:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739066.080972:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304244b40 x1874579665724544/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739066.081062:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304244b40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665724544:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739066.081105:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739066.081126:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07307054e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739066.081143:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665724544 00000100:00100000:1.0:1787739066.081150:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665724544 00000100:00100000:1.0:1787739066.081190:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3356 00000100:00100000:1.0:1787739066.081200:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304246d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665724544:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739066.081316:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304246d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665724544:12345-0@lo:1000:osp_up0-1.0 Request processed in 117us (209us total) trans 4294983430 rc 0/0 00000100:00100000:1.0:1787739066.081323:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3356 00000100:00100000:1.0:1787739066.081370:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304244b40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665724544:0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1787739066.081442:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1add33 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1add2c expref: 6 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.081454:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xcd:0x0].0x0 (ffffa07305ca2100) 00010000:00010000:1.0:1787739066.081459:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1add6b lrc: 3/1,0 mode: --/PR res: [0x240000401:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.081467:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1add6b lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.081474:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1add6b lrc: 4/1,0 mode: PR/PR res: [0x240000401:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.081492:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739066.084088:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304245680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665724160:12345-0@lo:103:ldlm_bl.0 Request processed in 10119us (10260us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739066.084104:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1562 00000100:00100000:0.0:1787739066.084193:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665724288 00000100:00100000:0.0:1787739066.084202:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665724288 00000100:00100000:0.0:1787739066.084255:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1360 00000100:00100000:0.0:1787739066.084263:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157eb0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+8:3651:x1874579665724288:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:0.0:1787739066.084277:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739066.084302:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1adca7 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x1903f68f bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1adca0 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.084318:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x1903f68f (ffffa07403a6f100) 00010000:00010000:0.0:1787739066.084327:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739066.084382:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157eb0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+7:3651:x1874579665724288:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 119us (3973us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739066.084389:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1360 00010000:00010000:3.0:1787739066.085502:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1add6b lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.085516:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1add6b lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739066.085575:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xcd:0x0]: returning size 4096 00000004:00200000:3.0:1787739066.085581:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739066.086433:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1add64 lrc: 0/0,0 mode: --/CR res: [0x240000401:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09521e expref: 9 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.086458:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1add6b lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09521e expref: 8 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.086472:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xcd:0x0].0x0 (ffffa07305ca2100) 00010000:00010000:3.0:1787739066.086476:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739066.086480:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa400, rc 0) 00000100:00100000:3.0:1787739066.086555:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157e9a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17672:x1874579652503552:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 20274us (20457us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739066.086564:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23575 00000100:00100000:2.0:1787739066.090296:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304244f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665724160:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739066.090427:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304244780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665724288:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:2.0:1787739066.099117:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739066.099217:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652503808 00000100:00100000:3.0:1787739066.099226:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652503808 00000100:00100000:3.0:1787739066.099272:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23576 00000100:00100000:3.0:1787739066.099281:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157e92c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17672:x1874579652503808:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:3.0:1787739066.099331:0:6525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739066.099359:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1add4f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1528:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095210 expref: 17 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.099377:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1528:0x0].0x0 (ffffa07305ca2600) 00010000:00010000:3.0:1787739066.099382:0:6525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739066.099405:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1add72 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.099432:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1add72 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.099441:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1add72 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.099452:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1add79 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.099464:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1add79 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.099469:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1add79 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.099599:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1add80 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1528:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.099606:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1add80 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1528:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.099611:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1add80 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1528:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.099625:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1add87 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xcd:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.099631:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1add5d lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095217 expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.099637:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1add87 lrc: 4/0,1 mode: --/EX res: [0x200000400:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.099660:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1add5d lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095217 expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.099670:0:6525:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1add5d lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095217 expref: 16 pid: 6524 timeout: 753 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739066.099689:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f770c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6525:1874579665724672:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739066.099766:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ea80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739066.103715:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f770c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:6525:1874579665724672:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739066.103746:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1add87 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739066.104293:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739066.104620:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652503936 00000100:00100000:3.0:1787739066.104628:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652503936 00010000:00010000:3.0:1787739066.104670:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1add5d lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095217 expref: 17 pid: 6524 timeout: 753 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739066.104712:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1563 00000100:00100000:3.0:1787739066.104721:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157ebc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652503936:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739066.104734:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739066.104741:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1add5d lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095217 expref: 17 pid: 6524 timeout: 753 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.104753:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1add5d lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095217 expref: 17 pid: 6524 timeout: 753 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.104785:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1add5d lrc: 0/0,0 mode: --/PR res: [0x200000400:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095217 expref: 17 pid: 6524 timeout: 753 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.104796:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xcd:0x0].0x0 (ffffa07305ca2e00) 00010000:00010000:3.0:1787739066.104800:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1add87 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.104810:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1add87 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.104817:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1add87 lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.104844:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739066.104901:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157ebc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:2409:x1874579652503936:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 180us (610us total) trans 0 rc 0/0 00010000:00010000:2.0:1787739066.104932:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1add87 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.104940:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1add87 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.104985:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1add8e lrc: 3/0,1 mode: --/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.105010:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1add8e lrc: 3/0,1 mode: --/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.105019:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1add8e lrc: 3/0,1 mode: --/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739066.105038:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41fc00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665724800:0@lo:101:mdt00_002.0 00000100:00100000:2.0:1787739066.105086:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739066.105115:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074332da840 going to sleep for 11 seconds 00000100:00100000:3.0:1787739066.105249:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1563 00000100:00100000:0.0:1787739066.105406:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665724800 00000100:00100000:0.0:1787739066.105442:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665724800 00000100:00100000:0.0:1787739066.105481:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3357 00000100:00100000:0.0:1787739066.105488:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6525:x1874579665724800:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787739066.105503:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739066.105516:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1add95 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1add8e expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.105536:0:6529:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1add6b lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09521e expref: 7 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.105543:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1add95 lrc: 4/0,0 mode: --/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1add8e expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.105565:0:6529:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1add6b lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09521e expref: 7 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.105577:0:6529:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1add6b lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09521e expref: 7 pid: 6523 timeout: 753 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739066.105588:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302528b40 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665724928:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739066.105639:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af600 going to sleep for 11 seconds 00000100:00100000:0.0:1787739066.111827:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739066.111876:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652504064 00000100:00100000:3.0:1787739066.111886:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652504064 00010000:00010000:3.0:1787739066.111930:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1add6b lrc: 5/0,0 mode: PR/PR res: [0x240000401:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09521e expref: 8 pid: 6523 timeout: 753 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739066.111991:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1564 00000100:00100000:3.0:1787739066.112021:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba4c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2408:x1874579652504064:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739066.112037:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739066.112044:0:6510:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1add6b lrc: 5/0,0 mode: PR/PR res: [0x240000401:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09521e expref: 8 pid: 6523 timeout: 753 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.112058:0:6510:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1add6b lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09521e expref: 8 pid: 6523 timeout: 753 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.112080:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xcd:0x0].0x0 (ffffa07305ca2100) 00010000:00010000:3.0:1787739066.112087:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1add95 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1add8e expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.112103:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1add95 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1add8e expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.112112:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1add95 lrc: 4/0,0 mode: EX/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1add8e expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.112152:0:6510:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1add95 lrc: 4/0,0 mode: EX/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1add8e expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739066.112167:0:6510:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f770c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6510:1874579665725056:0@lo:105: 00010000:00010000:1.0:1787739066.115074:0:6529:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1add6b lrc: 0/0,0 mode: --/PR res: [0x240000401:0xcd:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09521e expref: 8 pid: 6523 timeout: 753 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739066.115102:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302528b40 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665724928:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739066.115284:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1add95 lrc: 5/0,0 mode: EX/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1add8e expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.115296:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xcd:0x0].0x0 (ffffa07305ca2100) 00010000:00010000:1.0:1787739066.115300:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739066.115304:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b200, rc 0) 00000100:00100000:1.0:1787739066.115376:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6525:x1874579665724800:12345-0@lo:101:mdt00_002.0 Request processed in 9887us (10289us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739066.115385:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3357 00000100:00100000:3.0:1787739066.116166:0:6510:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739066.116274:0:6510:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ea80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739066.116296:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665725056 00000100:00100000:3.0:1787739066.116304:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665725056 00000100:00100000:3.0:1787739066.116351:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1195 00000100:00100000:3.0:1787739066.116360:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f76940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6510:x1874579665725056:12345-0@lo:105: 00010000:00010000:3.0:1787739066.116372:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1add8e lrc: 4/0,1 mode: --/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1add95 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.116383:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1add8e lrc: 4/0,1 mode: --/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1add95 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.116394:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1add8e lrc: 4/0,1 mode: --/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1add95 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.116403:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1add8e lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1add95 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.116428:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1add8e lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1add95 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.116438:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1add8e lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1add95 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.116455:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633da00) 00000100:00100000:2.0:1787739066.116465:0:6525:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968423 gen 2 00000004:00080000:2.0:1787739066.116473:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968415 committed transno 4294968423 00000020:00080000:2.0:1787739066.116480:0:6525:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa0743c0efb40 refcount 2 committed 0 result 0 batchid 388 00000020:00080000:2.0:1787739066.116484:0:6525:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa074356c7400 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa074154fce40 00000020:00080000:2.0:1787739066.116488:0:6525:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].385 00000020:00080000:2.0:1787739066.116491:0:6525:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2d00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073069bf600 00000020:00080000:2.0:1787739066.116494:0:6525:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].385 00000100:00100000:3.0:1787739066.116500:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f76940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6510:x1874579665725056:12345-0@lo:105: Request processed in 139us (335us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739066.116508:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1195 00000004:00080000:2.0:1787739066.116525:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968417 committed transno 4294968423 00000004:00080000:2.0:1787739066.116540:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968418 committed transno 4294968423 00000004:00080000:2.0:1787739066.116552:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968419 committed transno 4294968423 00000100:00100000:3.0:1787739066.116556:0:6510:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f770c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6510:1874579665725056:0@lo:105: 00000004:00080000:2.0:1787739066.116559:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968420 committed transno 4294968423 00010000:00010000:3.0:1787739066.116572:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000004:00080000:2.0:1787739066.116574:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968421 committed transno 4294968423 00000004:00080000:2.0:1787739066.116584:0:6525:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968423 committed transno 4294968423 00000100:00100000:2.0:1787739066.116596:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41fc00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665724800:0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787739066.116612:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1add8e lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1add95 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739066.116613:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba4c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652504064:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 4600us (4794us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739066.116629:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1564 00000020:00080000:3.0:1787739066.117585:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 388 committed batchid 387 00000020:00080000:3.0:1787739066.117649:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 387 00000100:00100000:2.0:1787739066.120713:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be7840 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665725184:0@lo:403:mdt00_002.0 00000100:00100000:2.0:1787739066.120800:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739066.120836:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074332da840 going to sleep for 11 seconds 00000100:00100000:0.0:1787739066.120919:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665725184 00000100:00100000:0.0:1787739066.120927:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665725184 00000100:00100000:0.0:1787739066.121151:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3358 00000100:00100000:0.0:1787739066.121165:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6525:x1874579665725184:12345-0@lo:403:mdt00_002.0 00000100:00100000:0.0:1787739066.122903:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6525:x1874579665725184:12345-0@lo:403:mdt00_002.0 Request processed in 1735us (2095us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739066.123014:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be7840 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:6525:1874579665725184:0@lo:403:mdt00_002.0 00000040:00080000:3.0:1787739066.123950:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].388, 664 off592656 00000040:00080000:3.0:1787739066.125716:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].387, 744 off641840 00000004:00080000:3.0:1787739066.125731:0:6525:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0730610e0c0 with version 738 00000004:00080000:3.0:1787739066.125762:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 738 rpc_version 738 00000020:00100000:3.0:1787739066.125798:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739066.125809:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303f743c0 x1874579665725440/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1704/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739066.125838:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f743c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665725440:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739066.125906:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739066.125946:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e000 going to sleep for 11 seconds 00000100:00100000:3.0:1787739066.125972:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665725440 00000100:00100000:3.0:1787739066.125980:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665725440 00000100:00100000:0.0:1787739066.126262:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3358 00000100:00100000:3.0:1787739066.126366:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3359 00000100:00100000:3.0:1787739066.126377:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f770c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665725440:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739066.128491:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f770c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665725440:12345-0@lo:1000:osp_up1-0.0 Request processed in 2110us (2577us total) trans 4294968424 rc 0/0 00000100:00100000:3.0:1787739066.128503:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3359 00000100:00100000:1.0:1787739066.130133:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f743c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665725440:0@lo:1000:osp_up1-0.0 00000004:00200000:2.0:1787739066.132054:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1528:0x0]: returning size 4096 00010000:00010000:2.0:1787739066.132084:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1add87 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132095:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1add87 lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132103:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1add87 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132109:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739066.132113:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1add87 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132120:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1add87 lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132136:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xcd:0x0].0x0 (ffffa07305ca2e00) 00010000:00010000:2.0:1787739066.132139:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739066.132143:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1add87 lrc: 2/0,0 mode: --/EX res: [0x200000400:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132148:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1add87 lrc: 0/0,0 mode: --/EX res: [0x200000400:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132164:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1add8e lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1add95 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132177:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1add8e lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1add95 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132184:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1add8e lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1add95 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132219:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739066.132229:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xcd:0x0] 00010000:00010000:2.0:1787739066.132237:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1add80 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1528:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132243:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1add80 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1528:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132248:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1add80 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1528:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132253:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739066.132255:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1add80 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1528:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132261:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1add80 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1528:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132268:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1528:0x0].0x0 (ffffa07305ca2600) 00010000:00010000:2.0:1787739066.132271:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739066.132274:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1add80 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1528:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132279:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1add80 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1528:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132297:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1add72 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132303:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1add72 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132308:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1add72 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132312:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1add72 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132317:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aae00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739066.132320:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1add72 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132325:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1add72 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132333:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2e00) 00010000:00010000:2.0:1787739066.132335:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739066.132338:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1add72 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132343:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1add72 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739066.132359:0:6525:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002f575f66 reply state 00000000fd61b159 transno 4294983431 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1add79 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132371:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1add79 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.132378:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xa20ac6a2 (ffffa07305ca2800) 00010000:00010000:2.0:1787739066.132380:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739066.132383:0:6525:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787739066.132496:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157e92c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17672:x1874579652503808:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 33213us (33387us total) trans 4294983431 rc 0/0 00000100:00100000:2.0:1787739066.132506:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23576 00010000:00010000:1.0:1787739066.133203:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739066.133212:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1add8e lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1add95 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.133221:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633da00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739066.133225:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1add8e lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1add95 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739066.133234:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1add8e lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1add95 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.133270:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1add8e lrc: 2/0,0 mode: --/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1add95 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.133277:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739066.133312:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1add8e lrc: 1/0,0 mode: --/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1add95 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.133320:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633da00/0x7ee4453e4a1add8e lrc: 0/0,0 mode: --/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1add95 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.133338:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739066.133340:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739066.133341:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739066.133350:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787739066.133373:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304245a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665725568:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739066.133450:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739066.133498:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665725568 00000100:00100000:1.0:1787739066.133505:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665725568 00000100:00100000:1.0:1787739066.133550:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1361 00000100:00100000:1.0:1787739066.133559:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304247480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+7:3653:x1874579665725568:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739066.133570:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739066.133588:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1add95 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xcd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1add8e expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.133599:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xcd:0x0].0x0 (ffffa07305ca2100) 00010000:00010000:1.0:1787739066.133606:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739066.133640:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304247480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+6:3653:x1874579665725568:12345-0@lo:103:ldlm_bl.0 Request processed in 82us (190us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739066.133647:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1361 00000100:00100000:1.0:1787739066.133688:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304245a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665725568:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739066.170642:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739066.177356:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652504192 00000100:00100000:3.0:1787739066.177372:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652504192 00000100:00100000:0.0:1787739066.177605:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23577 00000100:00100000:0.0:1787739066.177616:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073025283c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17673:x1874579652504192:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739066.177639:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739066.177661:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1add9c lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095225 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.177720:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1adda3 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.177734:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1adda3 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.177745:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1adda3 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.177762:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1addaa lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb73071de bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.177769:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1addaa lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.177775:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1addaa lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.177813:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1adda3 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.177819:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1adda3 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.177826:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1adda3 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.177832:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1adda3 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.177839:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ba00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739066.177843:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1adda3 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.177850:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1adda3 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.177862:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305ca2c00) 00010000:00010000:0.0:1787739066.177865:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739066.177869:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1adda3 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.177875:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1adda3 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.177888:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1addaa lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.177894:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1addaa lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.177900:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1addaa lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.177905:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609aa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739066.177908:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1addaa lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.177913:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1addaa lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.177919:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xb73071de (ffffa07305ca2a00) 00010000:00010000:0.0:1787739066.177922:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739066.177924:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1addaa lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.177929:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1addaa lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.177954:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1add9c lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095225 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.178122:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305ca2c00) 00010000:00010000:0.0:1787739066.178125:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739066.178129:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1add9c lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095225 expref: 18 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.178142:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609bc00, rc 0) 00000100:00100000:0.0:1787739066.178235:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073025283c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17673:x1874579652504192:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 620us (7605us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739066.178243:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23577 00000100:00100000:3.0:1787739066.187310:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739066.187382:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652504320 00000100:00100000:0.0:1787739066.187391:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652504320 00000100:00100000:0.0:1787739066.187477:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23578 00000100:00100000:0.0:1787739066.187486:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f75a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17673:x1874579652504320:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739066.187504:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739066.187526:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1addb1 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09522c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.187587:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1addb8 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.187601:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1addb8 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.187612:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1addb8 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.187623:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1addbf lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb73071de bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.187630:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1addbf lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.187635:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1addbf lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.187681:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1addb8 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.187687:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1addb8 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.187693:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1addb8 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.187700:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1addb8 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.187706:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609aa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739066.187710:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1addb8 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.187717:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1addb8 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.187729:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305ca2c00) 00010000:00010000:0.0:1787739066.187732:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739066.187737:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1addb8 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.187743:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609aa00/0x7ee4453e4a1addb8 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.187757:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1addbf lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.187762:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1addbf lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.187766:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1addbf lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.187770:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ba00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739066.187772:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1addbf lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.187777:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1addbf lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.187783:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xb73071de (ffffa07305ca2a00) 00010000:00010000:0.0:1787739066.187785:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739066.187787:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1addbf lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.187793:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1addbf lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.187821:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1addb1 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09522c expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.187831:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305ca2c00) 00010000:00010000:0.0:1787739066.187833:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739066.187837:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609bc00/0x7ee4453e4a1addb1 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09522c expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.187847:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609bc00, rc 0) 00000100:00100000:0.0:1787739066.187904:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f75a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17673:x1874579652504320:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 420us (602us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739066.187912:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23578 00000100:00100000:3.0:1787739066.193492:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739066.193604:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652504448 00000100:00100000:0.0:1787739066.193614:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652504448 00000100:00100000:0.0:1787739066.193680:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23579 00000100:00100000:0.0:1787739066.193689:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17673:x1874579652504448:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739066.193710:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739066.193732:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1addc6 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095233 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.193915:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1addcd lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.193928:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1addcd lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.193939:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1addcd lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739066.194062:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:0.0:1787739066.194068:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739066.194106:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b200/0x7ee4453e4a1addc6 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095233 expref: 19 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.194124:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1addcd lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095233 expref: 18 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.194133:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305ca2d00) 00010000:00010000:0.0:1787739066.194136:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739066.194139:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609be00, rc 0) 00000100:00100000:0.0:1787739066.194279:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17673:x1874579652504448:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 590us (796us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739066.194287:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23579 00000100:00100000:3.0:1787739066.209020:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739066.209092:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652504704 00000100:00100000:0.0:1787739066.209102:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652504704 00000100:00100000:0.0:1787739066.209164:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23580 00000100:00100000:0.0:1787739066.209174:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157eb0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17673:x1874579652504704:12345-192.168.203.15@tcp:36:lfs.0 00000100:00100000:0.0:1787739066.209285:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665725696:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739066.209333:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739066.209375:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052afd80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739066.209395:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665725696 00000100:00100000:0.0:1787739066.209399:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665725696 00000100:00100000:0.0:1787739066.209452:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3360 00000100:00100000:0.0:1787739066.209459:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665725696:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739066.209523:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665725696:12345-0@lo:1000:mdt00_004.0 Request processed in 64us (188us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739066.209530:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3360 00000100:00100000:0.0:1787739066.209574:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665725696:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739066.209614:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665725824:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739066.209635:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739066.209653:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052afd80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739066.209710:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665725824 00000100:00100000:3.0:1787739066.209714:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665725824 00000100:00100000:3.0:1787739066.209746:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3361 00000100:00100000:3.0:1787739066.209752:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665725824:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739066.209828:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665725824:12345-0@lo:1000:mdt00_004.0 Request processed in 76us (192us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739066.209835:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3361 00000100:00100000:0.0:1787739066.210215:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665725824:0@lo:1000:mdt00_004.0 00010000:00010000:0.0:1787739066.210271:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1addd4 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.210280:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1addd4 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.210290:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1addd4 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739066.210303:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665725952:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739066.210332:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739066.210356:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af240 going to sleep for 11 seconds 00000100:00100000:1.0:1787739066.210402:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665725952 00000100:00100000:1.0:1787739066.210408:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665725952 00000100:00100000:1.0:1787739066.214123:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3362 00000100:00100000:1.0:1787739066.214138:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:13793:x1874579665725952:12345-0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739066.214159:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739066.214181:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc000/0x7ee4453e4a1adddb lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1addd4 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.214214:0:12410:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1addcd lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095233 expref: 17 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.214225:0:12410:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc000/0x7ee4453e4a1adddb lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1addd4 expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.214261:0:12410:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1addcd lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095233 expref: 17 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.214274:0:12410:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1addcd lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095233 expref: 17 pid: 6525 timeout: 753 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739066.214288:0:12410:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304245a40 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:12410:1874579665726080:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739066.214382:0:12410:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4840 going to sleep for 11 seconds 00000100:00100000:2.0:1787739066.219271:0:12410:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304245a40 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:12410:1874579665726080:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739066.219305:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc000/0x7ee4453e4a1adddb lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1addd4 expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.219318:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305ca2d00) 00010000:00010000:2.0:1787739066.219321:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739066.219325:0:12410:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc000/0x7ee4453e4a1adddb lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1addd4 expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739066.219335:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fc000, rc 0) 00000100:00100000:2.0:1787739066.219393:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665725952:12345-0@lo:101:mdt00_004.0 Request processed in 5258us (9060us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739066.219403:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3362 00000100:00100000:0.0:1787739066.219508:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665725952:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739066.219526:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1addd4 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adddb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.219537:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1addd4 lrc: 4/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adddb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739066.225025:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739066.226532:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652504832 00000100:00100000:0.0:1787739066.226546:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652504832 00010000:00010000:0.0:1787739066.226591:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1addcd lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095233 expref: 18 pid: 6525 timeout: 753 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739066.226649:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1565 00000100:00100000:0.0:1787739066.226661:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157ebc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:2409:x1874579652504832:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739066.226678:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739066.226686:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1addcd lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095233 expref: 18 pid: 6525 timeout: 753 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.226699:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1addcd lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095233 expref: 18 pid: 6525 timeout: 753 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.226722:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1addcd lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095233 expref: 18 pid: 6525 timeout: 753 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.226736:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305ca2d00) 00010000:00010000:0.0:1787739066.226742:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc000/0x7ee4453e4a1adddb lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1addd4 expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.226754:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc000/0x7ee4453e4a1adddb lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1addd4 expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.226916:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc000/0x7ee4453e4a1adddb lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1addd4 expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.226955:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc000/0x7ee4453e4a1adddb lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1addd4 expref: 7 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739066.226971:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252a580 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665726208:0@lo:105: 00000100:00100000:0.0:1787739066.227045:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739066.227088:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739066.227140:0:6507:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:3.0:1787739066.227151:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665726208 00000100:00100000:3.0:1787739066.227154:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665726208 00000100:00100000:3.0:1787739066.227189:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1196 00000100:00100000:3.0:1787739066.227197:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f85c850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665726208:12345-0@lo:105: 00010000:00010000:3.0:1787739066.227208:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1addd4 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adddb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.227218:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1addd4 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adddb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.227227:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1addd4 lrc: 5/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adddb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.227234:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1addd4 lrc: 5/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1adddb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.227242:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1addd4 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1adddb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.227250:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1addd4 lrc: 6/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1adddb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.227265:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730609bc00) 00000100:00100000:3.0:1787739066.227309:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f85c850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:9525:x1874579665726208:12345-0@lo:105: Request processed in 113us (264us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739066.227316:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1196 00010000:00010000:0.0:1787739066.227390:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1addd4 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adddb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.227399:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1addd4 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adddb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739066.227452:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739066.227481:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1adde2 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xb73071de bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.227488:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1adde2 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.227498:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1adde2 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739066.227511:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665726336:0@lo:101:mdt00_004.0 00000100:00100000:0.0:1787739066.227536:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739066.227553:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052afd80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739066.227578:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252a580 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:9525:1874579665726208:0@lo:105: 00010000:00010000:0.0:1787739066.227589:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739066.227623:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157ebc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2409:x1874579652504832:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 965us (2604us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739066.227630:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1565 00000100:00100000:3.0:1787739066.227724:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665726336 00000100:00100000:3.0:1787739066.227728:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665726336 00000100:00100000:3.0:1787739066.227752:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3363 00000100:00100000:3.0:1787739066.227758:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:13793:x1874579665726336:12345-0@lo:101:mdt00_004.0 00010000:00010000:3.0:1787739066.227781:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739066.227795:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1adde9 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xb73071de bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1adde2 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.227814:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1adde9 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1adde2 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.227826:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1adde9 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adde2 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739066.227834:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xb73071de (ffffa074331c2800) 00010000:00010000:3.0:1787739066.227837:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739066.227839:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab800, rc 0) 00000100:00100000:3.0:1787739066.227870:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665726336:12345-0@lo:101:mdt00_004.0 Request processed in 112us (333us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739066.227875:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3363 00000100:00100000:0.0:1787739066.228920:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665726336:0@lo:101:mdt00_004.0 00010000:00010000:0.0:1787739066.228940:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1adde2 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adde9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.228952:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1adde2 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adde9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739066.228969:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:0.0:1787739066.229018:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665726464:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739066.229055:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739066.229082:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739066.229408:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665726464 00000100:00100000:3.0:1787739066.229448:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665726464 00000100:00100000:3.0:1787739066.229489:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3364 00000100:00100000:3.0:1787739066.229497:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665726464:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739066.229568:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665726464:12345-0@lo:1000:mdt00_004.0 Request processed in 72us (512us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739066.229574:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3364 00000100:00100000:0.0:1787739066.229720:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665726464:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739066.229759:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665726592:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739066.229839:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739066.229888:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739066.234361:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665726592 00000100:00100000:3.0:1787739066.234374:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665726592 00000100:00100000:3.0:1787739066.234439:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3365 00000100:00100000:3.0:1787739066.234449:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665726592:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739066.234579:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665726592:12345-0@lo:1000:mdt00_004.0 Request processed in 132us (4739us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739066.234590:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3365 00000100:00100000:0.0:1787739066.234686:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665726592:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739066.234791:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665726720:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739066.234823:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739066.234850:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739066.235004:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665726720 00000100:00100000:3.0:1787739066.235009:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665726720 00000100:00100000:3.0:1787739066.235043:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3366 00000100:00100000:3.0:1787739066.235051:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665726720:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739066.235106:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665726720:12345-0@lo:1000:mdt00_004.0 Request processed in 56us (282us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739066.235113:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3366 00000100:00100000:0.0:1787739066.235318:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665726720:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739066.235361:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665726848:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739066.235384:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739066.235405:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739066.235446:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665726848 00000100:00100000:0.0:1787739066.235450:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665726848 00000100:00100000:0.0:1787739066.235471:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3367 00000100:00100000:0.0:1787739066.235477:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665726848:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739066.235521:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665726848:12345-0@lo:1000:mdt00_004.0 Request processed in 44us (136us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739066.235527:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3367 00000100:00100000:0.0:1787739066.235551:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665726848:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739066.235585:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665726976:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739066.235606:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739066.235618:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739066.235628:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665726976 00000100:00100000:0.0:1787739066.235630:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665726976 00000100:00100000:0.0:1787739066.235883:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3368 00000100:00100000:0.0:1787739066.235891:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665726976:12345-0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1787739066.235948:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665726976:12345-0@lo:1000:mdt00_004.0 Request processed in 58us (340us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739066.235954:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3368 00000100:00100000:1.0:1787739066.236017:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665726976:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739066.236058:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304245a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665727104:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739066.236085:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739066.240874:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739066.241020:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665727104 00000100:00100000:3.0:1787739066.241029:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665727104 00000100:00100000:3.0:1787739066.241072:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3369 00000100:00100000:3.0:1787739066.241080:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304246580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665727104:12345-0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1787739066.241201:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304246580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665727104:12345-0@lo:1000:mdt00_004.0 Request processed in 121us (5115us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739066.241208:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3369 00000100:00100000:1.0:1787739066.242914:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304245a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665727104:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739066.242988:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304245a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665727232:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739066.243050:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739066.243081:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068b4480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739066.243103:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665727232 00000100:00100000:1.0:1787739066.243110:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665727232 00000100:00100000:1.0:1787739066.243150:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3370 00000100:00100000:1.0:1787739066.243157:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304245680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665727232:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1787739066.243229:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304245680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:13793:x1874579665727232:12345-0@lo:1000:mdt00_004.0 Request processed in 72us (179us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739066.243236:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3370 00000100:00100000:1.0:1787739066.243265:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304245a40 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-mdtlov_UUID:13793:1874579665727232:0@lo:1000:mdt00_004.0 00080000:02000000:1.0:1787739066.243470:0:13793:0:(osd_handler.c:2218:osd_trans_start()) *** cfs_fail_loc=19a, val=0*** 00000004:00080000:1.0:1787739066.243486:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa073068b4b40 with version 0 00000020:00080000:1.0:1787739066.243492:0:13793:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa073068b4480 refcount 3 committed 0 result -5 batchid 380 00000020:00080000:1.0:1787739066.243495:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743328fb00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result -5 sub_th ffffa073068b4b40 00000020:00080000:1.0:1787739066.243498:0:13793:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743328ff00 obd lustre-MDT0001-osd committed 1 started 0 stopped 1 result 0 sub_th ffffa07303c9b000 00000004:00080000:1.0:1787739066.243520:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739066.243531:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1addd4 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adddb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.243542:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1addd4 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1adddb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.243550:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1addd4 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1adddb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.243567:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739066.243573:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739066.243577:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1adde2 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adde9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.243584:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1adde2 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1adde9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.243592:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1adde2 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1adde9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.243606:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1787739066.243665:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157eb0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17673:x1874579652504704:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 34490us (34654us total) trans 0 rc -5/-5 00000100:00100000:1.0:1787739066.243672:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23580 00000020:00080000:1.0:1787739066.243693:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 377 00010000:00010000:0.0:1787739066.243852:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787739066.243859:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1addd4 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1adddb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.243868:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609bc00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739066.243874:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1addd4 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1adddb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739066.243883:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1addd4 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1adddb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.243910:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1addd4 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adddb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.243916:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739066.243936:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1addd4 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adddb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.243944:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609bc00/0x7ee4453e4a1addd4 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adddb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.243967:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739066.243970:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1adde2 lrc: 2/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1adde9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.243976:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739066.243980:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1adde2 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1adde9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739066.243987:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1adde2 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1adde9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.244012:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1adde2 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adde9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.244019:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739066.244033:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1adde2 lrc: 1/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adde9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.244040:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1adde2 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1adde9 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.244052:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739066.244053:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739066.244055:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739066.244452:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665727488:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739066.244489:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739066.244718:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665727488 00000100:00100000:0.0:1787739066.244725:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665727488 00000100:00100000:1.0:1787739066.245748:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665727360:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739066.245858:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739066.245900:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665727360 00000100:00100000:1.0:1787739066.245906:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665727360 00010000:00010000:2.0:1787739066.246007:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739066.246074:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1362 00000100:00100000:0.0:1787739066.246089:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d3c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+10:3650:x1874579665727488:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739066.246103:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739066.246125:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab800/0x7ee4453e4a1adde9 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xb73071de bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1adde2 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739066.246140:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xb73071de (ffffa074331c2800) 00010000:00010000:0.0:1787739066.246147:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739066.246190:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d3c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+9:3650:x1874579665727488:12345-0@lo:103:ldlm_bl.0 Request processed in 105us (1700us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739066.246241:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665727488:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739066.246878:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1363 00000100:00100000:1.0:1787739066.246889:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304245a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3653:x1874579665727360:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739066.246899:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739066.246913:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc000/0x7ee4453e4a1adddb lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1addd4 expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739066.246927:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305ca2d00) 00010000:00010000:1.0:1787739066.246935:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739066.246969:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304245a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+8:3653:x1874579665727360:12345-0@lo:103:ldlm_bl.0 Request processed in 84us (1112us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739066.246976:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1363 00000100:00100000:1.0:1787739066.247027:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665727360:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739066.248717:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1362 00010000:00010000:0.0:1787739066.691823:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739066.818659:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665727616:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787739066.818731:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739066.818864:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665727616 00000100:00100000:0.0:1787739066.818874:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665727616 00000100:00100000:0.0:1787739066.818939:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 360 00000100:00100000:0.0:1787739066.818949:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be6d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665727616:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787739066.818978:0:8441:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 25431488 pending 0 free 3923496960 avail 3691974656 00000100:00100000:0.0:1787739066.819034:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be6d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665727616:12345-0@lo:13:osp-pre-1-0.0 Request processed in 86us (302us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739066.819042:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 360 00000100:00100000:2.0:1787739066.820058:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665727616:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787739066.820179:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3651:1874579665728128:0@lo:400:kworker.0 00000100:00100000:2.0:1787739066.820233:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739066.820255:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665728640:0@lo:400:kworker.0 00000100:00100000:2.0:1787739066.820284:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739066.820308:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:2.0:1787739066.820314:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3651:1874579665728000:0@lo:400:kworker.0 00000100:00100000:2.0:1787739066.820333:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739066.820348:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665728512:0@lo:400:kworker.0 00000100:00100000:2.0:1787739066.820366:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739066.820378:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665729024:0@lo:400:kworker.0 00000100:00100000:2.0:1787739066.820394:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739066.820439:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665728128 00000100:00100000:2.0:1787739066.820444:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665728128 00000100:00100000:2.0:1787739066.820476:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665728000 00000100:00100000:2.0:1787739066.820479:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665728000 00000100:00100000:2.0:1787739066.820494:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 660 00000100:00100000:2.0:1787739066.820500:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1874579665728128:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739066.820537:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3651:x1874579665728128:12345-0@lo:400:kworker.0 Request processed in 37us (304us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739066.820544:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 660 00000100:00100000:2.0:1787739066.820557:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 661 00000100:00100000:2.0:1787739066.820563:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1874579665728000:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739066.820588:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3651:x1874579665728000:12345-0@lo:400:kworker.0 Request processed in 26us (254us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739066.820594:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 661 00000100:00100000:2.0:1787739066.820611:0:14661:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665728640 00000100:00100000:2.0:1787739066.820614:0:14661:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665728640 00000100:00100000:2.0:1787739066.820640:0:14661:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665728512 00000100:00100000:2.0:1787739066.820643:0:14661:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665728512 00000100:00100000:2.0:1787739066.820659:0:14661:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665729024 00000100:00100000:2.0:1787739066.820661:0:14661:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665729024 00000100:00100000:2.0:1787739066.820677:0:14661:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 352 00000100:00100000:2.0:1787739066.820683:0:14661:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665728640:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739066.820709:0:14661:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665728640:12345-0@lo:400:kworker.0 Request processed in 27us (431us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739066.820714:0:14661:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 352 00000100:00100000:2.0:1787739066.820722:0:14661:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 353 00000100:00100000:2.0:1787739066.820727:0:14661:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665728512:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739066.820746:0:14661:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41de00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665728512:12345-0@lo:400:kworker.0 Request processed in 19us (379us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739066.820751:0:14661:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 353 00000100:00100000:2.0:1787739066.820759:0:14661:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 354 00000100:00100000:2.0:1787739066.820765:0:14661:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665729024:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739066.820785:0:14661:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41cf00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665729024:12345-0@lo:400:kworker.0 Request processed in 21us (391us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739066.820791:0:14661:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 354 00000100:00100000:2.0:1787739066.820823:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3651:1874579665728000:0@lo:400:kworker.0 00000100:00100000:2.0:1787739066.820832:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665728512:0@lo:400:kworker.0 00000100:00100000:2.0:1787739066.820839:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665729024:0@lo:400:kworker.0 00000100:00100000:2.0:1787739066.820848:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3651:1874579665728128:0@lo:400:kworker.0 00000100:00100000:2.0:1787739066.820856:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665728640:0@lo:400:kworker.0 00000100:00080000:1.0:1787739066.821128:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1787739066.821133:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1787739066.821145:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2:3653:1874579665727872:0@lo:400:kworker.0 00000100:00100000:1.0:1787739066.821180:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739066.821218:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3653:1874579665728384:0@lo:400:kworker.0 00000100:00100000:1.0:1787739066.821241:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739066.821259:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3653:1874579665728896:0@lo:400:kworker.0 00000100:00100000:1.0:1787739066.821285:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739066.821305:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:1.0:1787739066.821311:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665727744:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1787739066.821334:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739066.821353:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3653:1874579665728256:0@lo:400:kworker.0 00000100:00100000:1.0:1787739066.821374:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739066.821389:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3653:1874579665728768:0@lo:400:kworker.0 00000100:00100000:1.0:1787739066.821410:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739066.821458:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665728384 00000100:00100000:1.0:1787739066.821463:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665728384 00000100:00100000:1.0:1787739066.821491:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665728896 00000100:00100000:1.0:1787739066.821494:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665728896 00000100:00100000:1.0:1787739066.821504:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665728256 00000100:00100000:1.0:1787739066.821506:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665728256 00000100:00100000:1.0:1787739066.821515:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665728768 00000100:00100000:1.0:1787739066.821517:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665728768 00000100:00100000:1.0:1787739066.821528:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 662 00000100:00100000:1.0:1787739066.821533:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304244000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3653:x1874579665728384:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787739066.821569:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304244000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3653:x1874579665728384:12345-0@lo:400:kworker.0 Request processed in 35us (327us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739066.821575:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 662 00000100:00100000:1.0:1787739066.821588:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 663 00000100:00100000:1.0:1787739066.821593:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042461c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874579665728896:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787739066.821618:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042461c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874579665728896:12345-0@lo:400:kworker.0 Request processed in 25us (333us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739066.821624:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 663 00000100:00100000:1.0:1787739066.821636:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 664 00000100:00100000:1.0:1787739066.821642:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304244f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874579665728256:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787739066.821667:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304244f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874579665728256:12345-0@lo:400:kworker.0 Request processed in 26us (291us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739066.821672:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 664 00000100:00100000:1.0:1787739066.821682:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 665 00000100:00100000:1.0:1787739066.821687:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3653:x1874579665728768:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787739066.821710:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3653:x1874579665728768:12345-0@lo:400:kworker.0 Request processed in 23us (299us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739066.821715:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 665 00000100:00100000:1.0:1787739066.821742:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665727744 00000100:00100000:1.0:1787739066.821745:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665727744 00000100:00100000:1.0:1787739066.821767:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 361 00000100:00100000:1.0:1787739066.821774:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304244780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874579665727744:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1787739066.821787:0:8440:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 25431488 pending 0 free 3923496960 avail 3691974656 00000100:00100000:1.0:1787739066.821815:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304244780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874579665727744:12345-0@lo:13:osp-pre-1-1.0 Request processed in 41us (479us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739066.821820:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 361 00000100:00080000:1.0:1787739066.821837:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1787739066.821839:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1787739066.821865:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665727744:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1787739066.821875:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3653:1874579665728256:0@lo:400:kworker.0 00000100:00100000:1.0:1787739066.821882:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3653:1874579665728768:0@lo:400:kworker.0 00000100:00100000:1.0:1787739066.821889:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be7c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3653:1874579665728384:0@lo:400:kworker.0 00000100:00100000:1.0:1787739066.821896:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3653:1874579665728896:0@lo:400:kworker.0 00000100:00100000:2.0:1787739066.822286:0:6518:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665727872 00000100:00100000:2.0:1787739066.822292:0:6518:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665727872 00000100:00100000:2.0:1787739066.822333:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 450 00000100:00100000:2.0:1787739066.822340:0:6518:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0740ea10c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2+24:3653:x1874579665727872:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739066.822379:0:6518:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0740ea10c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2+24:3653:x1874579665727872:12345-0@lo:400:kworker.0 Request processed in 40us (1198us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739066.822386:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 450 00000100:00100000:1.0:1787739066.826002:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2:3653:1874579665727872:0@lo:400:kworker.0 00000100:00100000:3.0:1787739066.944365:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739066.944409:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739066.944463:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739066.944484:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739066.944509:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739066.944535:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652505472 00000100:00100000:3.0:1787739066.944543:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652505472 00000100:00100000:3.0:1787739066.944594:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652505344 00000100:00100000:3.0:1787739066.944597:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652505344 00000100:00100000:3.0:1787739066.944630:0:8436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 355 00000100:00100000:1.0:1787739066.944743:0:6517:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652504960 00000100:00100000:1.0:1787739066.944748:0:6517:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652504960 00000100:00100000:1.0:1787739066.944778:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 451 00000100:00100000:1.0:1787739066.944786:0:6517:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0740ea12c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:53427813-ebb7-48fc-94b6-ef03330d7ee5+10:2407:x1874579652504960:12345-192.168.203.15@tcp:400:kworker.0 00000100:00100000:1.0:1787739066.944859:0:6517:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0740ea12c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:53427813-ebb7-48fc-94b6-ef03330d7ee5+10:2407:x1874579652504960:12345-192.168.203.15@tcp:400:kworker.0 Request processed in 74us (502us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739066.944865:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 451 00000100:00100000:2.0:1787739066.945303:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652505216 00000100:00100000:2.0:1787739066.945309:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652505216 00000100:00100000:2.0:1787739066.945340:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652505088 00000100:00100000:2.0:1787739066.945343:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652505088 00000100:00100000:2.0:1787739066.945374:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 666 00000100:00100000:2.0:1787739066.945383:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2407:x1874579652505216:12345-192.168.203.15@tcp:400:kworker.0 00000100:00100000:2.0:1787739066.945446:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2407:x1874579652505216:12345-192.168.203.15@tcp:400:kworker.0 Request processed in 64us (1035us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739066.945453:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 666 00000100:00100000:2.0:1787739066.945467:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 667 00000100:00100000:2.0:1787739066.945474:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f76940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2407:x1874579652505088:12345-192.168.203.15@tcp:400:kworker.0 00000100:00100000:2.0:1787739066.945499:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f76940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:2407:x1874579652505088:12345-192.168.203.15@tcp:400:kworker.0 Request processed in 26us (1013us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739066.945505:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 667 00000100:00100000:3.0:1787739066.945878:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2407:x1874579652505472:12345-192.168.203.15@tcp:400:kworker.0 00000100:00100000:3.0:1787739066.945934:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2407:x1874579652505472:12345-192.168.203.15@tcp:400:kworker.0 Request processed in 1294us (1468us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739066.945942:0:8436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 355 00000100:00100000:3.0:1787739066.945960:0:8436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 356 00000100:00100000:3.0:1787739066.945969:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562f840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2407:x1874579652505344:12345-192.168.203.15@tcp:400:kworker.0 00000100:00100000:3.0:1787739066.946008:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562f840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2407:x1874579652505344:12345-192.168.203.15@tcp:400:kworker.0 Request processed in 41us (1498us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739066.946016:0:8436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 356 00000100:00080000:3.0:1787739066.946048:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1787739066.946053:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1787739067.719940:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739067.878151:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739067.878583:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652505728 00000100:00100000:3.0:1787739067.878592:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652505728 00000100:00100000:3.0:1787739067.878666:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23581 00000100:00100000:3.0:1787739067.878676:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306393c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17692:x1874579652505728:12345-192.168.203.15@tcp:36:mkdir.0 00010000:00010000:3.0:1787739067.878788:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1addf0 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739067.878803:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1addf0 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739067.878814:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1addf0 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739067.878827:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1addf7 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739067.878834:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1addf7 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739067.878846:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1addf7 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739067.879285:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1addfe lrc: 3/0,1 mode: --/PW res: [0x200000402:0x152a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739067.879295:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1addfe lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x152a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739067.879303:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1addfe lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x152a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739067.879318:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000db28e46d reply state 00000000ca99a7ca transno 4294983432 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1addfe lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x152a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739067.879329:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1addfe lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x152a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739067.879336:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152a:0x0].0x0 (ffffa074331c2000) 00010000:00010000:3.0:1787739067.879339:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739067.879342:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:3.0:1787739067.879347:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x152a:0x0]: returning size 4096 00010000:00010000:3.0:1787739067.879355:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1addf0 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739067.879366:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1addf0 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739067.879373:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1addf0 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739067.879380:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1addf0 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739067.879385:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739067.879390:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1addf0 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739067.879689:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1addf0 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739067.879701:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074331c2c00) 00010000:00010000:3.0:1787739067.879703:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739067.879707:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1addf0 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739067.879714:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1addf0 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739067.879731:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000db28e46d reply state 00000000ca99a7ca transno 4294983432 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1addf7 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739067.879740:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1addf7 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739067.879746:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd04803f8 (ffffa074331c2300) 00010000:00010000:3.0:1787739067.879748:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739067.879751:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000100:00100000:3.0:1787739067.879849:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306393c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+19:17692:x1874579652505728:12345-192.168.203.15@tcp:36:mkdir.0 Request processed in 1172us (1706us total) trans 4294983432 rc 0/0 00000100:00100000:3.0:1787739067.879857:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23581 00000100:00080000:1.0:1787739067.880154:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787739067.916700:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739067.920657:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652505856 00000100:00100000:1.0:1787739067.920672:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652505856 00000100:00100000:1.0:1787739067.920754:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23582 00000100:00100000:1.0:1787739067.920765:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+19:17693:x1874579652505856:12345-192.168.203.15@tcp:101:rmdir.0 00010000:00010000:1.0:1787739067.920788:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739067.920811:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1ade05 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09523a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.920872:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc600/0x7ee4453e4a1ade0c lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.920886:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc600/0x7ee4453e4a1ade0c lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.920896:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc600/0x7ee4453e4a1ade0c lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.920905:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade13 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.920912:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade13 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.920917:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade13 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.920998:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd800/0x7ee4453e4a1ade1a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x152a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.921006:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd800/0x7ee4453e4a1ade1a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.921013:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd800/0x7ee4453e4a1ade1a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739067.921050:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x152a:0x0]: returning size 4096 00000004:00200000:1.0:1787739067.921055:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1787739067.921074:0:13793:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd800/0x7ee4453e4a1ade1a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.921085:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc600/0x7ee4453e4a1ade0c lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.921092:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc600/0x7ee4453e4a1ade0c lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.921098:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc600/0x7ee4453e4a1ade0c lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.921104:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc600/0x7ee4453e4a1ade0c lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.921114:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fc600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739067.921118:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc600/0x7ee4453e4a1ade0c lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.921125:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc600/0x7ee4453e4a1ade0c lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.921136:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235900) 00010000:00010000:1.0:1787739067.921139:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739067.921143:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc600/0x7ee4453e4a1ade0c lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.921149:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc600/0x7ee4453e4a1ade0c lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.921161:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade13 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.921166:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade13 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.921171:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade13 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.921175:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fd400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739067.921178:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade13 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.921183:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade13 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.921192:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd04803f8 (ffffa074331c2300) 00010000:00010000:1.0:1787739067.921195:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739067.921197:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade13 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.921203:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade13 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.921224:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1ade05 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09523a expref: 21 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.921237:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd800/0x7ee4453e4a1ade1a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x152a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09523a expref: 20 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.921244:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152a:0x0].0x0 (ffffa074331c2000) 00010000:00010000:1.0:1787739067.921247:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1787739067.921249:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fd800, rc 0) 00000100:00100000:1.0:1787739067.922643:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+20:17693:x1874579652505856:12345-192.168.203.15@tcp:101:rmdir.0 Request processed in 1873us (5944us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739067.922655:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23582 00000100:00100000:3.0:1787739067.932811:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739067.932919:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652506112 00000100:00100000:1.0:1787739067.932929:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652506112 00000100:00100000:1.0:1787739067.932993:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23583 00000100:00100000:1.0:1787739067.933003:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+20:17693:x1874579652506112:12345-192.168.203.15@tcp:36:rmdir.0 00010000:00010000:1.0:1787739067.933057:0:7944:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739067.933083:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd800/0x7ee4453e4a1ade1a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x152a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09523a expref: 20 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933105:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152a:0x0].0x0 (ffffa074331c2000) 00010000:00010000:1.0:1787739067.933110:0:7944:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739067.933130:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1ade21 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933141:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1ade21 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933150:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1ade21 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933160:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade28 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933167:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade28 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933173:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade28 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933245:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc600/0x7ee4453e4a1ade2f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x152a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933253:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc600/0x7ee4453e4a1ade2f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x152a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933260:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc600/0x7ee4453e4a1ade2f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x152a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739067.933679:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x152a:0x0]: returning size 4096 00010000:00010000:1.0:1787739067.933696:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc600/0x7ee4453e4a1ade2f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x152a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933703:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc600/0x7ee4453e4a1ade2f lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x152a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933710:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc600/0x7ee4453e4a1ade2f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x152a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933716:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fc600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739067.933720:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc600/0x7ee4453e4a1ade2f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x152a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933728:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc600/0x7ee4453e4a1ade2f lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x152a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933737:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152a:0x0].0x0 (ffffa074331c2000) 00010000:00010000:1.0:1787739067.933740:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739067.933744:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc600/0x7ee4453e4a1ade2f lrc: 2/0,0 mode: --/EX res: [0x200000402:0x152a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933750:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc600/0x7ee4453e4a1ade2f lrc: 0/0,0 mode: --/EX res: [0x200000402:0x152a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933763:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1ade21 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933769:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1ade21 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933775:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1ade21 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933781:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1ade21 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933787:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fd600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739067.933790:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1ade21 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933795:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1ade21 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933802:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07433235900) 00010000:00010000:1.0:1787739067.933804:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739067.933806:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1ade21 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933811:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1ade21 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739067.933824:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001d06ff7c reply state 00000000abd7d0d2 transno 4294983433 ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade28 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933835:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade28 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739067.933841:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd04803f8 (ffffa074331c2300) 00010000:00010000:1.0:1787739067.933844:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739067.933846:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787739067.934593:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:17693:x1874579652506112:12345-192.168.203.15@tcp:36:rmdir.0 Request processed in 1587us (1786us total) trans 4294983433 rc 0/0 00000100:00100000:1.0:1787739067.934603:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23583 00000020:00000080:0.0:1787739068.709501:0:20579:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=c00866e6 len=0 uarg=00007ffee0ff5030 00000004:00000080:0.0:1787739068.709522:0:20579:0:(mdt_handler.c:8179:mdt_iocontrol()) lustre-MDT0000: cmd=c00866e6 len=608 karg=ffffa0743f85c800 uarg=0000000000000000 00000004:00000080:0.0:1787739068.709568:0:20579:0:(mdd_device.c:2304:mdd_iocontrol()) lustre-MDD0000: cmd=c00866e6 len=0 karg=ffffb3dac864bd98 00100000:10000000:0.0:1787739068.709574:0:20579:0:(lfsck_lib.c:3085:lfsck_start()) lustre-MDT0000-osd: start master thread 00000100:00100000:0.0:1787739068.709914:0:20579:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job lctl:lustre-MDT0000-mdtlov_UUID:20579:1874579665729152:0@lo:1101:lctl.0 00000100:00100000:0.0:1787739068.709995:0:20579:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.710105:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665729152 00000100:00100000:1.0:1787739068.710113:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665729152 00000100:00100000:1.0:1787739068.710154:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3371 00000100:00100000:1.0:1787739068.710162:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:20579:x1874579665729152:12345-0@lo:1101:lctl.0 00100000:10000000:1.0:1787739068.710188:0:12410:0:(lfsck_lib.c:3085:lfsck_start()) lustre-MDT0001-osd: start master thread 00000100:00100000:0.0:1787739068.710325:0:20579:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af9c0 going to sleep for 11 seconds 00100000:10000000:0.0:1787739068.710900:0:20582:0:(scrub.c:295:scrub_thread_prep()) lustre-MDT0001: OI scrub prep, flags = 0x42 00100000:10000000:0.0:1787739068.710930:0:20582:0:(scrub.c:230:scrub_file_store()) lustre-MDT0001: store scrub file: rc = 0 00000100:00100000:2.0:1787739068.711047:0:20581:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0000-lwp-MDT0001_UUID:20581:1874579665729280:0@lo:900:lfsck.0 00000100:00100000:2.0:1787739068.711088:0:20581:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.711202:0:20581:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739068.711246:0:6534:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665729280 00000100:00100000:3.0:1787739068.711251:0:6534:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665729280 00000100:00100000:3.0:1787739068.711292:0:6534:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7 00000100:00100000:3.0:1787739068.711300:0:6534:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be7c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_fld_0000:lustre-MDT0000-lwp-MDT0001_UUID+5:20581:x1874579665729280:12345-0@lo:900:lfsck.0 00000100:00100000:3.0:1787739068.711366:0:6534:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be7c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_fld_0000:lustre-MDT0000-lwp-MDT0001_UUID+5:20581:x1874579665729280:12345-0@lo:900:lfsck.0 Request processed in 66us (275us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739068.711373:0:6534:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7 00000100:00100000:2.0:1787739068.711392:0:20581:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0000-lwp-MDT0001_UUID:20581:1874579665729280:0@lo:900:lfsck.0 00000100:00100000:1.0:1787739068.711516:0:20581:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306390b40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665729408:0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.711545:0:20581:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.711565:0:20581:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.711583:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665729408 00000100:00100000:2.0:1787739068.711587:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665729408 00000100:00100000:2.0:1787739068.711619:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3372 00000100:00100000:2.0:1787739068.711626:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306391680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665729408:12345-0@lo:1000:lfsck.0 00000100:00100000:2.0:1787739068.711692:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306391680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665729408:12345-0@lo:1000:lfsck.0 Request processed in 67us (145us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.711699:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3372 00000100:00100000:1.0:1787739068.711708:0:20581:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306390b40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665729408:0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.711747:0:20581:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306390b40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665729536:0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.711769:0:20581:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.711791:0:20581:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539000 going to sleep for 11 seconds 00000100:00100000:0.0:1787739068.711828:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665729536 00000100:00100000:0.0:1787739068.711832:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665729536 00000100:00100000:0.0:1787739068.711853:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3373 00000100:00100000:0.0:1787739068.711859:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306393480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:20581:x1874579665729536:12345-0@lo:1000:lfsck.0 00000100:00100000:0.0:1787739068.711933:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306393480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:20581:x1874579665729536:12345-0@lo:1000:lfsck.0 Request processed in 74us (162us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739068.711939:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3373 00000100:00100000:1.0:1787739068.712054:0:20581:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306390b40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665729536:0@lo:1000:lfsck.0 00010000:00010000:1.0:1787739068.712096:0:20581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fc000/0x7ee4453e4a1ade36 lrc: 3/0,1 mode: --/CW res: [0x200000002:0x3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 20581 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.712106:0:20581:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fc000/0x7ee4453e4a1ade36 lrc: 3/0,1 mode: --/CW res: [0x200000002:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20581 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.712189:0:20581:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fc000/0x7ee4453e4a1ade36 lrc: 3/0,1 mode: --/CW res: [0x200000002:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 20581 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739068.713580:0:20581:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306390b40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665729664:0@lo:101:lfsck.0 00000100:00100000:0.0:1787739068.713771:0:20581:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739068.713824:0:20581:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.714271:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665729664 00000100:00100000:1.0:1787739068.714280:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665729664 00000100:00100000:1.0:1787739068.714333:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3374 00000100:00100000:1.0:1787739068.714342:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665729664:12345-0@lo:101:lfsck.0 00010000:00010000:1.0:1787739068.714357:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739068.714376:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fdc00/0x7ee4453e4a1ade3d lrc: 2/0,0 mode: --/CW res: [0x200000002:0x3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ade36 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.714444:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fdc00/0x7ee4453e4a1ade3d lrc: 3/0,0 mode: CW/CW res: [0x200000002:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ade36 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.714461:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fdc00/0x7ee4453e4a1ade3d lrc: 3/0,0 mode: CW/CW res: [0x200000002:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ade36 expref: 8 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.714469:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x3:0x0].0x0 (ffffa07433235e00) 00010000:00010000:1.0:1787739068.714472:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739068.714476:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fdc00, rc 0) 00000100:00100000:1.0:1787739068.714521:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:20581:x1874579665729664:12345-0@lo:101:lfsck.0 Request processed in 179us (750us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739068.714530:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3374 00000100:00100000:0.0:1787739068.714683:0:20581:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306390b40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665729664:0@lo:101:lfsck.0 00010000:00010000:0.0:1787739068.714713:0:20581:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fc000/0x7ee4453e4a1ade36 lrc: 4/0,1 mode: CW/CW res: [0x200000002:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ade3d expref: -99 pid: 20581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.714723:0:20581:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fc000/0x7ee4453e4a1ade36 lrc: 4/0,1 mode: CW/CW res: [0x200000002:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ade3d expref: -99 pid: 20581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739068.714742:0:20581:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000002:0x3:0x0] 00010000:00010000:0.0:1787739068.714776:0:20581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1ade44 lrc: 3/0,1 mode: --/PW res: [0x200000002:0x3:0x0].0x4649583 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 20581 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.714782:0:20581:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1ade44 lrc: 3/0,1 mode: --/PW res: [0x200000002:0x3:0x0].0x4649583 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20581 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.714789:0:20581:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1ade44 lrc: 3/0,1 mode: --/PW res: [0x200000002:0x3:0x0].0x4649583 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 20581 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739068.714804:0:20581:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f711680 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665729792:0@lo:101:lfsck.0 00000100:00100000:0.0:1787739068.714846:0:20581:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739068.714872:0:20581:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410cf6000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.714991:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665729792 00000100:00100000:2.0:1787739068.714996:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665729792 00000100:00100000:3.0:1787739068.715122:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3375 00000100:00100000:3.0:1787739068.715131:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7103c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:20581:x1874579665729792:12345-0@lo:101:lfsck.0 00010000:00010000:3.0:1787739068.715140:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739068.715154:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ade4b lrc: 2/0,0 mode: --/PW res: [0x200000002:0x3:0x0].0x4649583 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ade44 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.715169:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ade4b lrc: 3/0,0 mode: PW/PW res: [0x200000002:0x3:0x0].0x4649583 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ade44 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.715180:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa600/0x7ee4453e4a1ade4b lrc: 3/0,0 mode: PW/PW res: [0x200000002:0x3:0x0].0x4649583 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ade44 expref: 9 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.715186:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x3:0x0].0x4649583 (ffffa074331c2c00) 00010000:00010000:3.0:1787739068.715188:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739068.715191:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa600, rc 0) 00000100:00100000:3.0:1787739068.715242:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7103c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:20581:x1874579665729792:12345-0@lo:101:lfsck.0 Request processed in 113us (396us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739068.715248:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3375 00000100:00100000:0.0:1787739068.715568:0:20581:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f711680 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665729792:0@lo:101:lfsck.0 00010000:00010000:0.0:1787739068.715580:0:20581:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1ade44 lrc: 4/0,1 mode: PW/PW res: [0x200000002:0x3:0x0].0x4649583 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ade4b expref: -99 pid: 20581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.715588:0:20581:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1ade44 lrc: 4/0,1 mode: PW/PW res: [0x200000002:0x3:0x0].0x4649583 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ade4b expref: -99 pid: 20581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739068.715600:0:20581:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000002:0x3:0x0] 00000100:00100000:0.0:1787739068.715625:0:20581:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f711680 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665729920:0@lo:1000:lfsck.0 00000100:00100000:0.0:1787739068.715651:0:20581:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739068.715676:0:20581:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410cf6d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.715726:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665729920 00000100:00100000:1.0:1787739068.715730:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665729920 00000100:00100000:1.0:1787739068.715748:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3376 00000100:00100000:1.0:1787739068.715754:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410b0ac50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:20581:x1874579665729920:12345-0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.715826:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410b0ac50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:20581:x1874579665729920:12345-0@lo:1000:lfsck.0 Request processed in 72us (174us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739068.715833:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3376 00000100:00100000:0.0:1787739068.715956:0:20581:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f711680 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665729920:0@lo:1000:lfsck.0 00000004:00080000:0.0:1787739068.719892:0:20581:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07410cf6d80 with version 0 00000020:00100000:0.0:1787739068.719923:0:20581:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739068.719934:0:20581:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743f711680 x1874579665730176/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 400/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739068.719956:0:20581:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f711680 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665730176:0@lo:1000:lfsck.0 00000100:00100000:0.0:1787739068.720030:0:20581:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739068.720063:0:20581:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410cf66c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.720756:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665730176 00000100:00100000:1.0:1787739068.720764:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665730176 00000100:00100000:1.0:1787739068.720817:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3377 00000100:00100000:1.0:1787739068.720826:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f711a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:20581:x1874579665730176:12345-0@lo:1000:lfsck.0 00000001:00080000:3.0:1787739068.723061:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983434 is committed 00000001:00080000:3.0:1787739068.723104:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983433 is committed 00000020:00080000:3.0:1787739068.723116:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 388 batchid updated 00000020:00080000:3.0:1787739068.723137:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa074052af480 refcount 2 committed 0 result 0 batchid 389 00000020:00080000:3.0:1787739068.723142:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2200 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa074052afd80 00000020:00080000:3.0:1787739068.723146:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].386 00000020:00080000:3.0:1787739068.723149:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2900 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304277e00 00000020:00080000:3.0:1787739068.723152:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].387 00000020:00080000:2.0:1787739068.723318:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 389 committed batchid 388 00000100:00100000:1.0:1787739068.723335:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f711a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665730176:12345-0@lo:1000:lfsck.0 Request processed in 2507us (3301us total) trans 4294983434 rc 0/0 00000100:00100000:1.0:1787739068.723345:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3377 00000004:00080000:2.0:1787739068.723446:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306234cc0 with version 739 00000020:00080000:2.0:1787739068.723462:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 388 cancel update log [0x240000401:0x3:0x0].385: rc = 0 00000004:00080000:3.0:1787739068.723477:0:20581:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983434 committed transno 4294983434 00000100:00100000:3.0:1787739068.723482:0:20581:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983434 gen 2 00000004:00080000:3.0:1787739068.723486:0:20581:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983428 committed transno 4294983434 00000020:00080000:3.0:1787739068.723489:0:20581:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07307054cc0 refcount 2 committed 0 result 0 batchid 379 00000020:00080000:3.0:1787739068.723493:0:20581:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743328fa00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa0743c0efd80 00000020:00080000:3.0:1787739068.723496:0:20581:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].371 00000020:00080000:3.0:1787739068.723498:0:20581:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743328f100 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073069be600 00000020:00080000:3.0:1787739068.723500:0:20581:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].371 00000020:00080000:2.0:1787739068.723500:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 388 cancel update log [0x200000400:0x3:0x0].385: rc = 0 00000020:00080000:2.0:1787739068.723511:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 388 00000004:00080000:3.0:1787739068.723522:0:20581:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983430 committed transno 4294983434 00000100:00100000:3.0:1787739068.723538:0:20581:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f711680 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665730176:0@lo:1000:lfsck.0 00010000:00010000:3.0:1787739068.723562:0:20581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1ade44 lrc: 3/0,1 mode: PW/PW res: [0x200000002:0x3:0x0].0x4649583 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ade4b expref: -99 pid: 20581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.723572:0:20581:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609a800/0x7ee4453e4a1ade44 lrc: 2/0,0 mode: PW/PW res: [0x200000002:0x3:0x0].0x4649583 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ade4b expref: -99 pid: 20581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.723581:0:20581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fc000/0x7ee4453e4a1ade36 lrc: 3/0,1 mode: CW/CW res: [0x200000002:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ade3d expref: -99 pid: 20581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.723588:0:20581:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fc000/0x7ee4453e4a1ade36 lrc: 2/0,0 mode: CW/CW res: [0x200000002:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ade3d expref: -99 pid: 20581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00100000:10000000:3.0:1787739068.723615:0:20581:0:(lfsck_lib.c:2526:lfsck_start_assistant()) lustre-MDT0001-osd: start lfsck_namespace assistant thread 00000004:00080000:1.0:1787739068.723712:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 739 rpc_version 739 00000020:00100000:1.0:1787739068.723738:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739068.723745:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306392d00 x1874579665730432/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739068.723765:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306392d00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665730432:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739068.723804:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.723839:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539000 going to sleep for 11 seconds 00000020:00080000:1.0:1787739068.723852:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 380 committed batchid 377 00000020:00080000:1.0:1787739068.723879:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 377 00000100:00100000:3.0:1787739068.724061:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665730432 00000100:00100000:3.0:1787739068.724068:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665730432 00000100:00100000:3.0:1787739068.724116:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3378 00000100:00100000:3.0:1787739068.724124:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306392580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665730432:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739068.724223:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306392580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665730432:12345-0@lo:1000:osp_up1-0.0 Request processed in 99us (418us total) trans 4294968425 rc 0/0 00000100:00100000:3.0:1787739068.724229:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3378 00000100:00100000:1.0:1787739068.724300:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306392d00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665730432:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787739068.724476:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1add17 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724486:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1add17 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724495:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1add17 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724503:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739068.724508:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1add17 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724517:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1add17 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724533:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xa20ac6a2 (ffffa07305ca2800) 00010000:00010000:0.0:1787739068.724536:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739068.724540:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1add17 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724547:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a400/0x7ee4453e4a1add17 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724559:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1add1e lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1528:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724566:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1add1e lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x1528:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724572:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1add1e lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1528:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724577:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739068.724580:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1add1e lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x1528:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724585:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1add1e lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x1528:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724595:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1528:0x0].0x0 (ffffa07305ca2600) 00010000:00010000:0.0:1787739068.724597:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739068.724599:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1add1e lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x1528:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724604:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1add1e lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x1528:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724619:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1add25 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724624:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1add25 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724629:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1add25 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724634:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739068.724636:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1add25 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724642:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1add25 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724649:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xcd:0x0].0x0 (ffffa07305ca2e00) 00010000:00010000:0.0:1787739068.724651:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739068.724654:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1add25 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724658:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b000/0x7ee4453e4a1add25 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xcd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724676:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1add79 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724682:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1add79 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724689:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1add79 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724694:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739068.724696:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1add79 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724702:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1add79 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724709:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xa20ac6a2 (ffffa07305ca2800) 00010000:00010000:0.0:1787739068.724712:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739068.724714:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1add79 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724720:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1add79 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xa20ac6a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724733:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1addf7 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724739:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1addf7 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724744:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1addf7 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724749:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739068.724751:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1addf7 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724756:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1addf7 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724764:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd04803f8 (ffffa074331c2300) 00010000:00010000:0.0:1787739068.724767:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739068.724769:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1addf7 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724775:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1addf7 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724784:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1addfe lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x152a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724789:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1addfe lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x152a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724795:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1addfe lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x152a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724799:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739068.724802:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1addfe lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x152a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724807:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1addfe lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x152a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724817:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152a:0x0].0x0 (ffffa074331c2000) 00010000:00010000:0.0:1787739068.724819:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739068.724825:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1addfe lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x152a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724830:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa800/0x7ee4453e4a1addfe lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x152a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724842:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade28 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724849:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade28 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724855:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade28 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724860:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fd400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739068.724863:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade28 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724869:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade28 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724876:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd04803f8 (ffffa074331c2300) 00010000:00010000:0.0:1787739068.724878:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739068.724881:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade28 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.724885:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade28 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xd04803f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00100000:10000000:3.0:1787739068.726845:0:20581:0:(lfsck_namespace.c:4580:lfsck_namespace_prep()) lustre-MDT0001-osd: namespace LFSCK prep done, start pos [1, [0x0:0x0:0x0], 0x0]: rc = 0 00100000:10000000:0.0:1787739068.726913:0:20583:0:(lfsck_engine.c:1551:lfsck_assistant_engine()) lustre-MDT0001-osd: lfsck_namespace LFSCK assistant thread start 00100000:10000000:2.0:1787739068.726961:0:20582:0:(osd_scrub.c:1263:osd_scrub_main()) lustre-MDT0001: OI scrub start, flags = 0x42, pos = 12 00100000:10000000:3.0:1787739068.731435:0:20581:0:(lfsck_namespace.c:4478:lfsck_namespace_checkpoint()) lustre-MDT0001-osd: namespace LFSCK checkpoint at the pos [13, [0x0:0x0:0x0], 0x0], status = 1: rc = 0 00100000:10000000:3.0:1787739068.731445:0:20581:0:(lfsck_engine.c:1034:lfsck_master_engine()) lustre-MDT0001-osd entry: oit_flags = 0x40003, dir_flags = 0x8006, oit_cookie = 13, dir_cookie = 0x0, parent = [0x0:0x0:0x0], pid = 20581 00000100:00100000:3.0:1787739068.731581:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20579:x1874579665729152:12345-0@lo:1101:lctl.0 Request processed in 21416us (21582us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739068.731594:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3371 00000100:00100000:0.0:1787739068.731950:0:20579:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060d0780 pname:cluuid:pid:xid:nid:opc:job lctl:lustre-MDT0000-mdtlov_UUID:20579:1874579665729152:0@lo:1101:lctl.0 00000100:00100000:3.0:1787739068.733525:0:20581:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562fc00 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665730560:0@lo:1000:lfsck.0 00000100:00100000:3.0:1787739068.733595:0:20581:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.733649:0:20581:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241600 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.733798:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665730560 00000100:00100000:2.0:1787739068.733806:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665730560 00000100:00100000:2.0:1787739068.733864:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3379 00000100:00100000:2.0:1787739068.733873:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665730560:12345-0@lo:1000:lfsck.0 00000100:00100000:2.0:1787739068.733953:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665730560:12345-0@lo:1000:lfsck.0 Request processed in 81us (358us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.733960:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3379 00000100:00100000:3.0:1787739068.734096:0:20581:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562fc00 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665730560:0@lo:1000:lfsck.0 00000100:00100000:3.0:1787739068.734156:0:20581:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562fc00 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665730688:0@lo:1000:lfsck.0 00000100:00100000:3.0:1787739068.734178:0:20581:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.734205:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665730688 00000100:00100000:3.0:1787739068.734209:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665730688 00000100:00100000:1.0:1787739068.734500:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073063903c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665730816:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.734529:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.734956:0:20581:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241600 going to sleep for 11 seconds 00100000:10000000:0.0:1787739068.737104:0:20584:0:(scrub.c:295:scrub_thread_prep()) lustre-MDT0000: OI scrub prep, flags = 0x0 00100000:10000000:0.0:1787739068.737178:0:20584:0:(scrub.c:230:scrub_file_store()) lustre-MDT0000: store scrub file: rc = 0 00010000:00010000:0.0:1787739068.737275:0:20580:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ade52 lrc: 3/0,1 mode: --/CW res: [0x200000002:0x3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 20580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.737293:0:20580:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ade52 lrc: 3/0,1 mode: CW/CW res: [0x200000002:0x3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.737306:0:20580:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ade52 lrc: 3/0,1 mode: CW/CW res: [0x200000002:0x3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 20580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.737326:0:20580:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ade59 lrc: 3/0,1 mode: --/PW res: [0x200000002:0x3:0x0].0xaaab9583 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.737333:0:20580:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ade59 lrc: 3/0,1 mode: PW/PW res: [0x200000002:0x3:0x0].0xaaab9583 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.737343:0:20580:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ade59 lrc: 3/0,1 mode: PW/PW res: [0x200000002:0x3:0x0].0xaaab9583 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.737801:0:20580:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ade59 lrc: 3/0,1 mode: PW/PW res: [0x200000002:0x3:0x0].0xaaab9583 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.737812:0:20580:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ade59 lrc: 2/0,0 mode: PW/PW res: [0x200000002:0x3:0x0].0xaaab9583 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.737819:0:20580:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ade59 lrc: 3/0,0 mode: PW/PW res: [0x200000002:0x3:0x0].0xaaab9583 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.737833:0:20580:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a000 already unused, calling callback (ffffffffc0cce440) 00010000:00010000:0.0:1787739068.737837:0:20580:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ade59 lrc: 3/0,0 mode: PW/PW res: [0x200000002:0x3:0x0].0xaaab9583 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.737847:0:20580:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ade59 lrc: 4/0,0 mode: PW/PW res: [0x200000002:0x3:0x0].0xaaab9583 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.737862:0:20580:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x3:0x0].0xaaab9583 (ffffa07305ca2200) 00010000:00010000:0.0:1787739068.737865:0:20580:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739068.737871:0:20580:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ade59 lrc: 2/0,0 mode: --/PW res: [0x200000002:0x3:0x0].0xaaab9583 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.737878:0:20580:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ade59 lrc: 0/0,0 mode: --/PW res: [0x200000002:0x3:0x0].0xaaab9583 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.737897:0:20580:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ade52 lrc: 3/0,1 mode: CW/CW res: [0x200000002:0x3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 20580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.737926:0:20580:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ade52 lrc: 2/0,0 mode: CW/CW res: [0x200000002:0x3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 20580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.737970:0:20580:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ade52 lrc: 3/0,0 mode: CW/CW res: [0x200000002:0x3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 20580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.738057:0:20580:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ac00 already unused, calling callback (ffffffffc0cce440) 00010000:00010000:0.0:1787739068.738060:0:20580:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ade52 lrc: 3/0,0 mode: CW/CW res: [0x200000002:0x3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 20580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.738067:0:20580:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ade52 lrc: 4/0,0 mode: CW/CW res: [0x200000002:0x3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 20580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.738393:0:20580:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x3:0x0].0x0 (ffffa07433235e00) 00010000:00010000:0.0:1787739068.738396:0:20580:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739068.738399:0:20580:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ade52 lrc: 2/0,0 mode: --/CW res: [0x200000002:0x3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 20580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.738405:0:20580:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ade52 lrc: 0/0,0 mode: --/CW res: [0x200000002:0x3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 20580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.738528:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00100000:10000000:0.0:1787739068.738543:0:20580:0:(lfsck_lib.c:2526:lfsck_start_assistant()) lustre-MDT0000-osd: start lfsck_namespace assistant thread 00000100:00100000:1.0:1787739068.740350:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539540 going to sleep for 11 seconds 00100000:10000000:1.0:1787739068.740389:0:20580:0:(lfsck_namespace.c:4580:lfsck_namespace_prep()) lustre-MDT0000-osd: namespace LFSCK prep done, start pos [1, [0x0:0x0:0x0], 0x0]: rc = 0 00100000:10000000:1.0:1787739068.740407:0:20585:0:(lfsck_engine.c:1551:lfsck_assistant_engine()) lustre-MDT0000-osd: lfsck_namespace LFSCK assistant thread start 00100000:10000000:1.0:1787739068.740549:0:20584:0:(osd_scrub.c:1263:osd_scrub_main()) lustre-MDT0000: OI scrub start, flags = 0x0, pos = 12 00000100:00100000:3.0:1787739068.740915:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665730816 00000100:00100000:3.0:1787739068.740924:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665730816 00000100:00100000:3.0:1787739068.740971:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3380 00100000:10000000:1.0:1787739068.741127:0:20580:0:(lfsck_namespace.c:4478:lfsck_namespace_checkpoint()) lustre-MDT0000-osd: namespace LFSCK checkpoint at the pos [13, [0x0:0x0:0x0], 0x0], status = 1: rc = 0 00100000:10000000:1.0:1787739068.741143:0:20580:0:(lfsck_engine.c:1034:lfsck_master_engine()) lustre-MDT0000-osd entry: oit_flags = 0x40000, dir_flags = 0x8006, oit_cookie = 13, dir_cookie = 0x0, parent = [0x0:0x0:0x0], pid = 20580 00100000:10000000:1.0:1787739068.742321:0:20584:0:(scrub.c:370:scrub_thread_post()) lustre-MDT0000: OI scrub post with result = 1 00100000:10000000:1.0:1787739068.742353:0:20584:0:(scrub.c:230:scrub_file_store()) lustre-MDT0000: store scrub file: rc = 0 00100000:10000000:1.0:1787739068.742356:0:20584:0:(osd_scrub.c:1303:osd_scrub_main()) lustre-MDT0000: OI scrub: stop, pos = 1024001: rc = 1 00000100:00100000:3.0:1787739068.742882:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:20581:x1874579665730688:12345-0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.742990:0:20580:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306393480 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0000-mdtlov_UUID:20580:1874579665730944:0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.743029:0:20580:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.743062:0:20580:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739068.743067:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:20581:x1874579665730688:12345-0@lo:1000:lfsck.0 Request processed in 194us (8887us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739068.743084:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3380 00000100:00100000:3.0:1787739068.743105:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665730944 00000100:00100000:3.0:1787739068.743111:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665730944 00000100:00100000:1.0:1787739068.743118:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306390b40 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665731072:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.743134:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3381 00000100:00100000:3.0:1787739068.743140:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306390f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665730816:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.743142:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.743153:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688dcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739068.743198:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306390f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665730816:12345-0@lo:1000:lfsck_namespace.0 Request processed in 57us (8667us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739068.743205:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3381 00000100:00100000:3.0:1787739068.743216:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665731072 00000100:00100000:3.0:1787739068.743219:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665731072 00000100:00100000:3.0:1787739068.743231:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3382 00000100:00100000:3.0:1787739068.743235:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306391680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:20580:x1874579665730944:12345-0@lo:1000:lfsck.0 00000100:00100000:3.0:1787739068.743529:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306391680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:20580:x1874579665730944:12345-0@lo:1000:lfsck.0 Request processed in 292us (499us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739068.743536:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3382 00000100:00100000:3.0:1787739068.743548:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3383 00000100:00100000:3.0:1787739068.743554:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304263c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665731072:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.743602:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304263c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665731072:12345-0@lo:1000:lfsck_namespace.0 Request processed in 48us (458us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739068.743608:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3383 00000100:00100000:3.0:1787739068.743659:0:20581:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562fc00 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665730688:0@lo:1000:lfsck.0 00000100:00100000:3.0:1787739068.743943:0:20581:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562fc00 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665731200:0@lo:1000:lfsck.0 00000100:00100000:3.0:1787739068.743972:0:20581:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.744076:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665731200 00000100:00100000:3.0:1787739068.744081:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665731200 00000100:00100000:3.0:1787739068.744103:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3384 00000100:00100000:3.0:1787739068.744108:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665731200:12345-0@lo:1000:lfsck.0 00000100:00100000:3.0:1787739068.744157:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665731200:12345-0@lo:1000:lfsck.0 Request processed in 48us (183us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739068.744162:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3384 00000100:00100000:3.0:1787739068.744195:0:20580:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306393480 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0000-mdtlov_UUID:20580:1874579665730944:0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.744373:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073063903c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665730816:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.744460:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304262d00 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665731328:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.744487:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.744509:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688de40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739068.744512:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665731328 00000100:00100000:3.0:1787739068.744516:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665731328 00000100:00100000:1.0:1787739068.744528:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306390b40 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665731072:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.744537:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3385 00000100:00100000:3.0:1787739068.744542:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304261e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665731328:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.744578:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304261680 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665731456:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.744593:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304261e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665731328:12345-0@lo:1000:lfsck_namespace.0 Request processed in 50us (106us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739068.744596:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304262d00 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665731328:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.744600:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3385 00000100:00100000:3.0:1787739068.744684:0:20580:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f3840 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0000-mdtlov_UUID:20580:1874579665731584:0@lo:1000:lfsck.0 00000100:00100000:2.0:1787739068.744819:0:20581:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410cf66c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.744834:0:20581:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562fc00 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665731200:0@lo:1000:lfsck.0 00000100:00100000:2.0:1787739068.744879:0:20581:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665731712:0@lo:1000:lfsck.0 00000100:00100000:3.0:1787739068.744881:0:20580:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.744906:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665731584 00000100:00100000:3.0:1787739068.744910:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665731584 00000100:00100000:3.0:1787739068.744931:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3386 00000100:00100000:3.0:1787739068.744936:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20580:x1874579665731584:12345-0@lo:1000:lfsck.0 00000100:00100000:2.0:1787739068.745012:0:20581:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.745027:0:20581:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234180 going to sleep for 11 seconds 00000100:00100000:3.0:1787739068.745075:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20580:x1874579665731584:12345-0@lo:1000:lfsck.0 Request processed in 138us (194us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739068.745375:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665731712 00000100:00100000:0.0:1787739068.745380:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665731712 00000100:00100000:3.0:1787739068.745393:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3386 00000100:00100000:0.0:1787739068.745402:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3387 00000100:00100000:0.0:1787739068.745408:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665731712:12345-0@lo:1000:lfsck.0 00000100:00100000:3.0:1787739068.745440:0:20580:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6ed80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739068.745455:0:20580:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f3840 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0000-mdtlov_UUID:20580:1874579665731584:0@lo:1000:lfsck.0 00000100:00100000:0.0:1787739068.745479:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665731712:12345-0@lo:1000:lfsck.0 Request processed in 72us (467us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739068.745485:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3387 00000100:00100000:3.0:1787739068.745561:0:20580:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f3840 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0000-mdtlov_UUID:20580:1874579665731840:0@lo:1000:lfsck.0 00000100:00100000:3.0:1787739068.745587:0:20580:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.745608:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665731840 00000100:00100000:3.0:1787739068.745611:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665731840 00000100:00100000:3.0:1787739068.745631:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3388 00000100:00100000:3.0:1787739068.745637:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20580:x1874579665731840:12345-0@lo:1000:lfsck.0 00000100:00100000:3.0:1787739068.745682:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20580:x1874579665731840:12345-0@lo:1000:lfsck.0 Request processed in 45us (93us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739068.745688:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3388 00000100:00100000:2.0:1787739068.745728:0:20581:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665731712:0@lo:1000:lfsck.0 00000100:00100000:3.0:1787739068.745781:0:20580:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303c6e180 going to sleep for 11 seconds 00000100:00100000:3.0:1787739068.745794:0:20580:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f3840 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0000-mdtlov_UUID:20580:1874579665731840:0@lo:1000:lfsck.0 00000100:00100000:2.0:1787739068.745858:0:20581:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665731968:0@lo:1000:lfsck.0 00000100:00100000:2.0:1787739068.745883:0:20581:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.745899:0:20581:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739068.745954:0:20580:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f3840 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0000-mdtlov_UUID:20580:1874579665732096:0@lo:1000:lfsck.0 00000100:00100000:3.0:1787739068.746184:0:20580:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.746214:0:20580:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a5f66c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.746246:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665731968 00000100:00100000:2.0:1787739068.746250:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665731968 00000100:00100000:2.0:1787739068.746277:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665732096 00000100:00100000:2.0:1787739068.746279:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665732096 00000100:00100000:2.0:1787739068.746293:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3389 00000100:00100000:2.0:1787739068.746299:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665731968:12345-0@lo:1000:lfsck.0 00000100:00100000:3.0:1787739068.746349:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3390 00000100:00100000:2.0:1787739068.746351:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665731968:12345-0@lo:1000:lfsck.0 Request processed in 52us (468us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739068.746356:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306390b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20580:x1874579665732096:12345-0@lo:1000:lfsck.0 00000100:00100000:2.0:1787739068.746357:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3389 00000100:00100000:2.0:1787739068.746382:0:20581:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665731968:0@lo:1000:lfsck.0 00000100:00100000:3.0:1787739068.746409:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306390b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20580:x1874579665732096:12345-0@lo:1000:lfsck.0 Request processed in 53us (224us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739068.746430:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3390 00000100:00100000:3.0:1787739068.746455:0:20580:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f3840 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0000-mdtlov_UUID:20580:1874579665732096:0@lo:1000:lfsck.0 00000100:00100000:3.0:1787739068.746934:0:20580:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073063903c0 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0000-mdtlov_UUID:20580:1874579665732224:0@lo:1000:lfsck.0 00000100:00100000:3.0:1787739068.746959:0:20580:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.747051:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665732224 00000100:00100000:3.0:1787739068.747055:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665732224 00000100:00100000:3.0:1787739068.747079:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3391 00000100:00100000:3.0:1787739068.747085:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306393480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20580:x1874579665732224:12345-0@lo:1000:lfsck.0 00000100:00100000:3.0:1787739068.747138:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306393480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20580:x1874579665732224:12345-0@lo:1000:lfsck.0 Request processed in 53us (178us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739068.747145:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3391 00000100:00100000:3.0:1787739068.747167:0:20580:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404300 going to sleep for 11 seconds 00000100:00100000:3.0:1787739068.747180:0:20580:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073063903c0 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0000-mdtlov_UUID:20580:1874579665732224:0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.747897:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.747927:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688dcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739068.748542:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665731456 00000100:00100000:0.0:1787739068.748550:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665731456 00000100:00100000:0.0:1787739068.748586:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3392 00000100:00100000:0.0:1787739068.748594:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665731456:12345-0@lo:1000:lfsck_namespace.0 00100000:10000000:2.0:1787739068.748824:0:20582:0:(scrub.c:370:scrub_thread_post()) lustre-MDT0001: OI scrub post with result = 1 00100000:10000000:2.0:1787739068.748854:0:20582:0:(scrub.c:230:scrub_file_store()) lustre-MDT0001: store scrub file: rc = 0 00100000:10000000:2.0:1787739068.748857:0:20582:0:(osd_scrub.c:1303:osd_scrub_main()) lustre-MDT0001: OI scrub: stop, pos = 1024001: rc = 1 00000100:00100000:1.0:1787739068.749246:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304263480 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665732352:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.749297:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.749321:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d180 going to sleep for 11 seconds 00000100:00100000:3.0:1787739068.749402:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665732352 00000100:00100000:3.0:1787739068.749407:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665732352 00000100:00100000:3.0:1787739068.749441:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3393 00000100:00100000:3.0:1787739068.749447:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665732352:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.749515:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665732352:12345-0@lo:1000:lfsck_namespace.0 Request processed in 67us (217us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739068.749521:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3393 00000100:00100000:3.0:1787739068.749563:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304263480 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665732352:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.749624:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073063903c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665732480:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.749652:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.749675:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46cc00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739068.749686:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665732480 00000100:00100000:3.0:1787739068.749689:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665732480 00000100:00100000:0.0:1787739068.749823:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665731456:12345-0@lo:1000:lfsck_namespace.0 Request processed in 1229us (1926us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739068.749830:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3392 00000100:00100000:0.0:1787739068.750482:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304261680 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665731456:0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.750591:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f712940 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665732608:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.750814:0:20581:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665732736:0@lo:1000:lfsck.0 00000100:00100000:2.0:1787739068.750847:0:20581:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.750883:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665732736 00000100:00100000:2.0:1787739068.750888:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665732736 00000100:00100000:2.0:1787739068.750922:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3394 00000100:00100000:2.0:1787739068.750928:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665732736:12345-0@lo:1000:lfsck.0 00000100:00100000:2.0:1787739068.750994:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665732736:12345-0@lo:1000:lfsck.0 Request processed in 66us (145us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.751000:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3394 00000100:00100000:2.0:1787739068.751027:0:20581:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068876c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.751039:0:20581:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665732736:0@lo:1000:lfsck.0 00000100:00100000:2.0:1787739068.751072:0:20581:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665732864:0@lo:1000:lfsck.0 00000100:00100000:0.0:1787739068.751312:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739068.751336:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803780 going to sleep for 11 seconds 00000100:00100000:3.0:1787739068.751861:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665732608 00000100:00100000:3.0:1787739068.751866:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665732608 00000100:00100000:3.0:1787739068.751880:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3395 00000100:00100000:3.0:1787739068.751886:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665732480:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.751947:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665732480:12345-0@lo:1000:lfsck_namespace.0 Request processed in 60us (2294us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739068.751952:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3395 00000100:00100000:3.0:1787739068.751966:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3396 00000100:00100000:3.0:1787739068.751971:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7121c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665732608:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.752090:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7121c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665732608:12345-0@lo:1000:lfsck_namespace.0 Request processed in 118us (777us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739068.752096:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3396 00000100:00100000:3.0:1787739068.752132:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073063903c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665732480:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.752225:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073063903c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665732992:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.752246:0:20581:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.752269:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665732864 00000100:00100000:3.0:1787739068.752273:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665732864 00000100:00100000:2.0:1787739068.752284:0:20581:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068876c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739068.752291:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3397 00000100:00100000:3.0:1787739068.752297:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665732864:12345-0@lo:1000:lfsck.0 00000100:00100000:3.0:1787739068.752355:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665732864:12345-0@lo:1000:lfsck.0 Request processed in 58us (108us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739068.752362:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3397 00000100:00100000:2.0:1787739068.752373:0:20581:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665732864:0@lo:1000:lfsck.0 00000100:00100000:3.0:1787739068.752379:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.752399:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665732992 00000100:00100000:2.0:1787739068.752402:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665732992 00000100:00100000:3.0:1787739068.752406:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.752447:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3398 00000100:00100000:2.0:1787739068.752453:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157eb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665732992:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.752499:0:20580:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074157e9e00 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0000-mdtlov_UUID:20580:1874579665733120:0@lo:1000:lfsck.0 00000100:00100000:2.0:1787739068.752501:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157eb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665732992:12345-0@lo:1000:lfsck_namespace.0 Request processed in 49us (121us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.752507:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3398 00000100:00100000:3.0:1787739068.752516:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073063903c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665732992:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.752554:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073063903c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665733248:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.752576:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.752595:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665733248 00000100:00100000:3.0:1787739068.752598:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665733248 00000100:00100000:3.0:1787739068.752619:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3399 00000100:00100000:3.0:1787739068.752624:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f75e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665733248:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.752669:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f75e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665733248:12345-0@lo:1000:lfsck_namespace.0 Request processed in 44us (91us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739068.752675:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3399 00000100:00100000:3.0:1787739068.752690:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46ccc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739068.752701:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073063903c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665733248:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.752940:0:20580:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.753233:0:20581:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665733376:0@lo:1000:lfsck.0 00000100:00100000:0.0:1787739068.753326:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f712940 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665732608:0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.753378:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f712940 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665733504:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.753842:0:20580:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46c600 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.755485:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665733120 00000100:00100000:1.0:1787739068.755499:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665733120 00000100:00100000:2.0:1787739068.755536:0:20581:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.755563:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665733376 00000100:00100000:1.0:1787739068.755567:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665733376 00000100:00100000:2.0:1787739068.755569:0:20581:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887840 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.755588:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3400 00000100:00100000:1.0:1787739068.755598:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20580:x1874579665733120:12345-0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.755727:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20580:x1874579665733120:12345-0@lo:1000:lfsck.0 Request processed in 130us (2786us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739068.755735:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3400 00000100:00100000:1.0:1787739068.755751:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3401 00000100:00100000:1.0:1787739068.755757:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665733376:12345-0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.756124:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665733376:12345-0@lo:1000:lfsck.0 Request processed in 366us (589us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739068.756133:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3401 00000100:00100000:1.0:1787739068.756253:0:20581:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be5a40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665733376:0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.756343:0:20581:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304260b40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665733632:0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.756386:0:20581:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.756408:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665733632 00000100:00100000:1.0:1787739068.757133:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665733632 00000100:00100000:1.0:1787739068.757185:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3402 00000100:00100000:1.0:1787739068.757193:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042630c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665733632:12345-0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.757275:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042630c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665733632:12345-0@lo:1000:lfsck.0 Request processed in 83us (888us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739068.757282:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3402 00000100:00100000:1.0:1787739068.757308:0:20581:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688df00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.757326:0:20581:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304260b40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665733632:0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.757636:0:20581:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304260b40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665733760:0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.757674:0:20581:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.757698:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665733760 00000100:00100000:1.0:1787739068.757701:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665733760 00000100:00100000:1.0:1787739068.757724:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3403 00000100:00100000:1.0:1787739068.757729:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304260f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665733760:12345-0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.757797:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304260f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665733760:12345-0@lo:1000:lfsck.0 Request processed in 67us (122us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739068.757803:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3403 00000100:00100000:1.0:1787739068.757820:0:20581:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.757830:0:20581:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304260b40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665733760:0@lo:1000:lfsck.0 00000100:00100000:0.0:1787739068.758229:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739068.758266:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803780 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.758462:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665733504 00000100:00100000:1.0:1787739068.758467:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665733504 00000100:00100000:1.0:1787739068.758497:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3404 00000100:00100000:1.0:1787739068.758503:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7112c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665733504:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.758631:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7112c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665733504:12345-0@lo:1000:lfsck_namespace.0 Request processed in 126us (401us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739068.758639:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3404 00000100:00100000:2.0:1787739068.758654:0:20580:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074157e9e00 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0000-mdtlov_UUID:20580:1874579665733120:0@lo:1000:lfsck.0 00100000:10000000:2.0:1787739068.758894:0:20580:0:(lfsck_engine.c:1053:lfsck_master_engine()) lustre-MDT0000-osd ffffa07305eccf80 exit: oit_flags = 0x40000, dir_flags = 0x8006, oit_cookie = 512152, dir_cookie = 0x35057a3f63a78e50, parent = [0x200000403:0x1:0x0], pid = 20580: rc = 1 00100000:10000000:2.0:1787739068.758904:0:20580:0:(lfsck_lib.c:2594:lfsck_post_generic()) lustre-MDT0000-osd: waiting for assistant to do lfsck_namespace post, rc = 1 00000100:00100000:0.0:1787739068.759034:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f712940 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665733504:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.759081:0:20581:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304260b40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665733888:0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.759116:0:20581:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739068.759134:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f712940 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665734016:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.759141:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665733888 00000100:00100000:1.0:1787739068.759145:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665733888 00000100:00100000:0.0:1787739068.759166:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.760660:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f770c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665734144:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.761251:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665734016 00000100:00100000:1.0:1787739068.761259:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665734016 00000100:00100000:1.0:1787739068.761300:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3405 00000100:00100000:1.0:1787739068.761309:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042621c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665733888:12345-0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.761410:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042621c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665733888:12345-0@lo:1000:lfsck.0 Request processed in 102us (2293us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739068.761436:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3405 00000100:00100000:1.0:1787739068.761454:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3406 00000100:00100000:1.0:1787739068.761460:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f712580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665734016:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.761512:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f712580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665734016:12345-0@lo:1000:lfsck_namespace.0 Request processed in 53us (2346us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739068.761529:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3406 00000100:00100000:1.0:1787739068.761561:0:20581:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688de40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.761582:0:20581:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304260b40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665733888:0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.761962:0:20581:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304260b40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665734272:0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.762003:0:20581:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.762029:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665734272 00000100:00100000:1.0:1787739068.762032:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665734272 00000100:00100000:1.0:1787739068.762058:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3407 00000100:00100000:1.0:1787739068.762065:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042612c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665734272:12345-0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.762123:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042612c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665734272:12345-0@lo:1000:lfsck.0 Request processed in 58us (118us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739068.762129:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3407 00000100:00100000:1.0:1787739068.762151:0:20581:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730688d480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.762165:0:20581:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304260b40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665734272:0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.762435:0:20581:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304260b40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665734400:0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.762463:0:20581:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.762486:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665734400 00000100:00100000:1.0:1787739068.762489:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665734400 00000100:00100000:1.0:1787739068.762514:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3408 00000100:00100000:1.0:1787739068.762521:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665734400:12345-0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.762566:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665734400:12345-0@lo:1000:lfsck.0 Request processed in 45us (102us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739068.762573:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3408 00000100:00100000:1.0:1787739068.762590:0:20581:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410cf66c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.762603:0:20581:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304260b40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665734400:0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.762638:0:20581:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304260b40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665734528:0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.762657:0:20581:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.762683:0:20581:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.762828:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665734528 00000100:00100000:2.0:1787739068.762833:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665734528 00000100:00100000:2.0:1787739068.762867:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3409 00000100:00100000:2.0:1787739068.762875:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304261a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665734528:12345-0@lo:1000:lfsck.0 00000100:00100000:2.0:1787739068.762929:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304261a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20581:x1874579665734528:12345-0@lo:1000:lfsck.0 Request processed in 55us (270us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.762936:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3409 00000100:00100000:1.0:1787739068.763060:0:20581:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304260b40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665734528:0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.763182:0:20581:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304260b40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665734656:0@lo:1000:lfsck.0 00000100:00100000:1.0:1787739068.763208:0:20581:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.763238:0:20581:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241600 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.763460:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803780 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.763479:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f712940 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665734016:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.763535:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304261680 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665734784:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.763560:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.763572:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062410c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739068.764463:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.764576:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665734656 00000100:00100000:2.0:1787739068.764582:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665734656 00000100:00100000:3.0:1787739068.764598:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46cd80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.764609:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665734784 00000100:00100000:2.0:1787739068.764612:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665734784 00000100:00100000:2.0:1787739068.764684:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665734144 00000100:00100000:2.0:1787739068.764688:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665734144 00000100:00100000:1.0:1787739068.764691:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3410 00000100:00100000:1.0:1787739068.764704:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304263840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:20581:x1874579665734656:12345-0@lo:1000:lfsck.0 00000100:00100000:2.0:1787739068.764712:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3411 00000100:00100000:2.0:1787739068.764721:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304263480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665734784:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.765075:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304263480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665734784:12345-0@lo:1000:lfsck_namespace.0 Request processed in 354us (1513us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739068.765084:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304263840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:20581:x1874579665734656:12345-0@lo:1000:lfsck.0 Request processed in 383us (1873us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.765088:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3411 00000100:00100000:1.0:1787739068.765097:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3410 00000100:00100000:2.0:1787739068.765115:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3412 00000100:00100000:2.0:1787739068.765125:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073063903c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665734144:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.765147:0:20581:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304260b40 pname:cluuid:pid:xid:nid:opc:job lfsck:lustre-MDT0001-mdtlov_UUID:20581:1874579665734656:0@lo:1000:lfsck.0 00000100:00100000:2.0:1787739068.765188:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073063903c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665734144:12345-0@lo:1000:lfsck_namespace.0 Request processed in 65us (726us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.765194:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3412 00100000:10000000:1.0:1787739068.765285:0:20581:0:(lfsck_engine.c:1053:lfsck_master_engine()) lustre-MDT0001-osd ffffa074126c8000 exit: oit_flags = 0x40003, dir_flags = 0x8006, oit_cookie = 512119, dir_cookie = 0x38efc4ac2165620f, parent = [0x240000401:0x9d:0x0], pid = 20581: rc = 1 00100000:10000000:1.0:1787739068.765306:0:20581:0:(lfsck_lib.c:2594:lfsck_post_generic()) lustre-MDT0001-osd: waiting for assistant to do lfsck_namespace post, rc = 1 00000100:00100000:3.0:1787739068.773043:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304261680 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665734784:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.773104:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f770c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665734144:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.773405:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f770c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665734912:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.773514:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4b40 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665735040:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.773573:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739068.773625:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665735040 00000100:00100000:0.0:1787739068.773632:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665735040 00000100:00100000:0.0:1787739068.773681:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3413 00000100:00100000:0.0:1787739068.773690:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665735040:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.773790:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.773907:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46cd80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739068.774815:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665735040:12345-0@lo:1000:lfsck_namespace.0 Request processed in 1121us (1237us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739068.774828:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3413 00000100:00100000:2.0:1787739068.774919:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665734912 00000100:00100000:2.0:1787739068.774927:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665734912 00000100:00100000:2.0:1787739068.774959:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3414 00000100:00100000:2.0:1787739068.774967:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665734912:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.775156:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665734912:12345-0@lo:1000:lfsck_namespace.0 Request processed in 190us (1364us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.775162:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3414 00000100:00100000:2.0:1787739068.775185:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.775205:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4b40 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665735040:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.775313:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4b40 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665735168:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.775344:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.775358:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739068.775407:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665735168 00000100:00100000:0.0:1787739068.775438:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665735168 00000100:00100000:0.0:1787739068.775463:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3415 00000100:00100000:0.0:1787739068.775470:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665735168:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.775542:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665735168:12345-0@lo:1000:lfsck_namespace.0 Request processed in 71us (196us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739068.775548:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3415 00000100:00100000:2.0:1787739068.775618:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4b40 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665735168:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.775670:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4b40 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665735296:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.775694:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.775714:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665735296 00000100:00100000:2.0:1787739068.775718:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665735296 00000100:00100000:2.0:1787739068.775741:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3416 00000100:00100000:2.0:1787739068.775747:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665735296:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.775804:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665735296:12345-0@lo:1000:lfsck_namespace.0 Request processed in 57us (109us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.775811:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3416 00000100:00100000:2.0:1787739068.775843:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.775855:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4b40 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665735296:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.775968:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4b40 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665735424:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.776004:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.776024:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665735424 00000100:00100000:2.0:1787739068.776027:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665735424 00000100:00100000:2.0:1787739068.776050:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3417 00000100:00100000:2.0:1787739068.776056:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665735424:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.776102:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665735424:12345-0@lo:1000:lfsck_namespace.0 Request processed in 46us (98us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.776109:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3417 00000100:00100000:2.0:1787739068.776124:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.776136:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4b40 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665735424:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.776181:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4b40 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665735552:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.776202:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.776220:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665735552 00000100:00100000:2.0:1787739068.776223:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665735552 00000100:00100000:2.0:1787739068.776240:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3418 00000100:00100000:2.0:1787739068.776245:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157e9e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665735552:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.776286:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157e9e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665735552:12345-0@lo:1000:lfsck_namespace.0 Request processed in 41us (82us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.776292:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3418 00000100:00100000:2.0:1787739068.776308:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.776319:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4b40 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665735552:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.776404:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4b40 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665735680:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.776453:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.776469:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665735680 00000100:00100000:2.0:1787739068.776471:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665735680 00000100:00100000:2.0:1787739068.776488:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3419 00000100:00100000:2.0:1787739068.776493:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665735680:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.776563:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665735680:12345-0@lo:1000:lfsck_namespace.0 Request processed in 69us (110us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739068.776569:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3419 00000100:00100000:2.0:1787739068.776584:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.776595:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4b40 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665735680:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.776684:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4b40 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665735808:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.776710:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.776729:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665735808 00000100:00100000:2.0:1787739068.776732:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665735808 00000100:00100000:2.0:1787739068.776752:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3420 00000100:00100000:2.0:1787739068.776759:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c383c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665735808:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.776833:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c383c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665735808:12345-0@lo:1000:lfsck_namespace.0 Request processed in 75us (122us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739068.776839:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3420 00000100:00100000:2.0:1787739068.776855:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.776866:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4b40 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665735808:0@lo:1000:lfsck_namespace.0 00010000:00010000:2.0:1787739068.776958:0:20585:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ade60 lrc: 3/0,1 mode: --/CW res: [0x200000402:0x13ae:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.776974:0:20585:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ade60 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x13ae:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.777001:0:20585:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ade60 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x13ae:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.777018:0:20585:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ade67 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x13ae:0x0].0xeadb8ace bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.777025:0:20585:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ade67 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13ae:0x0].0xeadb8ace bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.777032:0:20585:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ade67 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13ae:0x0].0xeadb8ace bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.777043:0:20585:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ade67 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13ae:0x0].0xeadb8ace bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.777049:0:20585:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ade67 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x13ae:0x0].0xeadb8ace bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.777056:0:20585:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ade67 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x13ae:0x0].0xeadb8ace bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.777063:0:20585:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d600 already unused, calling callback (ffffffffc0cce440) 00010000:00010000:2.0:1787739068.777066:0:20585:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ade67 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x13ae:0x0].0xeadb8ace bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.777074:0:20585:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ade67 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x13ae:0x0].0xeadb8ace bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.777086:0:20585:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13ae:0x0].0xeadb8ace (ffffa07403a6f400) 00010000:00010000:2.0:1787739068.777094:0:20585:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739068.777098:0:20585:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ade67 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x13ae:0x0].0xeadb8ace bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.777104:0:20585:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ade67 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x13ae:0x0].0xeadb8ace bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.777120:0:20585:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ade60 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x13ae:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.777128:0:20585:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ade60 lrc: 2/0,0 mode: CW/CW res: [0x200000402:0x13ae:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.777135:0:20585:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ade60 lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x13ae:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.777140:0:20585:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633da00 already unused, calling callback (ffffffffc0cce440) 00010000:00010000:2.0:1787739068.777142:0:20585:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ade60 lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x13ae:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739068.777146:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f770c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665734912:0@lo:1000:lfsck_namespace.0 00010000:00010000:2.0:1787739068.777148:0:20585:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ade60 lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x13ae:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.777157:0:20585:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13ae:0x0].0x0 (ffffa07403a6f000) 00010000:00010000:2.0:1787739068.777160:0:20585:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739068.777162:0:20585:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ade60 lrc: 2/0,0 mode: --/CW res: [0x200000402:0x13ae:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.777167:0:20585:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ade60 lrc: 0/0,0 mode: --/CW res: [0x200000402:0x13ae:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00100000:10000000:2.0:1787739068.777177:0:20585:0:(lfsck_namespace.c:5575:lfsck_namespace_repair_dangling()) lustre-MDT0000-osd: namespace LFSCK assistant found dangling reference for: parent [0x200000402:0x13ae:0x0], child [0x240000401:0x15:0x0], type 16877, name [0x240000401:0x15:0x0]:1. Create the lost MDT-object as required: rc = -19 00100000:10000000:2.0:1787739068.777183:0:20585:0:(lfsck_striped_dir.c:2252:lfsck_namespace_striped_dir_rescan()) lustre-MDT0000-osd: namespace LFSCK repair the shard 1 [0x240000401:0x15:0x0] of the striped directory [0x200000402:0x13ae:0x0] with dangling yes/no, rename no/no, llinkea no/no, repair_lmvea no/no: rc = -19 00000100:00100000:3.0:1787739068.777208:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252bc00 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665735936:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.777239:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.777267:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665735936 00000100:00100000:2.0:1787739068.777271:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665735936 00000100:00100000:3.0:1787739068.777280:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46cd80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.777296:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3421 00000100:00100000:2.0:1787739068.777302:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665735936:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.777358:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665735936:12345-0@lo:1000:lfsck_namespace.0 Request processed in 56us (117us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.777367:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3421 00000100:00100000:2.0:1787739068.777405:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665736064:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.777431:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252bc00 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665735936:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.777447:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.777467:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665736064 00000100:00100000:2.0:1787739068.777470:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665736064 00000100:00100000:3.0:1787739068.777471:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252bc00 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665736192:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.777492:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3422 00000100:00100000:3.0:1787739068.777494:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.777499:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c383840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665736064:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.777517:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46cd80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.777547:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c383840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665736064:12345-0@lo:1000:lfsck_namespace.0 Request processed in 49us (99us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.777554:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3422 00000100:00100000:2.0:1787739068.777570:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665736192 00000100:00100000:2.0:1787739068.777574:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665736192 00000100:00100000:2.0:1787739068.777589:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3423 00000100:00100000:2.0:1787739068.777594:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665736192:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.777655:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665736192:12345-0@lo:1000:lfsck_namespace.0 Request processed in 60us (159us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.777661:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3423 00000100:00100000:2.0:1787739068.777679:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.777692:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665736064:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.777741:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665736320:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.777767:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.777790:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665736320 00000100:00100000:2.0:1787739068.777794:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665736320 00000100:00100000:2.0:1787739068.777817:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3424 00000100:00100000:2.0:1787739068.777824:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c380780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665736320:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.777883:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c380780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665736320:12345-0@lo:1000:lfsck_namespace.0 Request processed in 59us (116us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.777889:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3424 00000100:00100000:2.0:1787739068.777908:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.777922:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665736320:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.778006:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665736448:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.778034:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.778056:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665736448 00000100:00100000:2.0:1787739068.778060:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665736448 00000100:00100000:2.0:1787739068.778087:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3425 00000100:00100000:2.0:1787739068.778094:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c3812c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665736448:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.778158:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c3812c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665736448:12345-0@lo:1000:lfsck_namespace.0 Request processed in 65us (124us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739068.778164:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3425 00000100:00100000:2.0:1787739068.778184:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.778200:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665736448:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.778286:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665736576:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.778312:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.778334:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665736576 00000100:00100000:2.0:1787739068.778337:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665736576 00000100:00100000:2.0:1787739068.778359:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3426 00000100:00100000:2.0:1787739068.778364:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c381a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665736576:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.778449:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c381a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665736576:12345-0@lo:1000:lfsck_namespace.0 Request processed in 84us (135us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739068.778459:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3426 00000100:00100000:2.0:1787739068.778478:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.778490:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665736576:0@lo:1000:lfsck_namespace.0 00010000:00010000:2.0:1787739068.778569:0:20585:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ade6e lrc: 3/0,1 mode: --/CW res: [0x200000402:0x1406:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.778581:0:20585:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ade6e lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x1406:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.778590:0:20585:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ade6e lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x1406:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.778602:0:20585:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ade75 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1406:0x0].0xad7a38ea bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.778610:0:20585:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ade75 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1406:0x0].0xad7a38ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.778617:0:20585:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ade75 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1406:0x0].0xad7a38ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.778635:0:20585:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ade75 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1406:0x0].0xad7a38ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.778641:0:20585:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ade75 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1406:0x0].0xad7a38ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.778648:0:20585:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ade75 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1406:0x0].0xad7a38ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.778653:0:20585:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d800 already unused, calling callback (ffffffffc0cce440) 00010000:00010000:2.0:1787739068.778658:0:20585:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ade75 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1406:0x0].0xad7a38ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.778664:0:20585:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ade75 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1406:0x0].0xad7a38ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.778675:0:20585:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1406:0x0].0xad7a38ea (ffffa07403a6fd00) 00010000:00010000:2.0:1787739068.778678:0:20585:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739068.778682:0:20585:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ade75 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1406:0x0].0xad7a38ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.778688:0:20585:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ade75 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1406:0x0].0xad7a38ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.778702:0:20585:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ade6e lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x1406:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.778707:0:20585:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ade6e lrc: 2/0,0 mode: CW/CW res: [0x200000402:0x1406:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.778713:0:20585:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ade6e lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x1406:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.778718:0:20585:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c200 already unused, calling callback (ffffffffc0cce440) 00010000:00010000:2.0:1787739068.778721:0:20585:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ade6e lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x1406:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.778727:0:20585:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ade6e lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x1406:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.778735:0:20585:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1406:0x0].0x0 (ffffa07403a6f100) 00010000:00010000:2.0:1787739068.778738:0:20585:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739068.778741:0:20585:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ade6e lrc: 2/0,0 mode: --/CW res: [0x200000402:0x1406:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.778747:0:20585:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ade6e lrc: 0/0,0 mode: --/CW res: [0x200000402:0x1406:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00100000:10000000:2.0:1787739068.778760:0:20585:0:(lfsck_namespace.c:5575:lfsck_namespace_repair_dangling()) lustre-MDT0000-osd: namespace LFSCK assistant found dangling reference for: parent [0x200000402:0x1406:0x0], child [0x240000401:0x3e:0x0], type 16877, name [0x240000401:0x3e:0x0]:1. Create the lost MDT-object as required: rc = 2 00100000:10000000:2.0:1787739068.778766:0:20585:0:(lfsck_striped_dir.c:2252:lfsck_namespace_striped_dir_rescan()) lustre-MDT0000-osd: namespace LFSCK repair the shard 1 [0x240000401:0x3e:0x0] of the striped directory [0x200000402:0x1406:0x0] with dangling yes/yes, rename no/no, llinkea no/no, repair_lmvea no/no: rc = 2 00000100:00100000:2.0:1787739068.778804:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665736704:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.778834:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.778858:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665736704 00000100:00100000:2.0:1787739068.778862:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665736704 00000100:00100000:2.0:1787739068.778889:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3427 00000100:00100000:2.0:1787739068.778897:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c382580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665736704:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.778951:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c382580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665736704:12345-0@lo:1000:lfsck_namespace.0 Request processed in 55us (117us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.778958:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3427 00000100:00100000:2.0:1787739068.778974:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739068.779395:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252bc00 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665736192:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.779489:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252bc00 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665736832:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.779520:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.779559:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46cd80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.779730:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665736832 00000100:00100000:2.0:1787739068.779734:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665736832 00000100:00100000:2.0:1787739068.779802:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3428 00000100:00100000:2.0:1787739068.779809:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665736832:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.779985:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665736832:12345-0@lo:1000:lfsck_namespace.0 Request processed in 167us (454us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.779993:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3428 00000100:00100000:0.0:1787739068.780053:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252bc00 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665736832:0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.780156:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f7130c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665736960:0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.780185:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739068.780263:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803780 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.780351:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665736960 00000100:00100000:2.0:1787739068.780356:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665736960 00000100:00100000:2.0:1787739068.780388:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3429 00000100:00100000:2.0:1787739068.780394:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665736960:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.780469:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665736960:12345-0@lo:1000:lfsck_namespace.0 Request processed in 74us (282us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.780475:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3429 00000100:00100000:2.0:1787739068.780502:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665736704:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.780572:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665737088:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.780597:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.780615:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665737088 00000100:00100000:2.0:1787739068.780618:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665737088 00000100:00100000:2.0:1787739068.780635:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3430 00000100:00100000:2.0:1787739068.780640:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c3830c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665737088:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.780682:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c3830c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665737088:12345-0@lo:1000:lfsck_namespace.0 Request processed in 42us (84us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.780688:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3430 00000100:00100000:2.0:1787739068.780706:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.780716:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665737088:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.780773:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665737216:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.780795:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.780812:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665737216 00000100:00100000:2.0:1787739068.780815:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665737216 00000100:00100000:2.0:1787739068.780994:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3431 00000100:00100000:2.0:1787739068.781003:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c383480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665737216:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.781050:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c383480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665737216:12345-0@lo:1000:lfsck_namespace.0 Request processed in 49us (254us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739068.781157:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f7130c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665736960:0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.781212:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f7130c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665737344:0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.781237:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739068.781345:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803780 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.781774:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3431 00000100:00100000:2.0:1787739068.781790:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665737344 00000100:00100000:2.0:1787739068.781794:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665737344 00000100:00100000:3.0:1787739068.781854:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3432 00000100:00100000:3.0:1787739068.781863:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665737344:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.781933:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665737344:12345-0@lo:1000:lfsck_namespace.0 Request processed in 71us (694us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739068.781939:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3432 00000100:00100000:0.0:1787739068.782103:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f7130c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665737344:0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.782203:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f7130c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665737472:0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.782229:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739068.782266:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803780 going to sleep for 11 seconds 00000100:00100000:3.0:1787739068.782385:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665737472 00000100:00100000:3.0:1787739068.782390:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665737472 00000100:00100000:3.0:1787739068.782440:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3433 00000100:00100000:3.0:1787739068.782446:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665737472:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.782497:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665737472:12345-0@lo:1000:lfsck_namespace.0 Request processed in 51us (268us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739068.782503:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3433 00000100:00100000:0.0:1787739068.782604:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f7130c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665737472:0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.782655:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f7130c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665737600:0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.782678:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739068.782694:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803780 going to sleep for 11 seconds 00000100:00100000:3.0:1787739068.782767:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665737600 00000100:00100000:3.0:1787739068.782771:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665737600 00000100:00100000:3.0:1787739068.782801:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3434 00000100:00100000:3.0:1787739068.782807:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665737600:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.783029:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665737600:12345-0@lo:1000:lfsck_namespace.0 Request processed in 221us (349us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739068.783036:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3434 00000100:00100000:0.0:1787739068.783091:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f7130c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665737600:0@lo:1000:lfsck_namespace.0 00100000:10000000:0.0:1787739068.783148:0:20583:0:(lfsck_engine.c:1612:lfsck_assistant_engine()) lustre-MDT0001-osd: lfsck_namespace LFSCK assistant thread post 00000100:00100000:0.0:1787739068.783166:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f7130c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665737728:0@lo:1101:lfsck_namespace.0 00000100:00100000:0.0:1787739068.783186:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739068.783207:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803b40 going to sleep for 11 seconds 00100000:10000000:0.0:1787739068.783217:0:20581:0:(lfsck_lib.c:2604:lfsck_post_generic()) lustre-MDT0001-osd: the assistant has done lfsck_namespace post, rc = 1 00100000:10000000:0.0:1787739068.783272:0:20581:0:(lfsck_namespace.c:4808:lfsck_namespace_post()) lustre-MDT0001-osd: namespace LFSCK post done: rc = 0 00100000:10000000:0.0:1787739068.783282:0:20581:0:(lfsck_lib.c:2621:lfsck_double_scan_generic()) lustre-MDT0001-osd: waiting for assistant to do lfsck_namespace double_scan, status 2 00000100:00100000:3.0:1787739068.783320:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665737728 00000100:00100000:3.0:1787739068.783324:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665737728 00000100:00100000:3.0:1787739068.783355:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3435 00000100:00100000:3.0:1787739068.783361:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665737728:12345-0@lo:1101:lfsck_namespace.0 00100000:10000000:3.0:1787739068.783376:0:6528:0:(lfsck_namespace.c:5114:lfsck_namespace_in_notify()) lustre-MDT0000-osd: namespace LFSCK handles notify 3 from MDT 1, status 1, flags 0 00000100:00100000:0.0:1787739068.783721:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f7130c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665737728:0@lo:1101:lfsck_namespace.0 00100000:10000000:0.0:1787739068.783730:0:20583:0:(lfsck_engine.c:1626:lfsck_assistant_engine()) lustre-MDT0001-osd: LFSCK assistant notified others for lfsck_namespace post: rc = 0 00100000:10000000:0.0:1787739068.783737:0:20583:0:(lfsck_engine.c:1644:lfsck_assistant_engine()) lustre-MDT0001-osd: LFSCK assistant sync before the second-stage scaning 00000004:00080000:0.0:1787739068.783753:0:20583:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 655756339527ns, next at 0ns, rc = 0 00080000:00080000:0.0:1787739068.783761:0:20583:0:(osd_handler.c:2849:osd_commit_async()) lustre-MDT0001: async commit OSD 00000004:00080000:0.0:1787739068.783788:0:20583:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 655756374341ns, next at 0ns, rc = 0 00080000:00080000:0.0:1787739068.783793:0:20583:0:(osd_handler.c:2849:osd_commit_async()) lustre-MDT0001: async commit OSD 00100000:10000000:0.0:1787739068.783805:0:20581:0:(lfsck_lib.c:2630:lfsck_double_scan_generic()) lustre-MDT0001-osd: the assistant has done lfsck_namespace double_scan, status 0 00000100:00100000:3.0:1787739068.783867:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665737728:12345-0@lo:1101:lfsck_namespace.0 Request processed in 505us (679us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739068.783875:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3435 00000100:00100000:2.0:1787739068.784076:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.784096:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665737216:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.784228:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665737856:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.784265:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.784288:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739068.784332:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665737856 00000100:00100000:0.0:1787739068.784337:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665737856 00000100:00100000:0.0:1787739068.784366:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3436 00000100:00100000:0.0:1787739068.784372:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c3821c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665737856:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.784548:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c3821c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665737856:12345-0@lo:1000:lfsck_namespace.0 Request processed in 174us (282us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739068.784555:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3436 00000100:00100000:2.0:1787739068.784609:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665737856:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.784710:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665737984:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.784732:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.784762:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739068.784801:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665737984 00000100:00100000:0.0:1787739068.784805:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665737984 00000100:00100000:0.0:1787739068.784836:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3437 00000100:00100000:0.0:1787739068.784843:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c380000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665737984:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.784898:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c380000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665737984:12345-0@lo:1000:lfsck_namespace.0 Request processed in 54us (163us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739068.784904:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3437 00000100:00100000:2.0:1787739068.784965:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665737984:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.785021:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665738112:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.785043:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.785066:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739068.785104:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665738112 00000100:00100000:0.0:1787739068.785109:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665738112 00000100:00100000:0.0:1787739068.785140:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3438 00000100:00100000:0.0:1787739068.785146:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c380f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665738112:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.785194:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c380f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665738112:12345-0@lo:1000:lfsck_namespace.0 Request processed in 49us (150us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739068.785200:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3438 00000100:00100000:2.0:1787739068.785250:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665738112:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.785304:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665738240:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.785325:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.785345:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739068.785381:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665738240 00000100:00100000:0.0:1787739068.785385:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665738240 00000100:00100000:0.0:1787739068.785438:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3439 00000100:00100000:0.0:1787739068.785444:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c381680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665738240:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.785508:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c381680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665738240:12345-0@lo:1000:lfsck_namespace.0 Request processed in 64us (181us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739068.785514:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3439 00000100:00100000:2.0:1787739068.785576:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665738240:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.785617:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665738368:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.785641:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.785665:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739068.785704:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665738368 00000100:00100000:0.0:1787739068.785708:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665738368 00000100:00100000:0.0:1787739068.785767:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3440 00000100:00100000:0.0:1787739068.785774:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665738368:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.785824:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665738368:12345-0@lo:1000:lfsck_namespace.0 Request processed in 50us (182us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739068.785829:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3440 00000100:00100000:2.0:1787739068.785929:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665738368:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.786060:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665738496:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.786084:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.786108:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665738496 00000100:00100000:2.0:1787739068.786112:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665738496 00000100:00100000:2.0:1787739068.786135:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3441 00000100:00100000:2.0:1787739068.786142:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665738496:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.786208:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665738496:12345-0@lo:1000:lfsck_namespace.0 Request processed in 66us (122us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.786214:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3441 00000100:00100000:2.0:1787739068.786233:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.786246:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665738496:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.786306:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665738624:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.786336:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.786355:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665738624 00000100:00100000:2.0:1787739068.786358:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665738624 00000100:00100000:2.0:1787739068.786379:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3442 00000100:00100000:2.0:1787739068.786384:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665738624:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.786451:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665738624:12345-0@lo:1000:lfsck_namespace.0 Request processed in 66us (113us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.786456:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3442 00000100:00100000:2.0:1787739068.786472:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.786483:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c3803c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665738624:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.786541:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432f840 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665738752:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.786564:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.786582:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665738752 00000100:00100000:2.0:1787739068.786585:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665738752 00000100:00100000:2.0:1787739068.786603:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3443 00000100:00100000:2.0:1787739068.786608:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665738752:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.786651:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665738752:12345-0@lo:1000:lfsck_namespace.0 Request processed in 42us (86us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.786656:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3443 00000100:00100000:2.0:1787739068.786671:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.786681:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432f840 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665738752:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.786735:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432f840 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665738880:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.786756:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.786772:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665738880 00000100:00100000:2.0:1787739068.786776:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665738880 00000100:00100000:2.0:1787739068.786795:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3444 00000100:00100000:2.0:1787739068.786801:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665738880:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.786840:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665738880:12345-0@lo:1000:lfsck_namespace.0 Request processed in 39us (83us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.786846:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3444 00000100:00100000:2.0:1787739068.786861:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.786872:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432f840 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665738880:0@lo:1000:lfsck_namespace.0 00100000:10000000:2.0:1787739068.786919:0:20585:0:(lfsck_engine.c:1612:lfsck_assistant_engine()) lustre-MDT0000-osd: lfsck_namespace LFSCK assistant thread post 00000100:00100000:2.0:1787739068.786936:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432f840 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665739008:0@lo:1101:lfsck_namespace.0 00000100:00100000:2.0:1787739068.786960:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.786991:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665739008 00000100:00100000:2.0:1787739068.786995:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665739008 00000100:00100000:2.0:1787739068.787016:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3445 00000100:00100000:2.0:1787739068.787021:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665739008:12345-0@lo:1101:lfsck_namespace.0 00100000:10000000:2.0:1787739068.787036:0:6529:0:(lfsck_namespace.c:5114:lfsck_namespace_in_notify()) lustre-MDT0001-osd: namespace LFSCK handles notify 3 from MDT 0, status 1, flags 2 00000100:00100000:2.0:1787739068.787068:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665739008:12345-0@lo:1101:lfsck_namespace.0 Request processed in 46us (106us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.787073:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3445 00000100:00100000:2.0:1787739068.787090:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887840 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.787104:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432f840 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665739008:0@lo:1101:lfsck_namespace.0 00100000:10000000:2.0:1787739068.787113:0:20585:0:(lfsck_engine.c:1626:lfsck_assistant_engine()) lustre-MDT0000-osd: LFSCK assistant notified others for lfsck_namespace post: rc = 0 00100000:10000000:3.0:1787739068.787673:0:20580:0:(lfsck_lib.c:2604:lfsck_post_generic()) lustre-MDT0000-osd: the assistant has done lfsck_namespace post, rc = 1 00100000:10000000:3.0:1787739068.787764:0:20580:0:(lfsck_namespace.c:4808:lfsck_namespace_post()) lustre-MDT0000-osd: namespace LFSCK post done: rc = 0 00100000:10000000:3.0:1787739068.787773:0:20580:0:(lfsck_lib.c:2621:lfsck_double_scan_generic()) lustre-MDT0000-osd: waiting for assistant to do lfsck_namespace double_scan, status 2 00100000:10000000:2.0:1787739068.790115:0:20585:0:(lfsck_engine.c:1644:lfsck_assistant_engine()) lustre-MDT0000-osd: LFSCK assistant sync before the second-stage scaning 00000004:00080000:2.0:1787739068.790144:0:20585:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 655762729414ns, next at 533789884169ns, rc = 0 00080000:00080000:2.0:1787739068.790159:0:20585:0:(osd_handler.c:2849:osd_commit_async()) lustre-MDT0000: async commit OSD 00000004:00080000:2.0:1787739068.790442:0:20585:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 655763027379ns, next at 521673230421ns, rc = 0 00080000:00080000:2.0:1787739068.790448:0:20585:0:(osd_handler.c:2849:osd_commit_async()) lustre-MDT0000: async commit OSD 00100000:10000000:0.0:1787739068.793636:0:20580:0:(lfsck_lib.c:2630:lfsck_double_scan_generic()) lustre-MDT0000-osd: the assistant has done lfsck_namespace double_scan, status 0 00000004:00080000:1.0:1787739068.793831:0:7453:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 2 committed 00000004:00080000:1.0:1787739068.793847:0:7453:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 2 committed 00000001:00080000:1.0:1787739068.793881:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968425 is committed 00000020:00080000:1.0:1787739068.793888:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 380 batchid updated 00000020:00080000:1.0:1787739068.793896:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 378 batchid updated 00000020:00080000:1.0:1787739068.793916:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 380 00000004:00080000:1.0:1787739068.793972:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07306241600 with version 709 00000020:00080000:1.0:1787739068.794156:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 378 cancel update log [0x200000401:0x3:0x0].370: rc = 0 00000020:00080000:1.0:1787739068.794229:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 378 cancel update log [0x240000400:0x3:0x0].370: rc = 0 00000004:00080000:1.0:1787739068.794258:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07306241c00 with version 710 00000020:00080000:1.0:1787739068.794262:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 379 cancel update log [0x200000401:0x3:0x0].371: rc = 0 00000020:00080000:1.0:1787739068.794282:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 379 cancel update log [0x240000400:0x3:0x0].371: rc = 0 00000020:00080000:1.0:1787739068.794290:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 380 00100000:10000000:1.0:1787739068.794301:0:20583:0:(lfsck_engine.c:1651:lfsck_assistant_engine()) lustre-MDT0001-osd: LFSCK assistant phase2 scan start, synced: rc = 0 00100000:10000000:1.0:1787739068.794312:0:20583:0:(lfsck_namespace.c:6700:lfsck_namespace_assistant_handler_p2()) lustre-MDT0001-osd: namespace LFSCK phase2 scan start 00100000:10000000:1.0:1787739068.794377:0:20583:0:(lfsck_namespace.c:6325:lfsck_namespace_scan_local_lpf()) lustre-MDT0001-osd: start to scan backend /lost+found 00100000:10000000:1.0:1787739068.794512:0:20583:0:(lfsck_namespace.c:6428:lfsck_namespace_scan_local_lpf()) lustre-MDT0001-osd: stop to scan backend /lost+found: rc = 1 00000100:00100000:1.0:1787739068.794602:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304260b40 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665739136:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.794669:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.794708:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665739136 00000100:00100000:1.0:1787739068.794714:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665739136 00000100:00100000:1.0:1787739068.794758:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3446 00000100:00100000:1.0:1787739068.794765:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304261680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665739136:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.794850:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304261680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665739136:12345-0@lo:1000:lfsck_namespace.0 Request processed in 85us (181us total) trans 0 rc 1/1 00000100:00100000:1.0:1787739068.794858:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3446 00000100:00100000:1.0:1787739068.794892:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062410c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.794914:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304260b40 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665739136:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.794974:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304260b40 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665739264:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.795010:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.795034:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665739264 00000100:00100000:1.0:1787739068.795037:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665739264 00000100:00100000:1.0:1787739068.795059:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3447 00000100:00100000:1.0:1787739068.795064:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356acb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665739264:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.795114:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356acb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665739264:12345-0@lo:1000:lfsck_namespace.0 Request processed in 50us (103us total) trans 0 rc 1/1 00000100:00100000:1.0:1787739068.795124:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3447 00000100:00100000:1.0:1787739068.795140:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.795152:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304260b40 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665739264:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.795213:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356af840 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665739392:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.795238:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.795255:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665739392 00000100:00100000:1.0:1787739068.795259:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665739392 00000100:00100000:1.0:1787739068.795279:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3448 00000100:00100000:1.0:1787739068.795285:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410b09450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665739392:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.795332:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410b09450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665739392:12345-0@lo:1000:lfsck_namespace.0 Request processed in 46us (93us total) trans 0 rc 1/1 00000100:00100000:1.0:1787739068.795337:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3448 00000100:00100000:1.0:1787739068.795556:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a459180 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.795567:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356af840 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665739392:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.795609:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356af840 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665739520:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.795673:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.795689:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665739520 00000100:00100000:1.0:1787739068.795692:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665739520 00000100:00100000:1.0:1787739068.795709:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3449 00000100:00100000:1.0:1787739068.795714:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356acb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665739520:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.795752:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356acb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665739520:12345-0@lo:1000:lfsck_namespace.0 Request processed in 37us (78us total) trans 0 rc 1/1 00000100:00100000:1.0:1787739068.795758:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3449 00000100:00100000:1.0:1787739068.795773:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062410c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.795785:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356af840 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665739520:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.795831:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356af840 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665739648:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.795853:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.795871:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665739648 00000100:00100000:1.0:1787739068.795874:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665739648 00000100:00100000:1.0:1787739068.795895:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3450 00000100:00100000:1.0:1787739068.795900:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665739648:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.795950:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665739648:12345-0@lo:1000:lfsck_namespace.0 Request processed in 50us (95us total) trans 0 rc 1/1 00000100:00100000:1.0:1787739068.795956:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3450 00000100:00100000:1.0:1787739068.795970:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.795992:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356af840 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665739648:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.796052:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356af840 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665739776:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.796075:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.796093:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665739776 00000100:00100000:1.0:1787739068.796096:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665739776 00000100:00100000:1.0:1787739068.796115:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3451 00000100:00100000:1.0:1787739068.796120:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665739776:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.796166:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665739776:12345-0@lo:1000:lfsck_namespace.0 Request processed in 45us (90us total) trans 0 rc 1/1 00000100:00100000:1.0:1787739068.796172:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3451 00000100:00100000:1.0:1787739068.796187:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a459180 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.796198:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356af840 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665739776:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.796264:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356af840 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665739904:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.796287:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.796302:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665739904 00000100:00100000:1.0:1787739068.796305:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665739904 00000100:00100000:1.0:1787739068.796321:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3452 00000100:00100000:1.0:1787739068.796326:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ada40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665739904:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.796368:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ada40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665739904:12345-0@lo:1000:lfsck_namespace.0 Request processed in 42us (80us total) trans 0 rc 1/1 00000100:00100000:1.0:1787739068.796374:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3452 00000100:00100000:1.0:1787739068.796387:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062410c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.796396:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356af840 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665739904:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.796454:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356af840 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665740032:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.796474:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.796489:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665740032 00000100:00100000:1.0:1787739068.796491:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665740032 00000100:00100000:1.0:1787739068.796507:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3453 00000100:00100000:1.0:1787739068.796512:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665740032:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.796550:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665740032:12345-0@lo:1000:lfsck_namespace.0 Request processed in 37us (74us total) trans 0 rc 1/1 00000100:00100000:1.0:1787739068.796555:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3453 00000100:00100000:1.0:1787739068.796568:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.796577:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356af840 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665740032:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.796631:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356af840 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665740160:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.796652:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.796669:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665740160 00000100:00100000:1.0:1787739068.796671:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665740160 00000100:00100000:1.0:1787739068.796691:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3454 00000100:00100000:1.0:1787739068.796697:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356aed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665740160:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.796741:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356aed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:20583:x1874579665740160:12345-0@lo:1000:lfsck_namespace.0 Request processed in 44us (88us total) trans 0 rc 1/1 00000100:00100000:1.0:1787739068.796747:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3454 00000004:00080000:2.0:1787739068.796964:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 709 rpc_version 709 00000020:00100000:2.0:1787739068.796994:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739068.797003:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0730432f840 x1874579665740416/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739068.797017:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432f840 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665740416:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739068.797045:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.797223:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a459180 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.797239:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356af840 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665740160:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.797309:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356af840 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665740544:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.797370:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887840 going to sleep for 11 seconds 00000100:00100000:0.0:1787739068.797465:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665740416 00000100:00100000:0.0:1787739068.797469:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665740416 00000100:00100000:0.0:1787739068.797504:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3455 00000100:00100000:0.0:1787739068.797511:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:7467:x1874579665740416:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739068.797639:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665740416:12345-0@lo:1000:osp_up0-1.0 Request processed in 128us (592us total) trans 4294983435 rc 0/0 00000100:00100000:0.0:1787739068.797646:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3455 00000100:00100000:2.0:1787739068.797896:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432f840 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665740416:0@lo:1000:osp_up0-1.0 00000004:00080000:2.0:1787739068.797907:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 710 rpc_version 710 00000020:00100000:2.0:1787739068.797919:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739068.797924:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0730432c000 x1874579665740800/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739068.797938:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432c000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665740800:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739068.797962:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.798026:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887840 going to sleep for 11 seconds 00000004:00080000:3.0:1787739068.798903:0:6502:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 74 committed 00000020:00080000:3.0:1787739068.798941:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 389 batchid updated 00000020:00080000:3.0:1787739068.799089:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 389 00000004:00080000:3.0:1787739068.799140:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0740a46cb40 with version 740 00000020:00080000:3.0:1787739068.799152:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 389 cancel update log [0x240000401:0x3:0x0].386: rc = 0 00000020:00080000:3.0:1787739068.799214:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 389 cancel update log [0x200000400:0x3:0x0].387: rc = 0 00000020:00080000:3.0:1787739068.799226:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 389 00000004:00080000:1.0:1787739068.799443:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 740 rpc_version 740 00000020:00100000:1.0:1787739068.799469:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739068.799476:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074356ae940 x1874579665741056/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739068.799496:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae940 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665741056:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739068.799547:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.799779:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a459180 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.799803:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739068.799840:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665740800 00000100:00100000:0.0:1787739068.799848:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665740800 00000100:00100000:0.0:1787739068.799893:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665741056 00000100:00100000:0.0:1787739068.799896:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665741056 00000100:00100000:0.0:1787739068.799913:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3456 00000100:00100000:0.0:1787739068.799920:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665740800:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739068.800101:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3457 00000100:00100000:3.0:1787739068.800108:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:7740:x1874579665741056:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739068.800218:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:7740:x1874579665741056:12345-0@lo:1000:osp_up1-0.0 Request processed in 110us (669us total) trans 4294968426 rc 0/0 00000100:00100000:3.0:1787739068.800225:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3457 00000100:00100000:0.0:1787739068.800463:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665740800:12345-0@lo:1000:osp_up0-1.0 Request processed in 543us (2500us total) trans 4294983436 rc 0/0 00000100:00100000:0.0:1787739068.800470:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3456 00000100:00100000:3.0:1787739068.800661:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432c000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665740800:0@lo:1000:osp_up0-1.0 00000001:00080000:3.0:1787739068.801397:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983435 is committed 00000004:00080000:3.0:1787739068.801405:0:6502:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 71 committed 00100000:10000000:3.0:1787739068.801453:0:20585:0:(lfsck_engine.c:1651:lfsck_assistant_engine()) lustre-MDT0000-osd: LFSCK assistant phase2 scan start, synced: rc = 0 00100000:10000000:3.0:1787739068.801465:0:20585:0:(lfsck_namespace.c:6700:lfsck_namespace_assistant_handler_p2()) lustre-MDT0000-osd: namespace LFSCK phase2 scan start 00100000:10000000:3.0:1787739068.801544:0:20585:0:(lfsck_namespace.c:6325:lfsck_namespace_scan_local_lpf()) lustre-MDT0000-osd: start to scan backend /lost+found 00100000:10000000:3.0:1787739068.801575:0:20585:0:(lfsck_namespace.c:6428:lfsck_namespace_scan_local_lpf()) lustre-MDT0000-osd: stop to scan backend /lost+found: rc = 1 00000100:00100000:3.0:1787739068.801676:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252a580 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665741184:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.802491:0:7740:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968425 gen 2 00000004:00080000:1.0:1787739068.802499:0:7740:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968424 committed transno 4294968425 00000020:00080000:1.0:1787739068.802505:0:7740:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa0730610e9c0 refcount 2 committed 0 result 0 batchid 390 00000020:00080000:1.0:1787739068.802509:0:7740:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07404f86400 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730610e0c0 00000020:00080000:1.0:1787739068.802513:0:7740:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].387 00000020:00080000:1.0:1787739068.802516:0:7740:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07404f86e00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073069bee00 00000020:00080000:1.0:1787739068.802519:0:7740:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].388 00000004:00080000:1.0:1787739068.802541:0:7740:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968425 committed transno 4294968425 00000100:00100000:1.0:1787739068.802557:0:7740:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffffa074356ae940 x1874579665741056/t4294968426(4294968426) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 1787739084 ref 3 fl Rpc:RQU/204/0 rc 0/0 job:'osp_up1-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787739068.802576:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae940 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665741056:0@lo:1000:osp_up1-0.0 00000020:00080000:1.0:1787739068.802837:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 390 committed batchid 389 00000020:00080000:1.0:1787739068.802890:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 389 00000100:00100000:3.0:1787739068.805106:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.806467:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062410c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739068.806743:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46c0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.806806:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665740544 00000100:00100000:2.0:1787739068.806815:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665740544 00000100:00100000:2.0:1787739068.806855:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665741184 00000100:00100000:2.0:1787739068.806858:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665741184 00000100:00100000:2.0:1787739068.806874:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3458 00000100:00100000:2.0:1787739068.806882:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ae580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:20583:x1874579665740544:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.807018:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ae580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:20583:x1874579665740544:12345-0@lo:1000:lfsck_namespace.0 Request processed in 136us (7213us total) trans 0 rc 1/1 00000100:00100000:2.0:1787739068.807026:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3458 00000100:00100000:2.0:1787739068.807043:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3459 00000100:00100000:2.0:1787739068.807049:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:20585:x1874579665741184:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.807112:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:20585:x1874579665741184:12345-0@lo:1000:lfsck_namespace.0 Request processed in 63us (2009us total) trans 0 rc 1/1 00000100:00100000:2.0:1787739068.807119:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3459 00000100:00100000:3.0:1787739068.807206:0:20583:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983435 gen 2 00000004:00080000:3.0:1787739068.807212:0:20583:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983435 committed transno 4294983435 00000100:00100000:3.0:1787739068.807234:0:20583:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffffa0730432c000 x1874579665740800/t4294983436(4294983436) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 1787739084 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'osp_up0-1.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787739068.807246:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356af840 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665740544:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.807388:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252b0c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665741312:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.807450:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.807472:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739068.807494:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252a580 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665741184:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.807557:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252a580 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665741440:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.807582:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.807594:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241900 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.807726:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665741312 00000100:00100000:2.0:1787739068.807730:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665741312 00000100:00100000:2.0:1787739068.807748:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665741440 00000100:00100000:2.0:1787739068.807751:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665741440 00000100:00100000:2.0:1787739068.807762:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3460 00000100:00100000:2.0:1787739068.807767:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:20583:x1874579665741312:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.807830:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:20583:x1874579665741312:12345-0@lo:1000:lfsck_namespace.0 Request processed in 61us (380us total) trans 0 rc 1/1 00000100:00100000:2.0:1787739068.807836:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3460 00000100:00100000:2.0:1787739068.807848:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3461 00000100:00100000:2.0:1787739068.807854:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:20585:x1874579665741440:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.807905:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:20585:x1874579665741440:12345-0@lo:1000:lfsck_namespace.0 Request processed in 51us (321us total) trans 0 rc 1/1 00000100:00100000:2.0:1787739068.807911:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3461 00000100:00100000:2.0:1787739068.807935:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252a580 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665741440:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.808017:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665741568:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.808043:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.808061:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665741568 00000100:00100000:2.0:1787739068.808064:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665741568 00000100:00100000:2.0:1787739068.808080:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3462 00000100:00100000:2.0:1787739068.808084:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:20585:x1874579665741568:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.808129:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:20585:x1874579665741568:12345-0@lo:1000:lfsck_namespace.0 Request processed in 43us (85us total) trans 0 rc 1/1 00000100:00100000:2.0:1787739068.808134:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3462 00000100:00100000:2.0:1787739068.808149:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068879c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.808159:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665741568:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.808225:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665741696:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.808248:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.808266:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665741696 00000100:00100000:2.0:1787739068.808268:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665741696 00000100:00100000:2.0:1787739068.808285:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3463 00000100:00100000:2.0:1787739068.808290:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:20585:x1874579665741696:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.808335:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:20585:x1874579665741696:12345-0@lo:1000:lfsck_namespace.0 Request processed in 44us (86us total) trans 0 rc 1/1 00000100:00100000:2.0:1787739068.808341:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3463 00000100:00100000:2.0:1787739068.808355:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068879c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.808366:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665741696:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.808440:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665741824:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.808463:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.808482:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665741824 00000100:00100000:2.0:1787739068.808485:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665741824 00000100:00100000:2.0:1787739068.808501:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3464 00000100:00100000:2.0:1787739068.808507:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:20585:x1874579665741824:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.808555:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:20585:x1874579665741824:12345-0@lo:1000:lfsck_namespace.0 Request processed in 48us (91us total) trans 0 rc 1/1 00000100:00100000:2.0:1787739068.808561:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3464 00000100:00100000:2.0:1787739068.808575:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068879c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.808586:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665741824:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.808732:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252b0c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665741312:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.808810:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252b0c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665741952:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.808835:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.808867:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46c0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.808913:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665741952 00000100:00100000:2.0:1787739068.808917:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665741952 00000100:00100000:2.0:1787739068.808933:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3465 00000100:00100000:2.0:1787739068.808939:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:20583:x1874579665741952:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.809051:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:20583:x1874579665741952:12345-0@lo:1000:lfsck_namespace.0 Request processed in 111us (214us total) trans 0 rc 1/1 00000100:00100000:2.0:1787739068.809057:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3465 00000100:00100000:2.0:1787739068.809097:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665742080:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.809120:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.809137:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665742080 00000100:00100000:2.0:1787739068.809140:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665742080 00000100:00100000:2.0:1787739068.809155:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3466 00000100:00100000:2.0:1787739068.809159:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:20585:x1874579665742080:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.809203:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:20585:x1874579665742080:12345-0@lo:1000:lfsck_namespace.0 Request processed in 43us (81us total) trans 0 rc 1/1 00000100:00100000:2.0:1787739068.809209:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3466 00000100:00100000:2.0:1787739068.809222:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068879c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.809236:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665742080:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.809284:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665742208:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.809307:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.809325:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665742208 00000100:00100000:2.0:1787739068.809329:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665742208 00000100:00100000:2.0:1787739068.809344:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3467 00000100:00100000:2.0:1787739068.809349:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:20585:x1874579665742208:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.809395:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:20585:x1874579665742208:12345-0@lo:1000:lfsck_namespace.0 Request processed in 45us (86us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739068.809401:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3467 00000100:00100000:2.0:1787739068.809434:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068879c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.809445:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665742208:0@lo:1000:lfsck_namespace.0 00010000:00010000:2.0:1787739068.809478:0:20585:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ade7c lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.809495:0:20585:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ade7c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.809506:0:20585:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ade7c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739068.809536:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665742336:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.809561:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.809580:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665742336 00000100:00100000:2.0:1787739068.809584:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665742336 00000100:00100000:2.0:1787739068.809600:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3468 00000100:00100000:2.0:1787739068.809605:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:20585:x1874579665742336:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.809763:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:20585:x1874579665742336:12345-0@lo:1000:lfsck_namespace.0 Request processed in 157us (200us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739068.809769:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3468 00000100:00100000:2.0:1787739068.809785:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068879c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.809796:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665742336:0@lo:1000:lfsck_namespace.0 00010000:00010000:2.0:1787739068.809810:0:20585:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ade7c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.809819:0:20585:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ade7c lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.809826:0:20585:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ade7c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.809833:0:20585:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633c200 already unused, calling callback (ffffffffc0cce440) 00010000:00010000:2.0:1787739068.809836:0:20585:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ade7c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.809844:0:20585:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ade7c lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.809857:0:20585:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1406:0x0].0x0 (ffffa074419d5b00) 00010000:00010000:2.0:1787739068.809860:0:20585:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 22 00010000:00010000:2.0:1787739068.809864:0:20585:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ade7c lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.809870:0:20585:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c200/0x7ee4453e4a1ade7c lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.809917:0:20585:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1ade83 lrc: 3/0,1 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.809924:0:20585:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1ade83 lrc: 3/0,1 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.809933:0:20585:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1ade83 lrc: 3/0,1 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739068.809944:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665742464:0@lo:101:lfsck_namespace.0 00000100:00100000:2.0:1787739068.809965:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.809997:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665742464 00000100:00100000:2.0:1787739068.810000:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665742464 00000100:00100000:2.0:1787739068.810017:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3469 00000100:00100000:2.0:1787739068.810023:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:20585:x1874579665742464:12345-0@lo:101:lfsck_namespace.0 00010000:00010000:2.0:1787739068.810031:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739068.810043:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ade8a lrc: 2/0,0 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ade83 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.810064:0:6529:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adb49 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951ae expref: 6 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.810077:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ade8a lrc: 4/0,0 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ade83 expref: 7 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.810097:0:6529:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adb49 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951ae expref: 6 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.810110:0:6529:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adb49 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951ae expref: 6 pid: 6524 timeout: 755 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739068.810121:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432e580 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665742592:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739068.810178:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068879c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.810187:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739068.814263:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252b0c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665741952:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.814448:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252b0c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665742720:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.814513:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.814556:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46c0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739068.814658:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665742720 00000100:00100000:0.0:1787739068.814665:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665742720 00000100:00100000:0.0:1787739068.814705:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3470 00000100:00100000:0.0:1787739068.814714:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:20583:x1874579665742720:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.814796:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:20583:x1874579665742720:12345-0@lo:1000:lfsck_namespace.0 Request processed in 83us (283us total) trans 0 rc 1/1 00000100:00100000:0.0:1787739068.814804:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3470 00000100:00100000:3.0:1787739068.814924:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252b0c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665742720:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.815002:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252b0c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665742848:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.815027:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.815054:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46c0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739068.815100:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665742848 00000100:00100000:0.0:1787739068.815104:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665742848 00000100:00100000:0.0:1787739068.815120:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3471 00000100:00100000:0.0:1787739068.815125:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:20583:x1874579665742848:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.815184:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:20583:x1874579665742848:12345-0@lo:1000:lfsck_namespace.0 Request processed in 58us (155us total) trans 0 rc 1/1 00000100:00100000:0.0:1787739068.815191:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3471 00000100:00100000:3.0:1787739068.815246:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252b0c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665742848:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.815296:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252b0c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665742976:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.815317:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.815340:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46c0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739068.815383:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665742976 00000100:00100000:0.0:1787739068.815387:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665742976 00000100:00100000:0.0:1787739068.815404:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3472 00000100:00100000:0.0:1787739068.815410:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:20583:x1874579665742976:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.815491:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:20583:x1874579665742976:12345-0@lo:1000:lfsck_namespace.0 Request processed in 80us (171us total) trans 0 rc 1/1 00000100:00100000:0.0:1787739068.815498:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3472 00000100:00100000:3.0:1787739068.815554:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252b0c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665742976:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.815614:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252b0c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665743104:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.815649:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.815672:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46c0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739068.815766:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665743104 00000100:00100000:0.0:1787739068.815770:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665743104 00000100:00100000:0.0:1787739068.815789:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3473 00000100:00100000:0.0:1787739068.815795:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:20583:x1874579665743104:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.815858:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:20583:x1874579665743104:12345-0@lo:1000:lfsck_namespace.0 Request processed in 63us (208us total) trans 0 rc 1/1 00000100:00100000:0.0:1787739068.815864:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3473 00000100:00100000:3.0:1787739068.815921:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252b0c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665743104:0@lo:1000:lfsck_namespace.0 00100000:10000000:3.0:1787739068.815939:0:20583:0:(lfsck_namespace.c:6719:lfsck_namespace_assistant_handler_p2()) lustre-MDT0001-osd: namespace LFSCK phase2 scan stop at the No. 16 trace file: rc = 1 00000100:00100000:3.0:1787739068.815956:0:20583:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252b0c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665743232:0@lo:1101:lfsck_namespace.0 00000100:00100000:3.0:1787739068.815991:0:20583:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.816013:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46cc00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739068.816057:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665743232 00000100:00100000:0.0:1787739068.816061:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665743232 00000100:00100000:0.0:1787739068.816078:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3474 00000100:00100000:0.0:1787739068.816083:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f743c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:20583:x1874579665743232:12345-0@lo:1101:lfsck_namespace.0 00100000:10000000:0.0:1787739068.816098:0:6528:0:(lfsck_namespace.c:5114:lfsck_namespace_in_notify()) lustre-MDT0000-osd: namespace LFSCK handles notify 4 from MDT 1, status 1, flags 0 00000100:00100000:0.0:1787739068.816141:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f743c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:20583:x1874579665743232:12345-0@lo:1101:lfsck_namespace.0 Request processed in 57us (149us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739068.816147:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3474 00000100:00100000:3.0:1787739068.816334:0:20583:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252b0c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0001-mdtlov_UUID:20583:1874579665743232:0@lo:1101:lfsck_namespace.0 00000100:00100000:3.0:1787739068.816340:0:20583:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46cc00 going to sleep for 0 seconds 00100000:10000000:3.0:1787739068.816348:0:20583:0:(lfsck_engine.c:1760:lfsck_assistant_engine()) lustre-MDT0001-osd: LFSCK assistant sync before exit 00000004:00080000:3.0:1787739068.816371:0:20583:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 655788957284ns, next at 0ns, rc = 0 00080000:00080000:3.0:1787739068.816385:0:20583:0:(osd_handler.c:2849:osd_commit_async()) lustre-MDT0001: async commit OSD 00000004:00080000:0.0:1787739068.818466:0:20583:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 655791048640ns, next at 0ns, rc = 0 00080000:00080000:0.0:1787739068.818482:0:20583:0:(osd_handler.c:2849:osd_commit_async()) lustre-MDT0001: async commit OSD 00000004:00080000:0.0:1787739068.823767:0:7453:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 3 committed 00000001:00080000:0.0:1787739068.823788:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968426 is committed 00000004:00080000:1.0:1787739068.826604:0:7453:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 3 committed 00100000:10000000:3.0:1787739068.832324:0:20583:0:(lfsck_engine.c:1766:lfsck_assistant_engine()) lustre-MDT0001-osd: LFSCK assistant synced before exit: rc = 0 00100000:10000000:3.0:1787739068.832512:0:20583:0:(lfsck_engine.c:1778:lfsck_assistant_engine()) lustre-MDT0001-osd: LFSCK assistant phase2 scan finished: rc = 1 00100000:10000000:3.0:1787739068.832528:0:20583:0:(lfsck_engine.c:1790:lfsck_assistant_engine()) lustre-MDT0001-osd: lfsck_namespace LFSCK assistant thread exit: rc = 1 00000100:00100000:2.0:1787739068.832814:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432e580 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665742592:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739068.832847:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ade8a lrc: 3/0,0 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ade83 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.832861:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:2.0:1787739068.832864:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739068.832868:0:6529:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ade8a lrc: 3/0,0 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ade83 expref: 6 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.832878:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633da00, rc 0) 00000100:00100000:2.0:1787739068.832931:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:20585:x1874579665742464:12345-0@lo:101:lfsck_namespace.0 Request processed in 22907us (22963us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.832939:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3469 00000100:00100000:2.0:1787739068.832992:0:20585:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968426 gen 2 00000004:00080000:2.0:1787739068.832998:0:20585:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968426 committed transno 4294968426 00000100:00100000:2.0:1787739068.833013:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665742464:0@lo:101:lfsck_namespace.0 00010000:00010000:2.0:1787739068.833031:0:20585:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1ade83 lrc: 4/0,1 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ade8a expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.833042:0:20585:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1ade83 lrc: 4/0,1 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ade8a expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739068.839583:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739068.839623:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652506240 00000100:00100000:3.0:1787739068.839634:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652506240 00010000:00010000:3.0:1787739068.839689:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adb49 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951ae expref: 7 pid: 6524 timeout: 755 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739068.839727:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1566 00000100:00100000:3.0:1787739068.839781:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f75e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2409:x1874579652506240:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739068.839795:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739068.839802:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adb49 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951ae expref: 7 pid: 6524 timeout: 755 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.839817:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adb49 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951ae expref: 7 pid: 6524 timeout: 755 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.839839:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adb49 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0951ae expref: 7 pid: 6524 timeout: 755 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.839852:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:3.0:1787739068.839858:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ade8a lrc: 2/0,0 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ade83 expref: 5 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.839870:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ade8a lrc: 2/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ade83 expref: 5 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.839885:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ade8a lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ade83 expref: 5 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.839914:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ade8a lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ade83 expref: 5 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739068.839928:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74b40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665743360:0@lo:105: 00000100:00100000:3.0:1787739068.839972:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.840022:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46cc00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739068.840038:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665743360 00000100:00100000:3.0:1787739068.840041:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665743360 00000100:00100000:3.0:1787739068.840064:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1197 00000100:00100000:3.0:1787739068.840070:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f770c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665743360:12345-0@lo:105: 00010000:00010000:3.0:1787739068.840080:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1ade83 lrc: 5/0,1 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ade8a expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.840091:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1ade83 lrc: 5/0,1 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ade8a expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.840099:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1ade83 lrc: 5/0,1 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ade8a expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.840107:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1ade83 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ade8a expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.840113:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1ade83 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ade8a expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.840122:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1ade83 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ade8a expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.840138:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa0730633d800) 00000100:00100000:3.0:1787739068.840169:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f770c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665743360:12345-0@lo:105: Request processed in 98us (195us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739068.840176:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1197 00000100:00100000:3.0:1787739068.840209:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74b40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665743360:0@lo:105: 00010000:00010000:3.0:1787739068.840222:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739068.840251:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f75e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2409:x1874579652506240:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 474us (676us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739068.840258:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1566 00000100:00080000:1.0:1787739068.840445:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:3.0:1787739068.841474:0:20585:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1ade83 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ade8a expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.841490:0:20585:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1ade83 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ade8a expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739068.841512:0:20585:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739068.841549:0:20585:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ade91 lrc: 3/0,1 mode: --/EX res: [0x240000401:0x5:0x0].0x23e4280e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.841556:0:20585:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ade91 lrc: 3/0,1 mode: --/EX res: [0x240000401:0x5:0x0].0x23e4280e bits 0x23/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.841566:0:20585:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ade91 lrc: 3/0,1 mode: --/EX res: [0x240000401:0x5:0x0].0x23e4280e bits 0x23/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739068.841586:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74b40 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665743488:0@lo:101:lfsck_namespace.0 00000100:00100000:3.0:1787739068.841630:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.841660:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46cc00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.841836:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665743488 00000100:00100000:1.0:1787739068.841842:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665743488 00000100:00100000:1.0:1787739068.841878:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3475 00000100:00100000:1.0:1787739068.841885:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:20585:x1874579665743488:12345-0@lo:101:lfsck_namespace.0 00010000:00010000:1.0:1787739068.841899:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739068.841913:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fcc00/0x7ee4453e4a1ade98 lrc: 2/0,0 mode: --/EX res: [0x240000401:0x5:0x0].0x23e4280e bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ade91 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.841935:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fcc00/0x7ee4453e4a1ade98 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x23e4280e bits 0x23/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ade91 expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.841948:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fcc00/0x7ee4453e4a1ade98 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x23e4280e bits 0x23/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ade91 expref: 7 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.841957:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x23e4280e (ffffa07305eaeb00) 00010000:00010000:1.0:1787739068.841960:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 23 00010000:00010000:1.0:1787739068.841963:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fcc00, rc 0) 00000100:00100000:1.0:1787739068.842008:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:20585:x1874579665743488:12345-0@lo:101:lfsck_namespace.0 Request processed in 123us (377us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739068.842014:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3475 00000100:00100000:1.0:1787739068.842045:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74b40 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665743488:0@lo:101:lfsck_namespace.0 00010000:00010000:1.0:1787739068.842057:0:20585:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ade91 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x5:0x0].0x23e4280e bits 0x23/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ade98 expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.842066:0:20585:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ade91 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x5:0x0].0x23e4280e bits 0x23/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ade98 expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739068.842079:0:20585:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000040:00080000:1.0:1787739068.842253:0:20585:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].389, 448 off593104 00000040:00080000:1.0:1787739068.842279:0:20585:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].388, 448 off642288 00000004:00080000:1.0:1787739068.842284:0:20585:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07435539000 with version 741 00000004:00080000:1.0:1787739068.842297:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 741 rpc_version 741 00000020:00100000:1.0:1787739068.842316:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739068.842323:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074356acf00 x1874579665743744/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1728/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739068.842338:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356acf00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665743744:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739068.842366:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.842387:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539240 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.842398:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665743744 00000100:00100000:1.0:1787739068.842401:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665743744 00000100:00100000:1.0:1787739068.842437:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3476 00000100:00100000:1.0:1787739068.842443:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:7740:x1874579665743744:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739068.842737:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:7740:x1874579665743744:12345-0@lo:1000:osp_up1-0.0 Request processed in 293us (369us total) trans 4294968427 rc 0/0 00000100:00100000:1.0:1787739068.842745:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3476 00000100:00100000:1.0:1787739068.842785:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356acf00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665743744:0@lo:1000:osp_up1-0.0 00010000:00010000:1.0:1787739068.842804:0:20585:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ade91 lrc: 3/0,1 mode: EX/EX res: [0x240000401:0x5:0x0].0x23e4280e bits 0x23/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ade98 expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.842814:0:20585:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ade91 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x23e4280e bits 0x23/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ade98 expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.842822:0:20585:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1ade83 lrc: 3/0,1 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ade8a expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.842829:0:20585:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1ade83 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ade8a expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00100000:10000000:1.0:1787739068.842836:0:20585:0:(lfsck_namespace.c:1395:lfsck_namespace_insert_normal()) lustre-MDT0000-osd: namespace LFSCK insert object [0x200000402:0x1406:0x0] with the name subdir112 and type 40000 to the parent [0x240000401:0x5:0x0]: rc = 1 00000100:00100000:1.0:1787739068.842921:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ac3c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665743872:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.842946:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.842959:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306248540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.842970:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665743872 00000100:00100000:1.0:1787739068.842973:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665743872 00000100:00100000:1.0:1787739068.843008:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3477 00000100:00100000:1.0:1787739068.843013:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:20585:x1874579665743872:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.843066:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:20585:x1874579665743872:12345-0@lo:1000:lfsck_namespace.0 Request processed in 52us (119us total) trans 0 rc 1/1 00000100:00100000:1.0:1787739068.843073:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3477 00000100:00100000:1.0:1787739068.843096:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ac3c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665743872:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.843156:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ac3c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665744000:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.843179:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.843193:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306248540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.843203:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665744000 00000100:00100000:1.0:1787739068.843207:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665744000 00000100:00100000:1.0:1787739068.843226:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3478 00000100:00100000:1.0:1787739068.843231:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356afc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:20585:x1874579665744000:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.843279:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356afc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:20585:x1874579665744000:12345-0@lo:1000:lfsck_namespace.0 Request processed in 48us (99us total) trans 0 rc 1/1 00000100:00100000:1.0:1787739068.843285:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3478 00000100:00100000:1.0:1787739068.843308:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ac3c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665744000:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.843400:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ac3c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665744128:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.843443:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.843457:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306248540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.843468:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665744128 00000100:00100000:1.0:1787739068.843471:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665744128 00000100:00100000:1.0:1787739068.843489:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3479 00000100:00100000:1.0:1787739068.843494:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ae940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:20585:x1874579665744128:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.843559:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ae940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:20585:x1874579665744128:12345-0@lo:1000:lfsck_namespace.0 Request processed in 63us (114us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739068.843565:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3479 00000100:00100000:1.0:1787739068.843585:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ac3c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665744128:0@lo:1000:lfsck_namespace.0 00010000:00010000:1.0:1787739068.843613:0:20585:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade9f lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.843624:0:20585:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade9f lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2b:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.843634:0:20585:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade9f lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2b:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739068.843676:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ac3c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665744256:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.843699:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.843711:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306248540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.843720:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665744256 00000100:00100000:1.0:1787739068.843723:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665744256 00000100:00100000:1.0:1787739068.843739:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3480 00000100:00100000:1.0:1787739068.843744:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356af840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:20585:x1874579665744256:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.843799:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356af840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:20585:x1874579665744256:12345-0@lo:1000:lfsck_namespace.0 Request processed in 54us (99us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739068.843806:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3480 00000100:00100000:1.0:1787739068.843829:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ac3c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665744256:0@lo:1000:lfsck_namespace.0 00010000:00010000:1.0:1787739068.843842:0:20585:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade9f lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2b:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.843849:0:20585:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade9f lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x2b:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.843856:0:20585:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade9f lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2b:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.843862:0:20585:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fd400 already unused, calling callback (ffffffffc0cce440) 00010000:00010000:1.0:1787739068.843867:0:20585:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade9f lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2b:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.843874:0:20585:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade9f lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x2b:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.843885:0:20585:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffffa07305eaea00) 00010000:00010000:1.0:1787739068.843888:0:20585:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 22 00010000:00010000:1.0:1787739068.843891:0:20585:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade9f lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2b:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.843897:0:20585:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1ade9f lrc: 0/0,0 mode: --/EX res: [0x200000401:0x2b:0x0].0x0 bits 0x22/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739068.843952:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ac3c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665744384:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.843973:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.844001:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306248540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.844011:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665744384 00000100:00100000:1.0:1787739068.844014:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665744384 00000100:00100000:1.0:1787739068.844032:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3481 00000100:00100000:1.0:1787739068.844037:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:20585:x1874579665744384:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.844077:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:20585:x1874579665744384:12345-0@lo:1000:lfsck_namespace.0 Request processed in 40us (103us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739068.844083:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3481 00000100:00100000:1.0:1787739068.844103:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ac3c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665744384:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.844136:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ac3c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665744512:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.844158:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.844169:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306248540 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.844179:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665744512 00000100:00100000:1.0:1787739068.844181:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665744512 00000100:00100000:1.0:1787739068.844197:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3482 00000100:00100000:1.0:1787739068.844202:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:20585:x1874579665744512:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.844248:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:20585:x1874579665744512:12345-0@lo:1000:lfsck_namespace.0 Request processed in 45us (89us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739068.844258:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3482 00000100:00100000:1.0:1787739068.844280:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ac3c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665744512:0@lo:1000:lfsck_namespace.0 00010000:00010000:1.0:1787739068.844312:0:20585:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fde00/0x7ee4453e4a1adea6 lrc: 3/0,1 mode: --/CW res: [0x240000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.844319:0:20585:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fde00/0x7ee4453e4a1adea6 lrc: 3/0,1 mode: --/CW res: [0x240000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.844329:0:20585:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fde00/0x7ee4453e4a1adea6 lrc: 3/0,1 mode: --/CW res: [0x240000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739068.844340:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f0000 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665744640:0@lo:101:lfsck_namespace.0 00000100:00100000:1.0:1787739068.844362:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.844376:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306248300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.844386:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665744640 00000100:00100000:1.0:1787739068.844388:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665744640 00000100:00100000:1.0:1787739068.844407:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3483 00000100:00100000:1.0:1787739068.844434:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:20585:x1874579665744640:12345-0@lo:101:lfsck_namespace.0 00010000:00010000:1.0:1787739068.844442:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739068.844454:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fce00/0x7ee4453e4a1adead lrc: 2/0,0 mode: --/CW res: [0x240000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1adea6 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.844472:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fce00/0x7ee4453e4a1adead lrc: 3/0,0 mode: CW/CW res: [0x240000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1adea6 expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.844482:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fce00/0x7ee4453e4a1adead lrc: 3/0,0 mode: CW/CW res: [0x240000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adea6 expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.844495:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1:0x0].0x0 (ffffa07305eae200) 00010000:00010000:1.0:1787739068.844497:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739068.844500:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fce00, rc 0) 00000100:00100000:1.0:1787739068.844528:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:20585:x1874579665744640:12345-0@lo:101:lfsck_namespace.0 Request processed in 115us (164us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739068.844547:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3483 00000100:00100000:1.0:1787739068.844643:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f0000 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665744640:0@lo:101:lfsck_namespace.0 00010000:00010000:1.0:1787739068.844654:0:20585:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fde00/0x7ee4453e4a1adea6 lrc: 4/0,1 mode: CW/CW res: [0x240000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adead expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.844663:0:20585:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fde00/0x7ee4453e4a1adea6 lrc: 4/0,1 mode: CW/CW res: [0x240000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adead expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739068.844674:0:20585:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1:0x0] 00010000:00010000:1.0:1787739068.845820:0:20585:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd800/0x7ee4453e4a1adeb4 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1:0x0].0x96e3818c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.845831:0:20585:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd800/0x7ee4453e4a1adeb4 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1:0x0].0x96e3818c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.845841:0:20585:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd800/0x7ee4453e4a1adeb4 lrc: 3/0,1 mode: --/PW res: [0x240000403:0x1:0x0].0x96e3818c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739068.846144:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f0000 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665744768:0@lo:101:lfsck_namespace.0 00000100:00100000:0.0:1787739068.846194:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739068.846257:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.854745:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665744768 00000100:00100000:1.0:1787739068.854759:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665744768 00000100:00100000:1.0:1787739068.854857:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3484 00000100:00100000:1.0:1787739068.854868:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7130c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:20585:x1874579665744768:12345-0@lo:101:lfsck_namespace.0 00010000:00010000:1.0:1787739068.854891:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739068.854913:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc600/0x7ee4453e4a1adebb lrc: 2/0,0 mode: --/PW res: [0x240000403:0x1:0x0].0x96e3818c bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1adeb4 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.854944:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc600/0x7ee4453e4a1adebb lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1:0x0].0x96e3818c bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1adeb4 expref: 10 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.854960:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc600/0x7ee4453e4a1adebb lrc: 3/0,0 mode: PW/PW res: [0x240000403:0x1:0x0].0x96e3818c bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adeb4 expref: 10 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.854970:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000403:0x1:0x0].0x96e3818c (ffffa07305eaef00) 00010000:00010000:1.0:1787739068.854973:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739068.854988:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fc600, rc 0) 00000100:00100000:1.0:1787739068.855070:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7130c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+10:20585:x1874579665744768:12345-0@lo:101:lfsck_namespace.0 Request processed in 204us (8874us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739068.855077:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3484 00000100:00100000:0.0:1787739068.855466:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f0000 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665744768:0@lo:101:lfsck_namespace.0 00010000:00010000:0.0:1787739068.855486:0:20585:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd800/0x7ee4453e4a1adeb4 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1:0x0].0x96e3818c bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adebb expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.855497:0:20585:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd800/0x7ee4453e4a1adeb4 lrc: 4/0,1 mode: PW/PW res: [0x240000403:0x1:0x0].0x96e3818c bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adebb expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739068.855515:0:20585:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000403:0x1:0x0] 00000100:00100000:0.0:1787739068.855556:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f711e00 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665744896:0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.855595:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739068.855618:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.855757:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665744896 00000100:00100000:2.0:1787739068.855762:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665744896 00000100:00100000:2.0:1787739068.855795:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3485 00000100:00100000:2.0:1787739068.855802:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:20585:x1874579665744896:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.855888:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:20585:x1874579665744896:12345-0@lo:1000:lfsck_namespace.0 Request processed in 85us (292us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739068.855895:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3485 00000100:00100000:2.0:1787739068.855925:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f711e00 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665744896:0@lo:1000:lfsck_namespace.0 00000004:00080000:2.0:1787739068.855971:0:20585:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306887900 with version 0 00000020:00100000:2.0:1787739068.855999:0:20585:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739068.856007:0:20585:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0730432e580 x1874579665745152/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1240/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739068.856021:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432e580 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665745152:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.856049:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.856065:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.856076:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665745152 00000100:00100000:2.0:1787739068.856080:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665745152 00000100:00100000:2.0:1787739068.856102:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3486 00000100:00100000:2.0:1787739068.856109:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:20585:x1874579665745152:12345-0@lo:1000:lfsck_namespace.0 00000001:00080000:1.0:1787739068.858029:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968428 is committed 00000100:00100000:2.0:1787739068.858370:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:20585:x1874579665745152:12345-0@lo:1000:lfsck_namespace.0 Request processed in 2257us (2315us total) trans 4294968428 rc 0/0 00000100:00100000:2.0:1787739068.858382:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3486 00000004:00080000:3.0:1787739068.858496:0:20585:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968428 committed transno 4294968428 00000100:00100000:3.0:1787739068.858501:0:20585:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968428 gen 2 00000004:00080000:3.0:1787739068.858505:0:20585:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968427 committed transno 4294968428 00000100:00100000:3.0:1787739068.858537:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432e580 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665745152:0@lo:1000:lfsck_namespace.0 00010000:00010000:3.0:1787739068.858560:0:20585:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd800/0x7ee4453e4a1adeb4 lrc: 3/0,1 mode: PW/PW res: [0x240000403:0x1:0x0].0x96e3818c bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adebb expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.858572:0:20585:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fd800/0x7ee4453e4a1adeb4 lrc: 2/0,0 mode: PW/PW res: [0x240000403:0x1:0x0].0x96e3818c bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adebb expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.858581:0:20585:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fde00/0x7ee4453e4a1adea6 lrc: 3/0,1 mode: CW/CW res: [0x240000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adead expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.858588:0:20585:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073063fde00/0x7ee4453e4a1adea6 lrc: 2/0,0 mode: CW/CW res: [0x240000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adead expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00100000:10000000:3.0:1787739068.858595:0:20585:0:(lfsck_namespace.c:1662:lfsck_namespace_create_orphan_dir()) lustre-MDT0000-osd: namespace LFSCK create orphan dir for the object [0x240000402:0x49:0x0], name = [0x240000402:0x49:0x0]-P-0: rc = 1 00010000:00010000:3.0:1787739068.858706:0:20585:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1adec2 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x49:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.858716:0:20585:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1adec2 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x49:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.858726:0:20585:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1adec2 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x49:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739068.858744:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042443c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665745280:0@lo:101:lfsck_namespace.0 00000100:00100000:3.0:1787739068.858782:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.858822:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46c0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.858869:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665745280 00000100:00100000:2.0:1787739068.858875:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665745280 00000100:00100000:2.0:1787739068.858912:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3487 00000100:00100000:2.0:1787739068.858919:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304247c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:20585:x1874579665745280:12345-0@lo:101:lfsck_namespace.0 00010000:00010000:2.0:1787739068.859017:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739068.859033:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1adec9 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x49:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1adec2 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.859056:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1adec9 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x49:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1adec2 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.859067:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633c200/0x7ee4453e4a1adec9 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x49:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adec2 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739068.859074:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x49:0x0].0x0 (ffffa074419d5b00) 00010000:00010000:2.0:1787739068.859077:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739068.859080:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c200, rc 0) 00000100:00100000:2.0:1787739068.859115:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304247c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:20585:x1874579665745280:12345-0@lo:101:lfsck_namespace.0 Request processed in 197us (333us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739068.859122:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3487 00000100:00100000:3.0:1787739068.859178:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042443c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665745280:0@lo:101:lfsck_namespace.0 00010000:00010000:3.0:1787739068.859190:0:20585:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1adec2 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x49:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adec9 expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.859200:0:20585:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1adec2 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x49:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adec9 expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739068.859214:0:20585:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000402:0x49:0x0] 00010000:00010000:3.0:1787739068.859233:0:20585:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa800/0x7ee4453e4a1aded0 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x49:0x0].0xcdab3ca2 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.859240:0:20585:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa800/0x7ee4453e4a1aded0 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x49:0x0].0xcdab3ca2 bits 0x23/0x0 rrc: 2 type: IBT flags: 0x0 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.859247:0:20585:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa800/0x7ee4453e4a1aded0 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x49:0x0].0xcdab3ca2 bits 0x23/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 20585 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739068.859259:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042443c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665745408:0@lo:101:lfsck_namespace.0 00000100:00100000:3.0:1787739068.859281:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.859311:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46c0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739068.859490:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665745408 00000100:00100000:0.0:1787739068.859495:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665745408 00000100:00100000:0.0:1787739068.859553:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3488 00000100:00100000:0.0:1787739068.859560:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304245e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:20585:x1874579665745408:12345-0@lo:101:lfsck_namespace.0 00010000:00010000:0.0:1787739068.859567:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739068.859579:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1aded7 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x49:0x0].0xcdab3ca2 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aded0 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.859594:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1aded7 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x49:0x0].0xcdab3ca2 bits 0x23/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aded0 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.859605:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b000/0x7ee4453e4a1aded7 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x49:0x0].0xcdab3ca2 bits 0x23/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aded0 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739068.859612:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x49:0x0].0xcdab3ca2 (ffffa07305ca2600) 00010000:00010000:0.0:1787739068.859614:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 23 00010000:00010000:0.0:1787739068.859617:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b000, rc 0) 00000100:00100000:0.0:1787739068.859664:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304245e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+11:20585:x1874579665745408:12345-0@lo:101:lfsck_namespace.0 Request processed in 104us (381us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739068.859673:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3488 00000100:00100000:3.0:1787739068.860281:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042443c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665745408:0@lo:101:lfsck_namespace.0 00010000:00010000:3.0:1787739068.860294:0:20585:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa800/0x7ee4453e4a1aded0 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x49:0x0].0xcdab3ca2 bits 0x23/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aded7 expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739068.860310:0:20585:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa800/0x7ee4453e4a1aded0 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x49:0x0].0xcdab3ca2 bits 0x23/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aded7 expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739068.860324:0:20585:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000402:0x49:0x0] 00000100:00100000:3.0:1787739068.860387:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042443c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665745536:0@lo:1000:lfsck_namespace.0 00000100:00100000:3.0:1787739068.860505:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739068.865833:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46c600 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.867728:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665745536 00000100:00100000:1.0:1787739068.867810:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665745536 00000100:00100000:1.0:1787739068.867866:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3489 00000100:00100000:1.0:1787739068.867875:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042452c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+11:20585:x1874579665745536:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.868024:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042452c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+11:20585:x1874579665745536:12345-0@lo:1000:lfsck_namespace.0 Request processed in 150us (7518us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739068.868031:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3489 00000100:00100000:3.0:1787739068.868524:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042443c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665745536:0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.868526:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f711e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665745664:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1787739068.868568:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739068.868605:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665745664 00000100:00100000:0.0:1787739068.868609:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665745664 00000004:00080000:3.0:1787739068.868725:0:20585:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0740a46c0c0 with version 0 00000100:00100000:1.0:1787739068.868729:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665745792:0@lo:13:osp-pre-0-0.0 00000020:00100000:3.0:1787739068.868741:0:20585:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739068.868749:0:20585:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073042443c0 x1874579665746048/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 328/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739068.868764:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042443c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665746048:0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.868850:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 362 00000100:00100000:0.0:1787739068.868858:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f711a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665745664:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1787739068.868879:0:8441:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8646080 pending 0 free 3923529728 avail 3692007424 00000100:00100000:3.0:1787739068.877540:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.877756:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46cc00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739068.877894:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f711a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665745664:12345-0@lo:13:osp-pre-0-1.0 Request processed in 9033us (9321us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739068.877907:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 362 00000100:00080000:0.0:1787739068.877951:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787739068.878186:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.878367:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665746048 00000100:00100000:1.0:1787739068.878376:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665746048 00000100:00100000:1.0:1787739068.878435:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3490 00000100:00100000:1.0:1787739068.878443:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042470c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+11:20585:x1874579665746048:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.878691:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042470c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+12:20585:x1874579665746048:12345-0@lo:1000:lfsck_namespace.0 Request processed in 248us (1159us total) trans 4294968429 rc 0/0 00000100:00100000:1.0:1787739068.878698:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3490 00000100:00100000:0.0:1787739068.878930:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665745792 00000100:00100000:0.0:1787739068.878934:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665745792 00000100:00100000:0.0:1787739068.878970:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 363 00000100:00100000:2.0:1787739068.878971:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f711e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665745664:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1787739068.878989:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874579665745792:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1787739068.879002:0:8440:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8646080 pending 0 free 3923529728 avail 3692007424 00000100:00100000:2.0:1787739068.879025:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042443c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665746048:0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.879039:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874579665745792:12345-0@lo:13:osp-pre-0-0.0 Request processed in 63us (854us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739068.879045:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 363 00000100:00080000:0.0:1787739068.879060:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787739068.879081:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432e580 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665746176:0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.879084:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f3840 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665745792:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787739068.879119:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.879146:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068873c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739068.879320:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665746176 00000100:00100000:0.0:1787739068.879324:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665746176 00000100:00100000:0.0:1787739068.879355:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3491 00000100:00100000:0.0:1787739068.879362:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+12:20585:x1874579665746176:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.879448:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+12:20585:x1874579665746176:12345-0@lo:1000:lfsck_namespace.0 Request processed in 86us (328us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739068.879455:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3491 00000100:00100000:2.0:1787739068.879459:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432e580 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665746176:0@lo:1000:lfsck_namespace.0 00000040:00080000:2.0:1787739068.879624:0:20585:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].390, 448 off593552 00000040:00080000:2.0:1787739068.880763:0:20585:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].389, 448 off642736 00000004:00080000:2.0:1787739068.880774:0:20585:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306887e40 with version 742 00000004:00080000:1.0:1787739068.880908:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 742 rpc_version 742 00000020:00100000:1.0:1787739068.880937:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739068.880946:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074080f3840 x1874579665746432/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739068.880965:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f3840 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665746432:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739068.881029:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.881062:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306248300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739068.881113:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665746432 00000100:00100000:2.0:1787739068.881119:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665746432 00000100:00100000:2.0:1787739068.881167:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3492 00000100:00100000:2.0:1787739068.881175:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+12:7740:x1874579665746432:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739068.881545:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+13:7740:x1874579665746432:12345-0@lo:1000:osp_up1-0.0 Request processed in 370us (514us total) trans 4294968430 rc 0/0 00000100:00100000:2.0:1787739068.881552:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3492 00000100:00100000:0.0:1787739068.881826:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f3840 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665746432:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739068.882804:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432e580 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665746560:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.882855:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.882886:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739068.883009:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665746560 00000100:00100000:0.0:1787739068.883015:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665746560 00000100:00100000:0.0:1787739068.883057:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3493 00000100:00100000:0.0:1787739068.883064:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c383c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+13:20585:x1874579665746560:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.883150:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c383c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+13:20585:x1874579665746560:12345-0@lo:1000:lfsck_namespace.0 Request processed in 86us (294us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739068.883157:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3493 00000100:00100000:1.0:1787739068.883293:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432e580 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665746560:0@lo:1000:lfsck_namespace.0 00000004:00080000:1.0:1787739068.883405:0:20585:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa073062480c0 with version 0 00000020:00100000:1.0:1787739068.883449:0:20585:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739068.883456:0:20585:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074356ac3c0 x1874579665746816/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 408/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739068.883471:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ac3c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665746816:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.883501:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.883526:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306248840 going to sleep for 11 seconds 00000100:00100000:0.0:1787739068.883545:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665746816 00000100:00100000:0.0:1787739068.883549:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665746816 00000100:00100000:0.0:1787739068.883582:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3494 00000100:00100000:0.0:1787739068.883589:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+13:20585:x1874579665746816:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.883724:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+14:20585:x1874579665746816:12345-0@lo:1000:lfsck_namespace.0 Request processed in 135us (221us total) trans 4294968431 rc 0/0 00000100:00100000:0.0:1787739068.883731:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3494 00000100:00100000:1.0:1787739068.883756:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ac3c0 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665746816:0@lo:1000:lfsck_namespace.0 00010000:00010000:1.0:1787739068.883773:0:20585:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa800/0x7ee4453e4a1aded0 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x49:0x0].0xcdab3ca2 bits 0x23/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aded7 expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.883788:0:20585:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa800/0x7ee4453e4a1aded0 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x49:0x0].0xcdab3ca2 bits 0x23/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aded7 expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.883797:0:20585:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1adec2 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x49:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adec9 expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739068.883804:0:20585:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab000/0x7ee4453e4a1adec2 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x49:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adec9 expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00100000:10000000:1.0:1787739068.883812:0:20585:0:(lfsck_namespace.c:1395:lfsck_namespace_insert_normal()) lustre-MDT0000-osd: namespace LFSCK insert object [0x200000401:0x2b:0x0] with the name [0x200000401:0x2b:0x0]:1 and type 40000 to the parent [0x240000402:0x49:0x0]: rc = 1 00000100:00100000:1.0:1787739068.883921:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306392d00 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665746944:0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.883945:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.883966:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306248480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739068.883996:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665746944 00000100:00100000:0.0:1787739068.884000:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665746944 00000100:00100000:0.0:1787739068.884028:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3495 00000100:00100000:0.0:1787739068.884034:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306392580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+14:20585:x1874579665746944:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:0.0:1787739068.884093:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306392580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+14:20585:x1874579665746944:12345-0@lo:1000:lfsck_namespace.0 Request processed in 59us (147us total) trans 0 rc 1/1 00000100:00100000:0.0:1787739068.884100:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3495 00000100:00100000:2.0:1787739068.884105:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306392d00 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665746944:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.884165:0:20585:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c383840 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665747072:0@lo:1000:lfsck_namespace.0 00000100:00100000:2.0:1787739068.884189:0:20585:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739068.884210:0:20585:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887600 going to sleep for 11 seconds 00000100:00100000:1.0:1787739068.884366:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665747072 00000100:00100000:1.0:1787739068.884370:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665747072 00000100:00100000:1.0:1787739068.884398:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3496 00000100:00100000:1.0:1787739068.884404:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c380780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+14:20585:x1874579665747072:12345-0@lo:1000:lfsck_namespace.0 00000100:00100000:1.0:1787739068.884474:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c380780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+14:20585:x1874579665747072:12345-0@lo:1000:lfsck_namespace.0 Request processed in 70us (283us total) trans 0 rc 1/1 00000100:00100000:1.0:1787739068.884480:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3496 00000100:00100000:2.0:1787739068.884554:0:20585:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c383840 pname:cluuid:pid:xid:nid:opc:job lfsck_namespace:lustre-MDT0000-mdtlov_UUID:20585:1874579665747072:0@lo:1000:lfsck_namespace.0 00100000:10000000:2.0:1787739068.884597:0:20585:0:(lfsck_namespace.c:6719:lfsck_namespace_assistant_handler_p2()) lustre-MDT0000-osd: namespace LFSCK phase2 scan stop at the No. 16 trace file: rc = 1 00100000:10000000:2.0:1787739068.884606:0:20585:0:(lfsck_engine.c:1760:lfsck_assistant_engine()) lustre-MDT0000-osd: LFSCK assistant sync before exit 00000004:00080000:2.0:1787739068.884621:0:20585:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 655857206735ns, next at 533789884169ns, rc = 0 00080000:00080000:2.0:1787739068.884630:0:20585:0:(osd_handler.c:2849:osd_commit_async()) lustre-MDT0000: async commit OSD 00000004:00080000:0.0:1787739068.885495:0:20585:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 655858079608ns, next at 521673230421ns, rc = 0 00080000:00080000:0.0:1787739068.885506:0:20585:0:(osd_handler.c:2849:osd_commit_async()) lustre-MDT0000: async commit OSD 00000004:00080000:2.0:1787739068.886361:0:6502:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 75 committed 00000020:00080000:2.0:1787739068.886394:0:6502:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07435539540 refcount 2 committed 0 result 0 batchid 391 00000020:00080000:2.0:1787739068.886398:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07302f96f00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07435539000 00000020:00080000:2.0:1787739068.886402:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].388 00000020:00080000:2.0:1787739068.886405:0:6502:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07302f96a00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304cf5a00 00000020:00080000:2.0:1787739068.886407:0:6502:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].389 00000020:00080000:2.0:1787739068.886441:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 390 batchid updated 00000001:00080000:2.0:1787739068.886449:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983436 is committed 00000020:00080000:0.0:1787739068.886528:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 391 committed batchid 390 00000004:00080000:0.0:1787739068.886649:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07303803b40 with version 743 00000020:00080000:0.0:1787739068.886666:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 390 cancel update log [0x240000401:0x3:0x0].387: rc = 0 00000020:00080000:0.0:1787739068.886704:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 390 cancel update log [0x200000400:0x3:0x0].388: rc = 0 00000020:00080000:0.0:1787739068.886712:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 390 00000004:00080000:1.0:1787739068.887028:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 743 rpc_version 743 00000020:00100000:1.0:1787739068.887059:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739068.887070:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306390f00 x1874579665747328/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739068.887091:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306390f00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665747328:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739068.887158:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739068.887195:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306248900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739068.887232:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665747328 00000100:00100000:0.0:1787739068.887239:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665747328 00000100:00100000:0.0:1787739068.887294:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3497 00000100:00100000:0.0:1787739068.887302:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306391680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+14:7740:x1874579665747328:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739068.887448:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306391680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+15:7740:x1874579665747328:12345-0@lo:1000:osp_up1-0.0 Request processed in 146us (288us total) trans 4294968432 rc 0/0 00000100:00100000:0.0:1787739068.887457:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3497 00000100:00100000:1.0:1787739068.887526:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306390f00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665747328:0@lo:1000:osp_up1-0.0 00000004:00080000:0.0:1787739068.887829:0:6502:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 72 committed 00100000:10000000:2.0:1787739068.887897:0:20585:0:(lfsck_engine.c:1766:lfsck_assistant_engine()) lustre-MDT0000-osd: LFSCK assistant synced before exit: rc = 0 00100000:10000000:2.0:1787739068.887945:0:20585:0:(lfsck_engine.c:1778:lfsck_assistant_engine()) lustre-MDT0000-osd: LFSCK assistant phase2 scan finished: rc = 1 00100000:10000000:2.0:1787739068.887961:0:20585:0:(lfsck_engine.c:1790:lfsck_assistant_engine()) lustre-MDT0000-osd: lfsck_namespace LFSCK assistant thread exit: rc = 1 00000100:00100000:1.0:1787739069.380557:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304246d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665747456:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1787739069.380729:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739069.380917:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665747456 00000100:00100000:3.0:1787739069.380925:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665747456 00000100:00100000:3.0:1787739069.380983:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3498 00000100:00100000:3.0:1787739069.380992:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306393480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+15:3653:x1874579665747456:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1787739069.381025:0:12410:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1445031936 avail 1298792448 00000100:00100000:3.0:1787739069.381080:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306393480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+15:3653:x1874579665747456:12345-0@lo:41:osp-pre-1-0.0 Request processed in 90us (351us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739069.381087:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3498 00000100:00100000:1.0:1787739069.382813:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304246d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665747456:0@lo:41:osp-pre-1-0.0 00010000:00010000:2.0:1787739069.765573:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739069.892337:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f7121c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665747584:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1787739069.892410:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739069.896754:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665747584 00000100:00100000:3.0:1787739069.896768:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665747584 00000100:00100000:3.0:1787739069.896838:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3499 00000100:00100000:3.0:1787739069.896858:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073063903c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3650:x1874579665747584:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1787739069.896893:0:6529:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1443950592 avail 1297711104 00000100:00100000:3.0:1787739069.896996:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073063903c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3650:x1874579665747584:12345-0@lo:41:osp-pre-0-1.0 Request processed in 149us (4587us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739069.897003:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3499 00000100:00100000:2.0:1787739069.905505:0:3650:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983436 gen 2 00000004:00080000:2.0:1787739069.905517:0:3650:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983436 committed transno 4294983436 00000100:00100000:2.0:1787739069.905554:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f7121c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665747584:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1787739070.780333:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739070.781038:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652506368 00000100:00100000:2.0:1787739070.781049:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652506368 00000100:00100000:2.0:1787739070.781127:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23584 00000100:00100000:2.0:1787739070.781139:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304245680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17725:x1874579652506368:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:2.0:1787739070.781161:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739070.781184:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1adede lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095241 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739070.781243:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1adee5 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739070.781257:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1adee5 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739070.781270:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1adee5 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787739070.781280:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000004:00200000:2.0:1787739070.781317:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x5:0x0]: returning size 4096 00000004:00200000:2.0:1787739070.781323:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739070.781379:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1adede lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095241 expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739070.781396:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1adee5 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095241 expref: 14 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739070.781405:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5400) 00010000:00010000:2.0:1787739070.781408:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739070.781442:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d000, rc 0) 00000100:00100000:2.0:1787739070.781594:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304245680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17725:x1874579652506368:12345-192.168.203.15@tcp:101:ls.0 Request processed in 457us (1268us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739070.781603:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23584 00010000:00010000:0.0:1787739070.788155:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739070.792542:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739070.792751:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652506496 00000100:00100000:0.0:1787739070.792766:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652506496 00000100:00080000:2.0:1787739070.792860:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787739070.792894:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23585 00000100:00100000:0.0:1787739070.792909:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304244b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:17725:x1874579652506496:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:0.0:1787739070.792936:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739070.792975:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adeec lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095248 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.793060:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1adef3 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.793083:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ade8a lrc: 2/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ade83 expref: 14 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.793093:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1adef3 lrc: 4/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.793143:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ade8a lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ade83 expref: 14 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739070.793164:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f7112c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665747712:0@lo:104: 00000100:00100000:0.0:1787739070.793243:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739070.793312:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665747712 00000100:00100000:0.0:1787739070.793316:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739070.794006:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665747712 00000100:00100000:2.0:1787739070.794076:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1198 00000100:00100000:2.0:1787739070.794086:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f712580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:13793:x1874579665747712:12345-0@lo:104: 00010000:00010000:2.0:1787739070.794104:0:15907:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1ade83 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ade8a expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739070.794406:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f7112c0 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001_UUID:13793:1874579665747712:0@lo:104: 00010000:00010000:0.0:1787739070.794458:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1adef3 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739070.794657:0:15907:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1787739070.794667:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f712580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:13793:x1874579665747712:12345-0@lo:104: Request processed in 582us (1422us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739070.794676:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1198 00010000:00010000:3.0:1787739070.794688:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739070.794699:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1ade83 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ade8a expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.794711:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d800 already unused, calling callback (ffffffffc0cce440) 00010000:00010000:3.0:1787739070.794716:0:10339:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1ade83 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1400400000000 nid: local remote: 0x7ee4453e4a1ade8a expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.794726:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1ade83 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ade8a expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.794885:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1ade83 lrc: 2/0,0 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ade8a expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.794894:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739070.794933:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1ade83 lrc: 1/0,0 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ade8a expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.794941:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa0730633d800/0x7ee4453e4a1ade83 lrc: 0/0,0 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x12 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ade8a expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.794979:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739070.794981:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739070.794983:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739070.795039:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304246d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665747840:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739070.795102:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739070.795537:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665747840 00000100:00100000:3.0:1787739070.795545:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665747840 00010000:00010000:3.0:1787739070.795582:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ade8a lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ade83 expref: 15 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739070.795627:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1567 00000100:00100000:3.0:1787739070.795637:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7112c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+15:3651:x1874579665747840:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739070.795651:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:2.0:1787739070.802826:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968432 is committed 00010000:00010000:3.0:1787739070.802999:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633da00/0x7ee4453e4a1ade8a lrc: 0/0,0 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ade83 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.803027:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:3.0:1787739070.803033:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1adef3 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.803045:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1adef3 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.803055:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1adef3 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.803100:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739070.803235:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7112c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+10:3651:x1874579665747840:12345-0@lo:103:ldlm_bl.0 Request processed in 7598us (8131us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739070.803247:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1567 00010000:00010000:0.0:1787739070.803337:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1adef3 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.803347:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1adef3 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739070.803461:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x5:0x0]: returning size 4096 00000004:00200000:0.0:1787739070.803468:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739070.803553:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b200/0x7ee4453e4a1adeec lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095248 expref: 8 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.803577:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1adef3 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095248 expref: 7 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.803585:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:0.0:1787739070.803588:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739070.803591:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609bc00, rc 0) 00000100:00100000:0.0:1787739070.803662:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304244b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17725:x1874579652506496:12345-192.168.203.15@tcp:101:ls.0 Request processed in 10757us (11135us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739070.803670:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23585 00000100:00100000:2.0:1787739070.804015:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304246d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665747840:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739070.833983:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739070.834070:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652506624 00000100:00100000:2.0:1787739070.834083:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652506624 00000100:00100000:2.0:1787739070.834156:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5013 00000100:00100000:2.0:1787739070.834169:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304246d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17725:x1874579652506624:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:1.0:1787739070.847269:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304246d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17725:x1874579652506624:12345-192.168.203.15@tcp:37:ls.0 Request processed in 13086us (13292us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739070.847286:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5013 00000100:00100000:2.0:1787739070.871825:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739070.871910:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652506752 00000100:00100000:0.0:1787739070.871925:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652506752 00000100:00100000:0.0:1787739070.872015:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5014 00000100:00100000:0.0:1787739070.872028:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074332a9050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17725:x1874579652506752:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:3.0:1787739070.884356:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074332a9050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17725:x1874579652506752:12345-192.168.203.15@tcp:37:ls.0 Request processed in 12315us (12531us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739070.884375:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5014 00000100:00100000:0.0:1787739070.896856:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739070.901386:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652507008 00000100:00100000:1.0:1787739070.901406:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652507008 00000100:00100000:1.0:1787739070.901553:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23586 00000100:00100000:1.0:1787739070.901566:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:17725:x1874579652507008:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739070.901594:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739070.901621:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd600/0x7ee4453e4a1adefa lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09524f expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739070.901780:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306392d00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID:7944:1874579665747968:0@lo:900:mdt00_003.0 00000100:00100000:1.0:1787739070.901869:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739070.901936:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887900 going to sleep for 11 seconds 00000100:00100000:1.0:1787739070.901997:0:6534:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665747968 00000100:00100000:1.0:1787739070.902001:0:6534:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665747968 00000100:00100000:1.0:1787739070.902035:0:6534:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8 00000100:00100000:1.0:1787739070.902045:0:6534:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306390b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_fld_0000:lustre-MDT0000-lwp-MDT0001_UUID+5:7944:x1874579665747968:12345-0@lo:900:mdt00_003.0 00000100:00100000:1.0:1787739070.902101:0:6534:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306390b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_fld_0000:lustre-MDT0000-lwp-MDT0001_UUID+5:7944:x1874579665747968:12345-0@lo:900:mdt00_003.0 Request processed in 57us (232us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739070.902110:0:6534:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8 00000100:00100000:1.0:1787739070.902162:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306392d00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID:7944:1874579665747968:0@lo:900:mdt00_003.0 00000100:00100000:1.0:1787739070.902227:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306392d00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665748096:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739070.902261:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739070.902278:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739070.905070:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665748096 00000100:00100000:0.0:1787739070.905088:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665748096 00000100:00100000:0.0:1787739070.905173:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3500 00000100:00100000:0.0:1787739070.905185:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157ea580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665748096:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739070.905345:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157ea580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665748096:12345-0@lo:1000:mdt00_003.0 Request processed in 163us (3082us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739070.905354:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3500 00000100:00100000:0.0:1787739070.910086:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306392d00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665748096:0@lo:1000:mdt00_003.0 00010000:00010000:0.0:1787739070.910213:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1adf01 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.910230:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1adf01 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.910241:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1adf01 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739070.910266:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710f00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665748224:0@lo:101:mdt00_003.0 00000100:00100000:0.0:1787739070.910349:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739070.910438:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062343c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739070.910528:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665748224 00000100:00100000:3.0:1787739070.910553:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665748224 00000100:00100000:3.0:1787739070.910610:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3501 00000100:00100000:3.0:1787739070.910621:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:7944:x1874579665748224:12345-0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739070.910640:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739070.910660:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1adf08 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1adf01 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.910692:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1adf08 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1adf01 expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.910709:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abc00/0x7ee4453e4a1adf08 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1adf01 expref: 8 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.910720:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa07401795300) 00010000:00010000:3.0:1787739070.910723:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1787739070.910727:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abc00, rc 0) 00000100:00100000:3.0:1787739070.910791:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665748224:12345-0@lo:101:mdt00_003.0 Request processed in 172us (442us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739070.910799:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3501 00000100:00100000:0.0:1787739070.911040:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710f00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665748224:0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739070.911057:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1adf01 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adf08 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.911069:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1adf01 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1adf08 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739070.911088:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787739070.911097:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1adf01 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adf08 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.911110:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa0730609b200/0x7ee4453e4a1adf01 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1adf08 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.911146:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1adf0f lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.911155:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1adf0f lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.911169:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1adf0f lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.911182:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adf16 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x23e4280e bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.911192:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fcc00/0x7ee4453e4a1ade98 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x23e4280e bits 0x23/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ade91 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.911201:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adf16 lrc: 4/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.911230:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fcc00/0x7ee4453e4a1ade98 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x23e4280e bits 0x23/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1ade91 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739070.911244:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710f00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665748352:0@lo:104: 00000100:00100000:0.0:1787739070.911279:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739070.911316:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062343c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739070.911369:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665748352 00000100:00100000:3.0:1787739070.911375:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665748352 00000100:00100000:3.0:1787739070.911445:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1199 00000100:00100000:3.0:1787739070.911453:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665748352:12345-0@lo:104: 00010000:00010000:3.0:1787739070.911470:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ade91 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x23e4280e bits 0x23/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ade98 expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.911526:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1787739070.911532:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:7944:x1874579665748352:12345-0@lo:104: Request processed in 81us (252us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739070.911550:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1199 00010000:00010000:3.0:1787739070.911569:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739070.911577:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ade91 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x23e4280e bits 0x23/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1ade98 expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.911586:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa000 already unused, calling callback (ffffffffc0cce440) 00010000:00010000:3.0:1787739070.911591:0:14665:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ade91 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x23e4280e bits 0x23/0x2 rrc: 2 type: IBT gid 0 flags: 0x1400400000000 nid: local remote: 0x7ee4453e4a1ade98 expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.911601:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ade91 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x23e4280e bits 0x23/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1ade98 expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.911632:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ade91 lrc: 2/0,0 mode: --/EX res: [0x240000401:0x5:0x0].0x23e4280e bits 0x23/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ade98 expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.911639:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739070.911666:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ade91 lrc: 1/0,0 mode: --/EX res: [0x240000401:0x5:0x0].0x23e4280e bits 0x23/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ade98 expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.911675:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa000/0x7ee4453e4a1ade91 lrc: 0/0,0 mode: --/EX res: [0x240000401:0x5:0x0].0x23e4280e bits 0x23/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1ade98 expref: -99 pid: 20585 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.911700:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739070.911702:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739070.911704:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739070.911716:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304246d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665748480:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739070.911753:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739070.911796:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665748480 00000100:00100000:3.0:1787739070.911800:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665748480 00010000:00010000:3.0:1787739070.911819:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fcc00/0x7ee4453e4a1ade98 lrc: 4/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x23e4280e bits 0x23/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1ade91 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739070.911849:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1568 00000100:00100000:3.0:1787739070.911856:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba6c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+10:3652:x1874579665748480:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739070.911867:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739070.912376:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x23e4280e (ffffa07305eaeb00) 00010000:00010000:3.0:1787739070.912385:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adf16 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.912395:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adf16 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.912401:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adf16 lrc: 4/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.912441:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739070.912488:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba6c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+10:3652:x1874579665748480:12345-0@lo:103:ldlm_bl.0 Request processed in 632us (734us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739070.912496:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1568 00000100:00100000:3.0:1787739070.912572:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304246d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665748480:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739070.912728:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fcc00/0x7ee4453e4a1ade98 lrc: 0/0,0 mode: --/EX res: [0x240000401:0x5:0x0].0x23e4280e bits 0x23/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1ade91 expref: 9 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739070.912744:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710f00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665748352:0@lo:104: 00010000:00010000:0.0:1787739070.912760:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adf16 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.912770:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adf16 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.912777:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adf16 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739070.912909:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743f710f00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665748608:0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739070.912950:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739070.912985:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062346c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739070.913039:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665748608 00000100:00100000:3.0:1787739070.913045:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665748608 00000100:00100000:3.0:1787739070.914484:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3502 00000100:00100000:3.0:1787739070.914504:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f713480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665748608:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739070.914645:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:7944:x1874579665748608:12345-0@lo:1000:mdt00_003.0 Request processed in 148us (1694us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739070.914653:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3502 00000100:00100000:1.0:1787739070.917744:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743f710f00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665748608:0@lo:1000:mdt00_003.0 00010000:00010000:1.0:1787739070.917846:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd400/0x7ee4453e4a1adf1d lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1406:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739070.917867:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd400/0x7ee4453e4a1adf1d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1406:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739070.917881:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd400/0x7ee4453e4a1adf1d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1406:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739070.917904:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1adf0f lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739070.917912:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1adf0f lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739070.917919:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1adf0f lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739070.917926:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1adf0f lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739070.917938:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739070.917943:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1adf0f lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739070.917952:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1adf0f lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739070.917979:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:1.0:1787739070.917983:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739070.917988:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1adf0f lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739070.917995:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1adf0f lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739070.918008:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adf16 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739070.918015:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adf16 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739070.918021:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adf16 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739070.918026:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739070.918029:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adf16 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739070.918035:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adf16 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739070.918047:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x23e4280e (ffffa07305eaeb00) 00010000:00010000:1.0:1787739070.918050:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739070.918053:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adf16 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739070.918059:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adf16 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739070.918116:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd600/0x7ee4453e4a1adefa lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09524f expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739070.918134:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd400/0x7ee4453e4a1adf1d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1406:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09524f expref: 8 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739070.918142:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1406:0x0].0x0 (ffffa0730713a400) 00010000:00010000:1.0:1787739070.918145:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739070.918147:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fd400, rc 0) 00000100:00100000:1.0:1787739070.918239:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f713c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17725:x1874579652507008:12345-192.168.203.15@tcp:101:ls.0 Request processed in 16675us (21397us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739070.918251:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23586 00000100:00100000:3.0:1787739070.927738:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739070.928185:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652507264 00000100:00100000:2.0:1787739070.928199:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652507264 00000100:00100000:2.0:1787739070.928311:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23587 00000100:00100000:2.0:1787739070.928323:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304246d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17725:x1874579652507264:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:2.0:1787739070.928349:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739070.928380:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adf24 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1406:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095256 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739070.928871:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1406:0x0]: returning size 4096 00010000:00010000:2.0:1787739070.929008:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adf24 lrc: 1/0,0 mode: --/CR res: [0x200000402:0x1406:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095256 expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739070.929026:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1406:0x0].0x0 (ffffa07403eebd00) 00010000:00010000:2.0:1787739070.929029:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739070.929035:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adf24 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1406:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095256 expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739070.929052:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633ce00, rc 0) 00000100:00100000:2.0:1787739070.929141:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304246d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17725:x1874579652507264:12345-192.168.203.15@tcp:101:ls.0 Request processed in 820us (1419us total) trans 4294983437 rc 301/301 00000100:00100000:2.0:1787739070.929149:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23587 00000100:00100000:2.0:1787739070.936052:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739070.941111:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652507392 00000100:00100000:0.0:1787739070.941143:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652507392 00000100:00100000:2.0:1787739070.941771:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23588 00000100:00100000:2.0:1787739070.941787:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c3812c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17725:x1874579652507392:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:2.0:1787739070.941820:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739070.941849:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adf32 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09525d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739070.941928:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adf39 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739070.941945:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adf39 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739070.941959:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adf39 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739070.942037:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x3e:0x0]: returning size 4096 00000004:00200000:2.0:1787739070.942043:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739070.942087:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adf32 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09525d expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739070.942104:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adf39 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09525d expref: 16 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739070.942115:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x3e:0x0].0x0 (ffffa07403eebd00) 00010000:00010000:2.0:1787739070.942119:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739070.942122:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c000, rc 0) 00000100:00100000:2.0:1787739070.942229:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c3812c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17725:x1874579652507392:12345-192.168.203.15@tcp:101:ls.0 Request processed in 445us (6190us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739070.942236:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23588 00000100:00100000:1.0:1787739070.946834:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739070.947010:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652507520 00000100:00100000:0.0:1787739070.947025:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652507520 00000100:00100000:0.0:1787739070.947105:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23589 00000100:00100000:0.0:1787739070.947116:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157eb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17725:x1874579652507520:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:0.0:1787739070.947154:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739070.947184:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adf40 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095264 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.947356:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1adf47 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.947374:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1adf47 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.947388:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1adf47 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.947400:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1adf47 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.947406:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1adf47 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.947449:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1adf47 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.947456:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739070.947462:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1adf47 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.947470:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1adf47 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.947482:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x3e:0x0].0x0 (ffffa07305ca2800) 00010000:00010000:0.0:1787739070.947485:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739070.947491:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1adf47 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.947497:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1adf47 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.947539:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adf40 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095264 expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.947551:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x3e:0x0].0x0 (ffffa07305ca2800) 00010000:00010000:0.0:1787739070.947553:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739070.947556:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adf40 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095264 expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.947571:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609ac00, rc 0) 00000100:00100000:0.0:1787739070.947654:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157eb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17725:x1874579652507520:12345-192.168.203.15@tcp:101:ls.0 Request processed in 540us (834us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739070.947662:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23589 00000100:00100000:0.0:1787739070.955528:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739070.956118:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652507648 00000100:00100000:2.0:1787739070.956150:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652507648 00000100:00100000:2.0:1787739070.956247:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23590 00000100:00100000:2.0:1787739070.956262:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f712940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17725:x1874579652507648:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:2.0:1787739070.956290:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739070.956323:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adf4e lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09526b expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739070.956456:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1adf55 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1406:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739070.956474:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1adf55 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1406:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739070.956489:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1adf55 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1406:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739070.956619:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1406:0x0]: returning size 4096 00000004:00200000:2.0:1787739070.956626:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:2.0:1787739070.956654:0:6524:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1adf55 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1406:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739070.956690:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adf4e lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09526b expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739070.956719:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1adf55 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1406:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09526b expref: 17 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739070.956854:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1406:0x0].0x0 (ffffa07403eeb000) 00010000:00010000:2.0:1787739070.956858:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739070.956863:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633cc00, rc 0) 00000100:00100000:2.0:1787739070.956952:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f712940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17725:x1874579652507648:12345-192.168.203.15@tcp:101:ls.0 Request processed in 694us (1433us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739070.956972:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23590 00000100:00100000:1.0:1787739070.961789:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739070.961936:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652507776 00000100:00100000:0.0:1787739070.962088:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652507776 00000100:00100000:0.0:1787739070.962234:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23591 00000100:00100000:0.0:1787739070.962258:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157e9e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17725:x1874579652507776:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:0.0:1787739070.962283:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739070.962314:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adf5c lrc: 2/0,0 mode: --/CR res: [0x240000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095272 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.962472:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1adf63 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.962489:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1adf63 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.962500:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1adf63 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.962510:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1adf63 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.962517:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1adf63 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.962523:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1adf63 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.962529:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739070.962533:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1adf63 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.962541:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1adf63 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.962552:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x3e:0x0].0x0 (ffffa07305ca2800) 00010000:00010000:0.0:1787739070.962555:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739070.962559:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1adf63 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.962565:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b800/0x7ee4453e4a1adf63 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.962601:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adf5c lrc: 1/0,0 mode: --/CR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095272 expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.962611:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x3e:0x0].0x0 (ffffa07305ca2800) 00010000:00010000:0.0:1787739070.962614:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739070.962618:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609ac00/0x7ee4453e4a1adf5c lrc: 0/0,0 mode: --/CR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095272 expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739070.962631:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609ac00, rc 0) 00000100:00100000:1.0:1787739070.963008:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157e9e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17725:x1874579652507776:12345-192.168.203.15@tcp:101:ls.0 Request processed in 750us (1220us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739070.963029:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23591 00000100:00100000:1.0:1787739070.967150:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739070.967212:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652507904 00000100:00100000:2.0:1787739070.967225:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652507904 00000100:00100000:2.0:1787739070.967310:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23592 00000100:00100000:2.0:1787739070.967321:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074157ebc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17725:x1874579652507904:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:2.0:1787739070.967358:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739070.967389:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1adf6a lrc: 2/0,0 mode: --/CR res: [0x240000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095279 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.967826:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1adf71 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.967845:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1adf71 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.967858:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1adf71 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.967869:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1adf71 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.967876:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1adf71 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.967883:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1adf71 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.967889:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739070.967894:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1adf71 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.967907:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1adf71 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.967919:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x3e:0x0].0x0 (ffffa07403eebb00) 00010000:00010000:3.0:1787739070.967923:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739070.967928:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1adf71 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.967934:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1adf71 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.967984:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1adf6a lrc: 1/0,0 mode: --/CR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095279 expref: 9 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.967995:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x3e:0x0].0x0 (ffffa07403eebb00) 00010000:00010000:3.0:1787739070.967998:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739070.968001:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1adf6a lrc: 0/0,0 mode: --/CR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095279 expref: 9 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.968019:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d200, rc 0) 00000100:00100000:3.0:1787739070.968097:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074157ebc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17725:x1874579652507904:12345-192.168.203.15@tcp:101:ls.0 Request processed in 778us (974us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739070.968106:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23592 00000100:00100000:0.0:1787739070.971892:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739070.971980:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652508032 00000100:00100000:3.0:1787739070.971993:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652508032 00000100:00100000:3.0:1787739070.972078:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23593 00000100:00100000:3.0:1787739070.972090:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7103c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17725:x1874579652508032:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:3.0:1787739070.972120:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739070.972152:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1adf78 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095280 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.972320:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1adf7f lrc: 3/1,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.972336:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1adf7f lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.972350:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1adf7f lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.972362:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1adf7f lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.972368:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1adf7f lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.972374:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1adf7f lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.972380:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739070.972385:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1adf7f lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.972393:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1adf7f lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.972405:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x3e:0x0].0x0 (ffffa0730608b400) 00010000:00010000:3.0:1787739070.972408:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739070.972440:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1adf7f lrc: 2/0,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.972452:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1adf7f lrc: 0/0,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.972493:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1adf78 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095280 expref: 9 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.972505:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x3e:0x0].0x0 (ffffa0730608b400) 00010000:00010000:3.0:1787739070.972508:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739070.972511:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1adf78 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095280 expref: 9 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739070.972525:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab400, rc 0) 00000100:00100000:3.0:1787739070.972604:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7103c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17725:x1874579652508032:12345-192.168.203.15@tcp:101:ls.0 Request processed in 516us (726us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739070.972679:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23593 00000100:00100000:1.0:1787739070.976691:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739070.976892:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652508160 00000100:00100000:2.0:1787739070.976906:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652508160 00000100:00100000:2.0:1787739070.976985:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5015 00000100:00100000:2.0:1787739070.976998:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17725:x1874579652508160:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:2.0:1787739070.982134:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17725:x1874579652508160:12345-192.168.203.15@tcp:37:ls.0 Request processed in 5121us (5441us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739070.982154:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5015 00000100:00100000:3.0:1787739070.992024:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739070.994968:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652508288 00000100:00100000:1.0:1787739070.994989:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652508288 00000100:00100000:1.0:1787739070.995081:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23594 00000100:00100000:1.0:1787739070.995092:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304247480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:17725:x1874579652508288:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739070.995118:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739070.995150:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adf86 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095287 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739070.995196:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adf86 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095287 expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739070.995214:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adf86 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095287 expref: 9 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739070.995224:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x3e:0x0].0x0 (ffffa0730713af00) 00010000:00010000:1.0:1787739070.995228:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739070.995232:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fda00, rc 0) 00000100:00100000:1.0:1787739070.995319:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304247480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17725:x1874579652508288:12345-192.168.203.15@tcp:101:ls.0 Request processed in 228us (3307us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739070.995326:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23594 00000100:00100000:1.0:1787739071.006210:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739071.006322:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652508416 00000100:00100000:3.0:1787739071.006335:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652508416 00000100:00100000:3.0:1787739071.006456:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5016 00000100:00100000:3.0:1787739071.006470:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17725:x1874579652508416:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:3.0:1787739071.006661:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17725:x1874579652508416:12345-192.168.203.15@tcp:37:ls.0 Request processed in 194us (465us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739071.006670:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5016 00000100:00100000:2.0:1787739071.024506:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.024640:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652508672 00000100:00100000:1.0:1787739071.024652:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652508672 00000100:00100000:1.0:1787739071.024736:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5017 00000100:00100000:1.0:1787739071.024750:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c381a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17725:x1874579652508672:12345-192.168.203.15@tcp:35:ls.0 00000100:00100000:1.0:1787739071.025376:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c381a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17725:x1874579652508672:12345-192.168.203.15@tcp:35:ls.0 Request processed in 628us (883us total) trans 4294983438 rc 0/0 00000100:00100000:1.0:1787739071.025390:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5017 00000100:00100000:0.0:1787739071.029699:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739071.029862:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652508928 00000100:00100000:3.0:1787739071.029875:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652508928 00000100:00100000:3.0:1787739071.029956:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23595 00000100:00100000:3.0:1787739071.029979:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f711680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:17725:x1874579652508928:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:3.0:1787739071.030006:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739071.030033:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1adf8d lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09528e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030144:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1adf94 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030162:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1adf94 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030178:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1adf94 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030196:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1adf9b lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030203:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1adf9b lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030210:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1adf9b lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030283:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1adfa2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1467:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030291:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1adfa2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1467:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030299:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1adfa2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1467:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030321:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1adf94 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030329:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1adf94 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030335:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1adf94 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030341:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1adf94 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030348:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739071.030353:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1adf94 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030361:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1adf94 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030374:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:3.0:1787739071.030378:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739071.030383:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1adf94 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030390:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1adf94 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030407:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1adf9b lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030485:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1adf9b lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030492:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1adf9b lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030499:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739071.030502:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1adf9b lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030509:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1adf9b lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030517:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x4ac2f730 (ffffa0730608b400) 00010000:00010000:3.0:1787739071.030520:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739071.030523:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1adf9b lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030528:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa000/0x7ee4453e4a1adf9b lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030568:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1adf8d lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09528e expref: 11 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030585:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1adfa2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1467:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09528e expref: 10 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.030593:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1467:0x0].0x0 (ffffa0730608b000) 00010000:00010000:3.0:1787739071.030596:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739071.030598:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab800, rc 0) 00000100:00100000:3.0:1787739071.030676:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f711680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17725:x1874579652508928:12345-192.168.203.15@tcp:101:ls.0 Request processed in 700us (993us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739071.030688:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23595 00000100:00100000:2.0:1787739071.035797:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739071.035875:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652509184 00000100:00100000:0.0:1787739071.035888:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652509184 00000100:00100000:0.0:1787739071.035972:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23596 00000100:00100000:0.0:1787739071.035983:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c382580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17725:x1874579652509184:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:0.0:1787739071.036012:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739071.036039:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1adfa9 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1467:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095295 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739071.036720:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1467:0x0]: returning size 4096 00010000:00010000:0.0:1787739071.036841:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1adfa9 lrc: 1/0,0 mode: --/CR res: [0x200000402:0x1467:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095295 expref: 20 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.036859:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1467:0x0].0x0 (ffffa07305ca2800) 00010000:00010000:0.0:1787739071.036862:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739071.036867:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1adfa9 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1467:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095295 expref: 20 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.036887:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b800, rc 0) 00000100:00100000:0.0:1787739071.036973:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c382580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+19:17725:x1874579652509184:12345-192.168.203.15@tcp:101:ls.0 Request processed in 992us (1188us total) trans 4294983439 rc 301/301 00000100:00100000:0.0:1787739071.036981:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23596 00000100:00100000:0.0:1787739071.042306:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.042367:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652509312 00000100:00100000:2.0:1787739071.042380:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652509312 00000100:00100000:2.0:1787739071.042471:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23597 00000100:00100000:2.0:1787739071.042482:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+19:17725:x1874579652509312:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:2.0:1787739071.042513:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739071.042544:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1adfb7 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09529c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.042623:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1adfbe lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1467:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.042638:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1adfbe lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1467:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.042654:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1adfbe lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1467:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739071.042980:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1467:0x0]: returning size 4096 00000004:00200000:2.0:1787739071.042987:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:2.0:1787739071.043016:0:6524:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1adfbe lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1467:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.043045:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1adfb7 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09529c expref: 21 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.043073:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1adfbe lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1467:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09529c expref: 20 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.043084:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1467:0x0].0x0 (ffffa07403eeb300) 00010000:00010000:2.0:1787739071.043087:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739071.043090:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c400, rc 0) 00000100:00100000:2.0:1787739071.043334:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+20:17725:x1874579652509312:12345-192.168.203.15@tcp:101:ls.0 Request processed in 852us (1038us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.043345:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23597 00000100:00100000:3.0:1787739071.048948:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.049555:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652509440 00000100:00100000:1.0:1787739071.049569:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652509440 00000100:00100000:1.0:1787739071.049642:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23598 00000100:00100000:1.0:1787739071.049655:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304245a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+20:17725:x1874579652509440:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.049682:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.049712:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1adfc5 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x6e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0952a3 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.049803:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1adfcc lrc: 3/1,0 mode: --/PR res: [0x200000400:0x6e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.049820:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1adfcc lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.049833:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1adfcc lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739071.049897:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x6e:0x0]: returning size 4096 00000004:00200000:1.0:1787739071.049903:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739071.049950:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1adfc5 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952a3 expref: 22 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.049987:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1adfcc lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952a3 expref: 21 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.049998:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x6e:0x0].0x0 (ffffa0730713a800) 00010000:00010000:1.0:1787739071.050007:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739071.050013:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fcc00, rc 0) 00000100:00100000:1.0:1787739071.050097:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304245a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:17725:x1874579652509440:12345-192.168.203.15@tcp:101:ls.0 Request processed in 445us (1164us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.050105:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23598 00000100:00100000:1.0:1787739071.056697:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.056769:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652509568 00000100:00100000:2.0:1787739071.056782:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652509568 00000100:00100000:2.0:1787739071.056860:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23599 00000100:00100000:2.0:1787739071.056873:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:17725:x1874579652509568:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:2.0:1787739071.056900:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739071.056932:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1adfd3 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x6e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0952aa expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.057037:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adfda lrc: 3/1,0 mode: --/PR res: [0x240000401:0x6e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.057054:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adfda lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.057068:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adfda lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739071.057135:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x6e:0x0]: returning size 4096 00000004:00200000:2.0:1787739071.057141:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739071.057192:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d400/0x7ee4453e4a1adfd3 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x6e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952aa expref: 12 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.057230:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adfda lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952aa expref: 11 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.057240:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x6e:0x0].0x0 (ffffa07403eeb200) 00010000:00010000:2.0:1787739071.057244:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739071.057248:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633ce00, rc 0) 00000100:00100000:2.0:1787739071.057328:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17725:x1874579652509568:12345-192.168.203.15@tcp:101:ls.0 Request processed in 457us (645us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.057336:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23599 00000100:00100000:2.0:1787739071.071519:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.071709:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652509696 00000100:00100000:1.0:1787739071.071722:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652509696 00000100:00100000:1.0:1787739071.071803:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5018 00000100:00100000:1.0:1787739071.071815:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c3830c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:17725:x1874579652509696:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:1.0:1787739071.078355:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c3830c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:17725:x1874579652509696:12345-192.168.203.15@tcp:37:ls.0 Request processed in 6526us (6832us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.078372:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5018 00000100:00100000:2.0:1787739071.084788:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739071.084888:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652509824 00000100:00100000:3.0:1787739071.084900:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652509824 00000100:00100000:3.0:1787739071.084981:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5019 00000100:00100000:3.0:1787739071.084994:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074332ab450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17725:x1874579652509824:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:3.0:1787739071.090915:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074332ab450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17725:x1874579652509824:12345-192.168.203.15@tcp:37:ls.0 Request processed in 5906us (6183us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739071.090934:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5019 00000100:00100000:0.0:1787739071.094971:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.095095:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652510080 00000100:00100000:1.0:1787739071.095108:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652510080 00000100:00100000:1.0:1787739071.095227:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5020 00000100:00100000:1.0:1787739071.095239:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7112c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:17725:x1874579652510080:12345-192.168.203.15@tcp:35:ls.0 00000100:00100000:1.0:1787739071.095577:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7112c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+22:17725:x1874579652510080:12345-192.168.203.15@tcp:35:ls.0 Request processed in 340us (630us total) trans 4294983440 rc 0/0 00000100:00100000:1.0:1787739071.095585:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5020 00000100:00100000:2.0:1787739071.099682:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739071.099887:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652510336 00000100:00100000:0.0:1787739071.099901:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652510336 00000100:00100000:0.0:1787739071.099987:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23600 00000100:00100000:0.0:1787739071.099998:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c383480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:17725:x1874579652510336:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:0.0:1787739071.100027:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739071.100054:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1adfe1 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0952b1 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100158:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1adfe8 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100176:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1adfe8 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100189:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1adfe8 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100209:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adfef lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100217:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adfef lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100224:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adfef lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100306:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1adff6 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1470:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100315:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1adff6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1470:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100335:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1adff6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1470:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100356:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1adfe8 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100364:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1adfe8 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100370:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1adfe8 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100377:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1adfe8 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100384:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739071.100389:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1adfe8 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100399:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1adfe8 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100468:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:0.0:1787739071.100472:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739071.100477:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1adfe8 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100484:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a000/0x7ee4453e4a1adfe8 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100501:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adfef lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100508:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adfef lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100514:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adfef lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100519:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609a200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739071.100522:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adfef lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100528:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adfef lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100536:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x4a1b0d34 (ffffa07305ca2800) 00010000:00010000:0.0:1787739071.100538:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739071.100541:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adfef lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100551:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1adfef lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100594:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609b600/0x7ee4453e4a1adfe1 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952b1 expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100618:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1adff6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1470:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952b1 expref: 12 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.100625:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1470:0x0].0x0 (ffffa07305ca2200) 00010000:00010000:0.0:1787739071.100628:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739071.100630:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a400, rc 0) 00000100:00100000:0.0:1787739071.101020:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c383480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:17725:x1874579652510336:12345-192.168.203.15@tcp:101:ls.0 Request processed in 1013us (1343us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739071.101031:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23600 00000100:00100000:0.0:1787739071.106664:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739071.106956:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652510592 00000100:00100000:3.0:1787739071.106978:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652510592 00000100:00100000:3.0:1787739071.107056:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23601 00000100:00100000:3.0:1787739071.107069:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f710b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+22:17725:x1874579652510592:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:3.0:1787739071.107102:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739071.107135:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1adffd lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1470:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0952b8 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739071.107361:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1470:0x0]: returning size 4096 00010000:00010000:3.0:1787739071.107521:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1adffd lrc: 1/0,0 mode: --/CR res: [0x200000402:0x1470:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952b8 expref: 24 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.107541:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1470:0x0].0x0 (ffffa0730608b400) 00010000:00010000:3.0:1787739071.107544:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739071.107549:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1adffd lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1470:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952b8 expref: 24 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.107576:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa200, rc 0) 00000100:00100000:3.0:1787739071.107978:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f710b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+23:17725:x1874579652510592:12345-192.168.203.15@tcp:101:ls.0 Request processed in 911us (1329us total) trans 4294983441 rc 301/301 00000100:00100000:3.0:1787739071.107988:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23601 00000100:00100000:0.0:1787739071.114829:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.115245:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652510720 00000100:00100000:2.0:1787739071.115258:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652510720 00000100:00100000:2.0:1787739071.115345:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23602 00000100:00100000:2.0:1787739071.115357:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7121c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+23:17725:x1874579652510720:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:2.0:1787739071.115388:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739071.115440:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ae00b lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0952bf expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.115528:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ae012 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1470:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.115545:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ae012 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1470:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.115559:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ae012 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1470:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739071.115701:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1470:0x0]: returning size 4096 00000004:00200000:2.0:1787739071.115708:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:2.0:1787739071.115743:0:7944:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ae012 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1470:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.115773:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ae00b lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952bf expref: 25 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.115800:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ae012 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1470:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952bf expref: 24 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.115811:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1470:0x0].0x0 (ffffa07403eeb600) 00010000:00010000:2.0:1787739071.115814:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739071.115818:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633da00, rc 0) 00000100:00100000:2.0:1787739071.115889:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7121c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+24:17725:x1874579652510720:12345-192.168.203.15@tcp:101:ls.0 Request processed in 533us (1075us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.115896:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23602 00000100:00100000:2.0:1787739071.120874:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739071.121019:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652510848 00000100:00100000:0.0:1787739071.121032:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652510848 00000100:00100000:0.0:1787739071.121110:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23603 00000100:00100000:0.0:1787739071.121121:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c3821c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+24:17725:x1874579652510848:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:0.0:1787739071.121150:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739071.121181:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ae019 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x73:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0952c6 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.121268:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ae020 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x73:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.121287:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ae020 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x73:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.121301:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ae020 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x73:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739071.121374:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x73:0x0]: returning size 4096 00000004:00200000:0.0:1787739071.121381:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739071.121453:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ae019 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x73:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952c6 expref: 26 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.121474:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ae020 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x73:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952c6 expref: 25 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.121483:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x73:0x0].0x0 (ffffa07305ca2800) 00010000:00010000:0.0:1787739071.121486:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739071.121489:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a000, rc 0) 00000100:00100000:0.0:1787739071.121575:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c3821c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+25:17725:x1874579652510848:12345-192.168.203.15@tcp:101:ls.0 Request processed in 456us (716us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739071.121583:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23603 00000100:00100000:0.0:1787739071.125875:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739071.126021:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652510976 00000100:00100000:3.0:1787739071.126035:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652510976 00000100:00100000:3.0:1787739071.126502:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23604 00000100:00100000:3.0:1787739071.126514:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f711e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:17725:x1874579652510976:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:3.0:1787739071.126539:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739071.126568:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae027 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x73:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0952cd expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.126718:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae02e lrc: 3/1,0 mode: --/PR res: [0x240000401:0x73:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.126744:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae02e lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x73:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.126759:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae02e lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x73:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739071.126950:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x73:0x0]: returning size 4096 00000004:00200000:3.0:1787739071.126957:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739071.127017:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae027 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x73:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952cd expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.127039:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae02e lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x73:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952cd expref: 13 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.127050:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x73:0x0].0x0 (ffffa0730608b400) 00010000:00010000:3.0:1787739071.127053:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739071.127057:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab400, rc 0) 00000100:00100000:3.0:1787739071.127145:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f711e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17725:x1874579652510976:12345-192.168.203.15@tcp:101:ls.0 Request processed in 634us (1285us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739071.127153:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23604 00000100:00100000:3.0:1787739071.133337:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739071.133726:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652511104 00000100:00100000:0.0:1787739071.133740:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652511104 00000100:00100000:0.0:1787739071.133866:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5021 00000100:00100000:0.0:1787739071.133879:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304246580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+25:17725:x1874579652511104:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:0.0:1787739071.139207:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304246580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+25:17725:x1874579652511104:12345-192.168.203.15@tcp:37:ls.0 Request processed in 5312us (5866us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739071.139229:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5021 00000100:00100000:2.0:1787739071.144171:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.144271:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652511232 00000100:00100000:1.0:1787739071.144282:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652511232 00000100:00100000:1.0:1787739071.144349:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5022 00000100:00100000:1.0:1787739071.144359:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c380000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17725:x1874579652511232:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:1.0:1787739071.149343:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c380000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17725:x1874579652511232:12345-192.168.203.15@tcp:37:ls.0 Request processed in 4969us (5172us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.149360:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5022 00000100:00100000:3.0:1787739071.153912:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739071.154030:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652511488 00000100:00100000:0.0:1787739071.154044:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652511488 00000100:00100000:0.0:1787739071.154121:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5023 00000100:00100000:0.0:1787739071.154134:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f75e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+25:17725:x1874579652511488:12345-192.168.203.15@tcp:35:ls.0 00000100:00100000:0.0:1787739071.154507:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f75e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+26:17725:x1874579652511488:12345-192.168.203.15@tcp:35:ls.0 Request processed in 373us (608us total) trans 4294983442 rc 0/0 00000100:00100000:0.0:1787739071.154516:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5023 00000100:00100000:2.0:1787739071.159140:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.159300:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652511744 00000100:00100000:1.0:1787739071.159313:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652511744 00000100:00100000:1.0:1787739071.159402:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23605 00000100:00100000:1.0:1787739071.159456:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c380f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+26:17725:x1874579652511744:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.159493:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.159521:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1ae035 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0952d4 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.159630:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1ae03c lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.159648:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1ae03c lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.159663:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1ae03c lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.159680:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1ae043 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x649afea4 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.159687:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1ae043 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.159694:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1ae043 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739071.159879:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x13a5:0x0]: returning size 4096 00010000:00010000:1.0:1787739071.159986:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1ae03c lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.159996:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1ae03c lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.160004:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1ae03c lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.160011:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1ae03c lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.160018:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fc800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739071.160024:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1ae03c lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.160032:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1ae03c lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.160047:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5400) 00010000:00010000:1.0:1787739071.160051:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739071.160056:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1ae03c lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.160062:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1ae03c lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.160080:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1ae043 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.160087:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1ae043 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.160093:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1ae043 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.160097:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fd000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739071.160100:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1ae043 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.160105:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1ae043 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.160112:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x649afea4 (ffffa0730713a200) 00010000:00010000:1.0:1787739071.160114:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739071.160117:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1ae043 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.160122:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1ae043 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.160172:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1ae035 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952d4 expref: 28 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.160183:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5400) 00010000:00010000:1.0:1787739071.160185:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739071.160188:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1ae035 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952d4 expref: 28 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.160197:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fd600, rc 0) 00000100:00100000:1.0:1787739071.160279:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c380f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+27:17725:x1874579652511744:12345-192.168.203.15@tcp:101:ls.0 Request processed in 868us (1155us total) trans 4294983443 rc 301/301 00000100:00100000:1.0:1787739071.160288:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23605 00000100:00100000:1.0:1787739071.164533:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739071.164704:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652511872 00000100:00100000:3.0:1787739071.164715:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652511872 00000100:00100000:3.0:1787739071.164791:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23606 00000100:00100000:3.0:1787739071.164802:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+27:17725:x1874579652511872:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:3.0:1787739071.164829:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739071.164856:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae051 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x13a5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0952db expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.164991:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ae058 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x13a5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.165009:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ae058 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x13a5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.165022:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ae058 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x13a5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739071.165112:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x13a5:0x0]: returning size 4096 00000004:00200000:3.0:1787739071.165118:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:3.0:1787739071.165141:0:13793:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ae058 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x13a5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.165169:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae051 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x13a5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952db expref: 29 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.165188:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ae058 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x13a5:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952db expref: 28 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.165199:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13a5:0x0].0x0 (ffffa0730608b800) 00010000:00010000:3.0:1787739071.165202:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787739071.165205:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa000, rc 0) 00000100:00100000:3.0:1787739071.165281:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+28:17725:x1874579652511872:12345-192.168.203.15@tcp:101:ls.0 Request processed in 481us (760us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739071.165289:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23606 00000100:00100000:3.0:1787739071.169364:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.169590:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652512000 00000100:00100000:2.0:1787739071.169610:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652512000 00000100:00100000:2.0:1787739071.169675:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23607 00000100:00100000:2.0:1787739071.169686:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+28:17725:x1874579652512000:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:2.0:1787739071.169711:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739071.169882:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ae05f lrc: 2/0,0 mode: --/CR res: [0x200000400:0x11:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0952e2 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.169974:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ae066 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x11:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.169991:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ae066 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x11:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.170004:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ae066 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x11:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739071.170071:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x11:0x0]: returning size 4096 00000004:00200000:2.0:1787739071.170078:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739071.170125:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ae05f lrc: 0/0,0 mode: --/CR res: [0x200000400:0x11:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952e2 expref: 30 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.170146:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ae066 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952e2 expref: 29 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.170156:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x11:0x0].0x0 (ffffa07403eeb400) 00010000:00010000:2.0:1787739071.170159:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739071.170164:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633c600, rc 0) 00000100:00100000:2.0:1787739071.170281:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+29:17725:x1874579652512000:12345-192.168.203.15@tcp:101:ls.0 Request processed in 595us (929us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.170289:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23607 00000100:00100000:2.0:1787739071.174526:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739071.174773:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652512128 00000100:00100000:0.0:1787739071.174787:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652512128 00000100:00100000:0.0:1787739071.174870:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23608 00000100:00100000:0.0:1787739071.174883:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c381680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:17725:x1874579652512128:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:0.0:1787739071.174911:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739071.174942:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ae06d lrc: 2/0,0 mode: --/CR res: [0x240000401:0x11:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0952e9 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.175035:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ae074 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x11:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.175053:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ae074 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.175065:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ae074 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739071.175131:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x11:0x0]: returning size 4096 00000004:00200000:0.0:1787739071.175138:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739071.175187:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ae06d lrc: 0/0,0 mode: --/CR res: [0x240000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952e9 expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.175205:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ae074 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952e9 expref: 14 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.175215:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x11:0x0].0x0 (ffffa07305ca2300) 00010000:00010000:0.0:1787739071.175218:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739071.175223:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609aa00, rc 0) 00000100:00100000:0.0:1787739071.175308:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c381680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17725:x1874579652512128:12345-192.168.203.15@tcp:101:ls.0 Request processed in 428us (797us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739071.175315:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23608 00000100:00100000:1.0:1787739071.185716:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.185783:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652512256 00000100:00100000:2.0:1787739071.185796:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652512256 00000100:00100000:2.0:1787739071.185869:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5024 00000100:00100000:2.0:1787739071.185882:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074332a8c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+29:17725:x1874579652512256:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:2.0:1787739071.190312:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074332a8c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+29:17725:x1874579652512256:12345-192.168.203.15@tcp:37:ls.0 Request processed in 4418us (4591us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.190328:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5024 00000100:00100000:3.0:1787739071.194594:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.194997:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652512384 00000100:00100000:2.0:1787739071.195011:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652512384 00000100:00100000:2.0:1787739071.195084:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5025 00000100:00100000:2.0:1787739071.195095:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17725:x1874579652512384:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:2.0:1787739071.200352:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17725:x1874579652512384:12345-192.168.203.15@tcp:37:ls.0 Request processed in 5243us (5756us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.200372:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5025 00000100:00100000:1.0:1787739071.208135:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.208274:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652512640 00000100:00100000:2.0:1787739071.208287:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652512640 00000100:00100000:2.0:1787739071.208365:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5026 00000100:00100000:2.0:1787739071.208377:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+29:17725:x1874579652512640:12345-192.168.203.15@tcp:35:ls.0 00000040:00080000:2.0:1787739071.208796:0:6527:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].391, 344 off593896 00000040:00080000:2.0:1787739071.208864:0:6527:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].390, 424 off643160 00000004:00080000:2.0:1787739071.208871:0:6527:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306241600 with version 744 00000004:00080000:1.0:1787739071.209027:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 744 rpc_version 744 00000020:00100000:1.0:1787739071.209066:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739071.209079:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743562e940 x1874579665748864/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739071.209115:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562e940 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665748864:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739071.209192:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739071.209279:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435539d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739071.209449:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665748864 00000100:00100000:2.0:1787739071.209455:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665748864 00000100:00100000:2.0:1787739071.209495:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3503 00000100:00100000:2.0:1787739071.209503:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be43c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665748864:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739071.210024:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be43c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665748864:12345-0@lo:1000:osp_up1-0.0 Request processed in 519us (829us total) trans 4294968433 rc 0/0 00000100:00100000:2.0:1787739071.210034:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3503 00000100:00100000:1.0:1787739071.210108:0:7740:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968432 gen 2 00000004:00080000:1.0:1787739071.210115:0:7740:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968429 committed transno 4294968432 00000020:00080000:1.0:1787739071.210121:0:7740:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa0740a46c600 refcount 2 committed 0 result 0 batchid 392 00000020:00080000:1.0:1787739071.210125:0:7740:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2700 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa0740a46c0c0 00000020:00080000:1.0:1787739071.210128:0:7740:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2800 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073027db800 00000020:00080000:0.0:1787739071.210167:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 392 committed batchid 390 00000020:00080000:0.0:1787739071.210202:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 390 00000004:00080000:1.0:1787739071.210277:0:7740:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968430 committed transno 4294968432 00000020:00080000:1.0:1787739071.210282:0:7740:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07306887240 refcount 2 committed 0 result 0 batchid 393 00000020:00080000:1.0:1787739071.210286:0:7740:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073063edf00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07306887e40 00000020:00080000:1.0:1787739071.210289:0:7740:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].389 00000020:00080000:1.0:1787739071.210292:0:7740:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073063ed200 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304010200 00000020:00080000:1.0:1787739071.210295:0:7740:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].390 00000020:00080000:0.0:1787739071.210326:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 393 committed batchid 390 00000020:00080000:0.0:1787739071.210344:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 390 00000004:00080000:1.0:1787739071.210381:0:7740:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968431 committed transno 4294968432 00000020:00080000:1.0:1787739071.210385:0:7740:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07306248b40 refcount 2 committed 0 result 0 batchid 394 00000020:00080000:1.0:1787739071.210388:0:7740:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07302f96600 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa073062480c0 00000020:00080000:1.0:1787739071.210391:0:7740:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07302f96900 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304010800 00000020:00080000:0.0:1787739071.210429:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 394 committed batchid 390 00000020:00080000:0.0:1787739071.210445:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 390 00000004:00080000:1.0:1787739071.210553:0:7740:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968432 committed transno 4294968432 00000100:00100000:1.0:1787739071.210571:0:7740:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffffa0743562e940 x1874579665748864/t4294968433(4294968433) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 1787739087 ref 3 fl Rpc:RQU/204/0 rc 0/0 job:'osp_up1-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787739071.210645:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562e940 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665748864:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739071.210841:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:17725:x1874579652512640:12345-192.168.203.15@tcp:35:ls.0 Request processed in 2464us (2719us total) trans 4294983444 rc 0/0 00000100:00100000:2.0:1787739071.210852:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5026 00000100:00100000:2.0:1787739071.214797:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.214902:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652512896 00000100:00100000:1.0:1787739071.214915:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652512896 00000100:00100000:1.0:1787739071.214999:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23609 00000100:00100000:1.0:1787739071.215010:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c380000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:17725:x1874579652512896:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.215041:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.215125:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ae07b lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0952f0 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215224:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ae082 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215241:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ae082 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215256:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ae082 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215274:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae089 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x98a090de bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215283:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae089 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x98a090de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215290:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae089 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x98a090de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215566:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ae090 lrc: 3/1,0 mode: --/PR res: [0x240000402:0xe0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215575:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ae090 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xe0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215587:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ae090 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xe0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215673:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ae082 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215682:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ae082 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215688:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ae082 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215695:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ae082 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215702:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1cc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739071.215707:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ae082 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215715:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ae082 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215797:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5400) 00010000:00010000:1.0:1787739071.215801:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739071.215807:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ae082 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215814:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ae082 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215832:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae089 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x98a090de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215838:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae089 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x98a090de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215844:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae089 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x98a090de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215850:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ce00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739071.215853:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae089 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x98a090de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215860:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae089 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x98a090de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215867:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x98a090de (ffffa0730713a200) 00010000:00010000:1.0:1787739071.215870:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739071.215873:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae089 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x98a090de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215880:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae089 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x98a090de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215921:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ae07b lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952f0 expref: 32 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215936:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ae090 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0xe0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952f0 expref: 31 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.215943:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe0:0x0].0x0 (ffffa0730713ae00) 00010000:00010000:1.0:1787739071.215946:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739071.215949:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c000, rc 0) 00000100:00100000:1.0:1787739071.216161:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c380000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+31:17725:x1874579652512896:12345-192.168.203.15@tcp:101:ls.0 Request processed in 1151us (1560us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.216171:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23609 00000100:00100000:1.0:1787739071.220729:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.220866:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652513152 00000100:00100000:2.0:1787739071.220879:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652513152 00000100:00100000:2.0:1787739071.221073:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23610 00000100:00100000:2.0:1787739071.221086:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:17725:x1874579652513152:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:2.0:1787739071.221114:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739071.221143:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ae097 lrc: 2/0,0 mode: --/CR res: [0x240000402:0xe0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0952f7 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739071.221381:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xe0:0x0]: returning size 4096 00010000:00010000:2.0:1787739071.221532:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ae097 lrc: 1/0,0 mode: --/CR res: [0x240000402:0xe0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952f7 expref: 16 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.221552:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe0:0x0].0x0 (ffffa07403eeb900) 00010000:00010000:2.0:1787739071.221555:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739071.221561:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d200/0x7ee4453e4a1ae097 lrc: 0/0,0 mode: --/CR res: [0x240000402:0xe0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952f7 expref: 16 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.221582:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d200, rc 0) 00000100:00100000:2.0:1787739071.221671:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17725:x1874579652513152:12345-192.168.203.15@tcp:101:ls.0 Request processed in 588us (958us total) trans 4294968434 rc 301/301 00000100:00100000:2.0:1787739071.221679:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23610 00000100:00100000:2.0:1787739071.233610:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739071.233708:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652513280 00000100:00100000:3.0:1787739071.233722:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652513280 00000100:00100000:3.0:1787739071.233807:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23611 00000100:00100000:3.0:1787739071.233818:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:17725:x1874579652513280:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:3.0:1787739071.233850:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739071.233880:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae0a5 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0952fe expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.233980:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ae0ac lrc: 3/1,0 mode: --/PR res: [0x240000402:0xe0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.233997:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ae0ac lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xe0:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.234021:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ae0ac lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xe0:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739071.234144:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xe0:0x0]: returning size 4096 00000004:00200000:3.0:1787739071.234150:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:3.0:1787739071.234173:0:13793:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ae0ac lrc: 3/1,0 mode: PR/PR res: [0x240000402:0xe0:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.234207:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae0a5 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952fe expref: 17 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.234232:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ae0ac lrc: 3/0,0 mode: PR/PR res: [0x240000402:0xe0:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952fe expref: 16 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.234241:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe0:0x0].0x0 (ffffa0730608b700) 00010000:00010000:3.0:1787739071.234244:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739071.234248:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aae00, rc 0) 00000100:00100000:3.0:1787739071.234325:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c382940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17725:x1874579652513280:12345-192.168.203.15@tcp:101:ls.0 Request processed in 508us (731us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739071.234334:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23611 00000100:00100000:3.0:1787739071.238946:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.239102:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652513408 00000100:00100000:1.0:1787739071.239116:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652513408 00000100:00100000:1.0:1787739071.239250:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23612 00000100:00100000:1.0:1787739071.239263:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:17725:x1874579652513408:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.239289:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.239338:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae0b3 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x76:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095305 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.239454:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ae0ba lrc: 3/1,0 mode: --/PR res: [0x240000400:0x76:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.239470:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ae0ba lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x76:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.239485:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ae0ba lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x76:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739071.239542:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x76:0x0]: returning size 4096 00000004:00200000:1.0:1787739071.239548:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739071.239593:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae0b3 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x76:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095305 expref: 18 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.239616:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ae0ba lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x76:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095305 expref: 17 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.239633:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x76:0x0].0x0 (ffffa0730713a200) 00010000:00010000:1.0:1787739071.239637:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739071.239641:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1cc00, rc 0) 00000100:00100000:1.0:1787739071.239722:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17725:x1874579652513408:12345-192.168.203.15@tcp:101:ls.0 Request processed in 462us (790us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.239729:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23612 00000100:00100000:1.0:1787739071.251076:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739071.251271:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652513536 00000100:00100000:0.0:1787739071.251283:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652513536 00000100:00100000:0.0:1787739071.251365:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23613 00000100:00100000:0.0:1787739071.251376:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+31:17725:x1874579652513536:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:0.0:1787739071.251404:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739071.251457:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ae0c1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09530c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.251539:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ae0c8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.251556:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ae0c8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.251566:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ae0c8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739071.251633:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x76:0x0]: returning size 4096 00000004:00200000:0.0:1787739071.251640:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739071.251697:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ae0c1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x76:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09530c expref: 33 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.251720:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ae0c8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09530c expref: 32 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.251730:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffffa07305ca2000) 00010000:00010000:0.0:1787739071.251733:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739071.251737:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609ba00, rc 0) 00000100:00100000:0.0:1787739071.251886:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+32:17725:x1874579652513536:12345-192.168.203.15@tcp:101:ls.0 Request processed in 511us (825us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739071.251894:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23613 00000100:00100000:3.0:1787739071.265497:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.265668:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652513664 00000100:00100000:2.0:1787739071.265682:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652513664 00000100:00100000:2.0:1787739071.265746:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5027 00000100:00100000:2.0:1787739071.265757:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17725:x1874579652513664:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:2.0:1787739071.276173:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17725:x1874579652513664:12345-192.168.203.15@tcp:37:ls.0 Request processed in 9535us (9809us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.276195:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5027 00000100:00100000:1.0:1787739071.287782:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.288263:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652513792 00000100:00100000:2.0:1787739071.288280:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652513792 00000100:00100000:2.0:1787739071.288362:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5028 00000100:00100000:2.0:1787739071.288374:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+32:17725:x1874579652513792:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:2.0:1787739071.304120:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+32:17725:x1874579652513792:12345-192.168.203.15@tcp:37:ls.0 Request processed in 15733us (16342us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.304136:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5028 00000100:00100000:2.0:1787739071.311358:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.311777:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652514048 00000100:00100000:1.0:1787739071.311791:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652514048 00000100:00100000:1.0:1787739071.311897:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5029 00000100:00100000:1.0:1787739071.311907:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074332aac50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:17725:x1874579652514048:12345-192.168.203.15@tcp:35:ls.0 00000100:00100000:1.0:1787739071.312325:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074332aac50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17725:x1874579652514048:12345-192.168.203.15@tcp:35:ls.0 Request processed in 418us (981us total) trans 4294968435 rc 0/0 00000100:00100000:1.0:1787739071.312334:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5029 00000100:00100000:1.0:1787739071.317556:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739071.317655:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652514304 00000100:00100000:3.0:1787739071.317668:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652514304 00000100:00100000:3.0:1787739071.317748:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23614 00000100:00100000:3.0:1787739071.317759:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:17725:x1874579652514304:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:3.0:1787739071.317790:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739071.317819:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae0cf lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095313 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.317918:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae0d6 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.317937:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae0d6 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.317951:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae0d6 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.317979:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae0dd lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x44749f58 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.317986:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae0dd lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x44749f58 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.317992:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae0dd lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x44749f58 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.318063:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ae0e4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14c0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.318072:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ae0e4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14c0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.318079:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ae0e4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14c0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.318096:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae0d6 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.318103:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae0d6 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.318111:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae0d6 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.318119:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae0d6 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.318126:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739071.318131:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae0d6 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.318139:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae0d6 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.318153:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:3.0:1787739071.318156:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739071.318162:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae0d6 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.318168:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae0d6 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.318186:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae0dd lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x44749f58 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.318193:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae0dd lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x44749f58 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.318199:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae0dd lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x44749f58 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.318390:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739071.318394:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae0dd lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x44749f58 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.318400:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae0dd lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x44749f58 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.318409:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x44749f58 (ffffa0730608ba00) 00010000:00010000:3.0:1787739071.318707:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739071.318712:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae0dd lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x44749f58 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.318718:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae0dd lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x44749f58 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.318769:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae0cf lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095313 expref: 20 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.318783:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ae0e4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14c0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095313 expref: 19 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.318791:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14c0:0x0].0x0 (ffffa0730608bf00) 00010000:00010000:3.0:1787739071.318793:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739071.318796:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa400, rc 0) 00000100:00100000:2.0:1787739071.319630:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+19:17725:x1874579652514304:12345-192.168.203.15@tcp:101:ls.0 Request processed in 1865us (2080us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.319644:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23614 00000100:00100000:3.0:1787739071.332159:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.332324:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652514560 00000100:00100000:1.0:1787739071.332338:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652514560 00000100:00100000:1.0:1787739071.332478:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23615 00000100:00100000:1.0:1787739071.332506:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+32:17725:x1874579652514560:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.332534:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.332564:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ae0eb lrc: 2/0,0 mode: --/CR res: [0x200000402:0x14c0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09531a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739071.332786:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14c0:0x0]: returning size 4096 00010000:00010000:1.0:1787739071.332903:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ae0eb lrc: 1/0,0 mode: --/CR res: [0x200000402:0x14c0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09531a expref: 34 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.332920:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14c0:0x0].0x0 (ffffa0730713a600) 00010000:00010000:1.0:1787739071.332923:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739071.332929:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ae0eb lrc: 0/0,0 mode: --/CR res: [0x200000402:0x14c0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09531a expref: 34 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.332949:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d800, rc 0) 00000100:00100000:1.0:1787739071.333037:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+33:17725:x1874579652514560:12345-192.168.203.15@tcp:101:ls.0 Request processed in 550us (897us total) trans 4294983445 rc 301/301 00000100:00100000:1.0:1787739071.333045:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23615 00000100:00100000:3.0:1787739071.340856:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.341157:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652514688 00000100:00100000:1.0:1787739071.341171:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652514688 00000100:00100000:1.0:1787739071.341256:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23616 00000100:00100000:1.0:1787739071.341268:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+33:17725:x1874579652514688:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.341300:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.341330:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ae0f9 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095321 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.341465:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ae100 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14c0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.341481:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ae100 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14c0:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.341494:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ae100 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14c0:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739071.341612:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14c0:0x0]: returning size 4096 00000004:00200000:1.0:1787739071.341618:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:1.0:1787739071.341643:0:6525:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ae100 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14c0:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.341934:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ae0f9 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095321 expref: 35 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.341976:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ae100 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14c0:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095321 expref: 34 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.341987:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14c0:0x0].0x0 (ffffa0730713aa00) 00010000:00010000:1.0:1787739071.341990:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739071.341994:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c800, rc 0) 00000100:00100000:1.0:1787739071.342078:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+34:17725:x1874579652514688:12345-192.168.203.15@tcp:101:ls.0 Request processed in 812us (1236us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.342086:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23616 00000100:00100000:2.0:1787739071.347872:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.348014:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652514816 00000100:00100000:1.0:1787739071.348027:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652514816 00000100:00100000:1.0:1787739071.348108:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23617 00000100:00100000:1.0:1787739071.348120:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c3803c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+34:17725:x1874579652514816:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.348148:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.348179:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ae107 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x9d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095328 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.348260:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ae10e lrc: 3/1,0 mode: --/PR res: [0x200000400:0x9d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.348282:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ae10e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.348307:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ae10e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739071.348370:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x9d:0x0]: returning size 4096 00000004:00200000:1.0:1787739071.348376:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739071.348462:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ae107 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x9d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095328 expref: 36 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.348485:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ae10e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095328 expref: 35 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.348495:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x9d:0x0].0x0 (ffffa0730713a100) 00010000:00010000:1.0:1787739071.348497:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739071.348501:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d800, rc 0) 00000100:00100000:1.0:1787739071.348581:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c3803c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:17725:x1874579652514816:12345-192.168.203.15@tcp:101:ls.0 Request processed in 464us (724us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.348589:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23617 00000100:00100000:3.0:1787739071.353642:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739071.353869:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652514944 00000100:00100000:0.0:1787739071.353882:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652514944 00000100:00100000:0.0:1787739071.354006:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23618 00000100:00100000:0.0:1787739071.354018:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+19:17725:x1874579652514944:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:0.0:1787739071.354055:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739071.354087:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ae115 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x9d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09532f expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.354186:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ae11c lrc: 3/1,0 mode: --/PR res: [0x240000401:0x9d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.354203:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ae11c lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.354216:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ae11c lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739071.354281:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x9d:0x0]: returning size 4096 00000004:00200000:0.0:1787739071.354288:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739071.354348:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a200/0x7ee4453e4a1ae115 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x9d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09532f expref: 21 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.354375:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ae11c lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09532f expref: 20 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.354385:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x9d:0x0].0x0 (ffffa07305ca2b00) 00010000:00010000:0.0:1787739071.354388:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739071.354392:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a600, rc 0) 00000100:00100000:0.0:1787739071.354510:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+20:17725:x1874579652514944:12345-192.168.203.15@tcp:101:ls.0 Request processed in 494us (882us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739071.354519:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23618 00000100:00100000:2.0:1787739071.363672:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.363883:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652515072 00000100:00100000:1.0:1787739071.363894:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652515072 00000100:00100000:1.0:1787739071.364016:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5030 00000100:00100000:1.0:1787739071.364029:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c381a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:17725:x1874579652515072:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:1.0:1787739071.368536:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c381a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:17725:x1874579652515072:12345-192.168.203.15@tcp:37:ls.0 Request processed in 4497us (4866us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.368549:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5030 00000100:00100000:3.0:1787739071.373600:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.373668:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652515200 00000100:00100000:1.0:1787739071.373682:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652515200 00000100:00100000:1.0:1787739071.373755:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5031 00000100:00100000:1.0:1787739071.373766:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+20:17725:x1874579652515200:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:1.0:1787739071.377972:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+20:17725:x1874579652515200:12345-192.168.203.15@tcp:37:ls.0 Request processed in 4182us (4356us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.377988:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5031 00000100:00100000:2.0:1787739071.383152:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.383281:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652515456 00000100:00100000:1.0:1787739071.383294:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652515456 00000100:00100000:1.0:1787739071.383530:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5032 00000100:00100000:1.0:1787739071.383544:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c3830c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:17725:x1874579652515456:12345-192.168.203.15@tcp:35:ls.0 00000100:00100000:1.0:1787739071.383861:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c3830c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+36:17725:x1874579652515456:12345-192.168.203.15@tcp:35:ls.0 Request processed in 320us (725us total) trans 4294983446 rc 0/0 00000100:00100000:1.0:1787739071.383870:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5032 00000100:00100000:3.0:1787739071.388279:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.394760:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652515712 00000100:00100000:2.0:1787739071.394773:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652515712 00000100:00100000:2.0:1787739071.395108:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23619 00000100:00100000:2.0:1787739071.395119:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+36:17725:x1874579652515712:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:2.0:1787739071.395143:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739071.395163:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ae123 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095336 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.395238:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ae12a lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.395253:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ae12a lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.395267:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ae12a lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.395282:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae131 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xac50785c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.395290:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae131 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.395297:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae131 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739071.397541:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14c4:0x0]: returning size 4096 00010000:00010000:2.0:1787739071.397969:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ae12a lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.397982:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ae12a lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.397988:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ae12a lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.397994:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ae12a lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.398000:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739071.398004:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ae12a lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.398012:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ae12a lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.398242:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5400) 00010000:00010000:2.0:1787739071.398245:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739071.398250:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ae12a lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.398256:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d800/0x7ee4453e4a1ae12a lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.398271:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae131 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.398277:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae131 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.398282:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae131 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.398287:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739071.398290:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae131 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.398296:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae131 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.398303:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xac50785c (ffffa07403eeb900) 00010000:00010000:2.0:1787739071.398305:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739071.398308:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae131 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.398313:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae131 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.398363:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ae123 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095336 expref: 38 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.398374:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5400) 00010000:00010000:2.0:1787739071.398376:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739071.398379:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ae123 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095336 expref: 38 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.398388:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633de00, rc 0) 00000100:00100000:2.0:1787739071.398517:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+37:17725:x1874579652515712:12345-192.168.203.15@tcp:101:ls.0 Request processed in 3399us (10248us total) trans 4294983447 rc 301/301 00000100:00100000:2.0:1787739071.398528:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23619 00000100:00100000:2.0:1787739071.402358:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739071.402668:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652515840 00000100:00100000:3.0:1787739071.402679:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652515840 00000100:00100000:3.0:1787739071.402744:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23620 00000100:00100000:3.0:1787739071.402762:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c383840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+37:17725:x1874579652515840:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:3.0:1787739071.402790:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739071.403143:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae13f lrc: 2/0,0 mode: --/CR res: [0x200000402:0x14c4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09533d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.403254:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae146 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14c4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.403269:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae146 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14c4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.403287:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae146 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14c4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739071.403364:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14c4:0x0]: returning size 4096 00000004:00200000:3.0:1787739071.403369:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:3.0:1787739071.403392:0:6523:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae146 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14c4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.403676:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae13f lrc: 0/0,0 mode: --/CR res: [0x200000402:0x14c4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09533d expref: 39 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.403693:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae146 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14c4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09533d expref: 38 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.403701:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14c4:0x0].0x0 (ffffa0730608ba00) 00010000:00010000:3.0:1787739071.403703:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787739071.403707:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abe00, rc 0) 00000100:00100000:3.0:1787739071.403901:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c383840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+38:17725:x1874579652515840:12345-192.168.203.15@tcp:101:ls.0 Request processed in 1145us (1551us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739071.403912:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23620 00000100:00100000:1.0:1787739071.411671:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739071.412340:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652515968 00000100:00100000:0.0:1787739071.412354:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652515968 00000100:00100000:0.0:1787739071.412461:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23621 00000100:00100000:0.0:1787739071.412471:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+38:17725:x1874579652515968:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:0.0:1787739071.412494:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739071.412518:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ae14d lrc: 2/0,0 mode: --/CR res: [0x200000400:0x9f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095344 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.412577:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ae154 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x9f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.412590:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ae154 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.412601:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ae154 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739071.412649:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0x9f:0x0]: returning size 4096 00000004:00200000:0.0:1787739071.412655:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:0.0:1787739071.412691:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ae14d lrc: 0/0,0 mode: --/CR res: [0x200000400:0x9f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095344 expref: 40 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.412710:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ae154 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095344 expref: 39 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.412719:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x9f:0x0].0x0 (ffffa07305ca2a00) 00010000:00010000:0.0:1787739071.412722:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739071.412725:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609b600, rc 0) 00000100:00100000:0.0:1787739071.412807:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+39:17725:x1874579652515968:12345-192.168.203.15@tcp:101:ls.0 Request processed in 328us (1145us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739071.412814:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23621 00000100:00100000:3.0:1787739071.418865:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.419059:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652516096 00000100:00100000:2.0:1787739071.419074:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652516096 00000100:00100000:2.0:1787739071.419144:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23622 00000100:00100000:2.0:1787739071.419154:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+20:17725:x1874579652516096:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:2.0:1787739071.419173:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739071.419194:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae15b lrc: 2/0,0 mode: --/CR res: [0x240000401:0x9f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09534b expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.419257:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ae162 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x9f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.419276:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ae162 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.419288:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ae162 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739071.419336:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0x9f:0x0]: returning size 4096 00000004:00200000:2.0:1787739071.419341:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739071.419377:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae15b lrc: 0/0,0 mode: --/CR res: [0x240000401:0x9f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09534b expref: 22 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.419396:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ae162 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09534b expref: 21 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.419674:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x9f:0x0].0x0 (ffffa07403eeb900) 00010000:00010000:2.0:1787739071.419677:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739071.419682:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d800, rc 0) 00000100:00100000:2.0:1787739071.419747:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:17725:x1874579652516096:12345-192.168.203.15@tcp:101:ls.0 Request processed in 594us (896us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.419764:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23622 00000100:00100000:2.0:1787739071.427502:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.427666:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652516224 00000100:00100000:1.0:1787739071.427676:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652516224 00000100:00100000:1.0:1787739071.427736:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5033 00000100:00100000:1.0:1787739071.428158:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042443c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+39:17725:x1874579652516224:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:1.0:1787739071.434153:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042443c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+39:17725:x1874579652516224:12345-192.168.203.15@tcp:37:ls.0 Request processed in 6398us (6650us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.434166:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5033 00000100:00100000:1.0:1787739071.451040:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.451142:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652516352 00000100:00100000:2.0:1787739071.451153:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652516352 00000100:00100000:2.0:1787739071.451213:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5034 00000100:00100000:2.0:1787739071.451223:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074332a9450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:17725:x1874579652516352:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:1.0:1787739071.456275:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074332a9450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:17725:x1874579652516352:12345-192.168.203.15@tcp:37:ls.0 Request processed in 5042us (5234us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.456288:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5034 00000100:00100000:3.0:1787739071.472407:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.473648:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652516608 00000100:00100000:1.0:1787739071.473661:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652516608 00000100:00100000:1.0:1787739071.473725:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5035 00000100:00100000:1.0:1787739071.473735:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+39:17725:x1874579652516608:12345-192.168.203.15@tcp:35:ls.0 00000100:00100000:1.0:1787739071.473968:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+40:17725:x1874579652516608:12345-192.168.203.15@tcp:35:ls.0 Request processed in 225us (1562us total) trans 4294983448 rc 0/0 00000100:00100000:1.0:1787739071.473975:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5035 00000100:00100000:2.0:1787739071.477973:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.478280:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652516864 00000100:00100000:1.0:1787739071.478290:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652516864 00000100:00100000:1.0:1787739071.478362:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23623 00000100:00100000:1.0:1787739071.478381:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304246580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:17725:x1874579652516864:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.478401:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.478469:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae169 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095352 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478554:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae170 lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478573:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae170 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478585:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae170 lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478600:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ae177 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478608:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ae177 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478619:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ae177 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478679:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ae17e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14cd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478688:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ae17e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14cd:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478695:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ae17e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14cd:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478708:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae170 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478716:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae170 lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478723:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae170 lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478730:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae170 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478736:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1ce00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739071.478740:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae170 lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478747:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae170 lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478875:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:1.0:1787739071.478879:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739071.478885:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae170 lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478893:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae170 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478909:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ae177 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478914:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ae177 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478920:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ae177 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478925:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1c200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739071.478928:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ae177 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478934:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ae177 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478943:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x32e1d295 (ffffa0730713a600) 00010000:00010000:1.0:1787739071.478945:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739071.478948:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ae177 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478953:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ae177 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.478996:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae169 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095352 expref: 23 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.479015:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ae17e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14cd:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095352 expref: 22 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.479023:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14cd:0x0].0x0 (ffffa0730713a900) 00010000:00010000:1.0:1787739071.479025:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739071.479028:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1ca00, rc 0) 00000100:00100000:1.0:1787739071.479245:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304246580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+22:17725:x1874579652516864:12345-192.168.203.15@tcp:101:ls.0 Request processed in 865us (1288us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.479253:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23623 00000100:00100000:2.0:1787739071.488213:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.488995:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652517120 00000100:00100000:1.0:1787739071.489006:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652517120 00000100:00100000:1.0:1787739071.489107:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23624 00000100:00100000:1.0:1787739071.489118:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042443c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+40:17725:x1874579652517120:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.489141:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.489164:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae185 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x14cd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095359 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739071.489330:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14cd:0x0]: returning size 4096 00010000:00010000:1.0:1787739071.489470:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae185 lrc: 1/0,0 mode: --/CR res: [0x200000402:0x14cd:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095359 expref: 42 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.489485:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14cd:0x0].0x0 (ffffa0730713a600) 00010000:00010000:1.0:1787739071.489488:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739071.489493:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae185 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x14cd:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095359 expref: 42 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.489516:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d600, rc 0) 00000100:00100000:1.0:1787739071.489593:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042443c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+41:17725:x1874579652517120:12345-192.168.203.15@tcp:101:ls.0 Request processed in 477us (1396us total) trans 4294983449 rc 301/301 00000100:00100000:1.0:1787739071.493170:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23624 00000100:00100000:3.0:1787739071.501125:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.501238:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652517248 00000100:00100000:2.0:1787739071.501247:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652517248 00000100:00100000:2.0:1787739071.501308:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23625 00000100:00100000:2.0:1787739071.501325:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+41:17725:x1874579652517248:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:2.0:1787739071.501348:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739071.501371:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ae193 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095360 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.501477:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ae19a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14cd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.501493:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ae19a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14cd:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.501504:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ae19a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14cd:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739071.501605:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14cd:0x0]: returning size 4096 00000004:00200000:2.0:1787739071.501850:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:2.0:1787739071.501876:0:7944:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ae19a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14cd:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.501903:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ae193 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095360 expref: 43 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.501926:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ae19a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14cd:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095360 expref: 42 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.501935:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14cd:0x0].0x0 (ffffa07403eeb800) 00010000:00010000:2.0:1787739071.501937:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739071.501942:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d400, rc 0) 00000100:00100000:2.0:1787739071.502126:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+42:17725:x1874579652517248:12345-192.168.203.15@tcp:101:ls.0 Request processed in 809us (1010us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.502134:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23625 00000100:00100000:2.0:1787739071.517039:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.517104:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652517376 00000100:00100000:1.0:1787739071.517114:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652517376 00000100:00100000:1.0:1787739071.517174:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23626 00000100:00100000:1.0:1787739071.517185:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f75e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+42:17725:x1874579652517376:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.517206:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.517228:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae1a1 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xa3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095367 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.517286:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ae1a8 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xa3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.517299:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ae1a8 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.517323:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ae1a8 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739071.517367:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xa3:0x0]: returning size 4096 00000004:00200000:1.0:1787739071.517372:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739071.517461:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae1a1 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095367 expref: 44 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.517481:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ae1a8 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095367 expref: 43 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.517490:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xa3:0x0].0x0 (ffffa0730713a600) 00010000:00010000:1.0:1787739071.517493:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739071.517496:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c200, rc 0) 00000100:00100000:1.0:1787739071.517765:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f75e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+43:17725:x1874579652517376:12345-192.168.203.15@tcp:101:ls.0 Request processed in 573us (726us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.517774:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23626 00000100:00100000:2.0:1787739071.524449:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.524542:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652517504 00000100:00100000:1.0:1787739071.524558:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652517504 00000100:00100000:1.0:1787739071.524626:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23627 00000100:00100000:1.0:1787739071.524635:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+22:17725:x1874579652517504:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.524655:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.524677:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae1af lrc: 2/0,0 mode: --/CR res: [0x240000401:0xa3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09536e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.524741:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae1b6 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xa3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.524755:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae1b6 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.525161:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae1b6 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739071.525213:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xa3:0x0]: returning size 4096 00000004:00200000:1.0:1787739071.525218:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739071.525255:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae1af lrc: 0/0,0 mode: --/CR res: [0x240000401:0xa3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09536e expref: 24 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.525280:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae1b6 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09536e expref: 23 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.525288:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xa3:0x0].0x0 (ffffa0730713a300) 00010000:00010000:1.0:1787739071.525292:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739071.525295:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1ce00, rc 0) 00000100:00100000:1.0:1787739071.525985:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+23:17725:x1874579652517504:12345-192.168.203.15@tcp:101:ls.0 Request processed in 1347us (1567us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.525996:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23627 00000100:00100000:0.0:1787739071.536852:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.536974:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652517632 00000100:00100000:1.0:1787739071.536984:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652517632 00000100:00100000:1.0:1787739071.537160:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5036 00000100:00100000:1.0:1787739071.537169:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7112c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+43:17725:x1874579652517632:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:1.0:1787739071.544897:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7112c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+43:17725:x1874579652517632:12345-192.168.203.15@tcp:37:ls.0 Request processed in 7718us (8043us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.544911:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5036 00000100:00100000:3.0:1787739071.557535:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739071.557647:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652517760 00000100:00100000:0.0:1787739071.557658:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652517760 00000100:00100000:0.0:1787739071.557725:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5037 00000100:00100000:0.0:1787739071.557735:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+23:17725:x1874579652517760:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:2.0:1787739071.568152:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+23:17725:x1874579652517760:12345-192.168.203.15@tcp:37:ls.0 Request processed in 10407us (10614us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.568166:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5037 00000100:00100000:1.0:1787739071.576077:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.576244:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652518016 00000100:00100000:2.0:1787739071.576254:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652518016 00000100:00100000:2.0:1787739071.576454:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5038 00000100:00100000:2.0:1787739071.576467:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+43:17725:x1874579652518016:12345-192.168.203.15@tcp:35:ls.0 00000100:00100000:2.0:1787739071.576696:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+44:17725:x1874579652518016:12345-192.168.203.15@tcp:35:ls.0 Request processed in 231us (627us total) trans 4294983450 rc 0/0 00000100:00100000:2.0:1787739071.581507:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5038 00000100:00100000:3.0:1787739071.583877:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739071.584089:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652518272 00000100:00100000:0.0:1787739071.584099:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652518272 00000100:00100000:0.0:1787739071.584155:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23628 00000100:00100000:0.0:1787739071.584165:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060d2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+44:17725:x1874579652518272:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:0.0:1787739071.584188:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739071.584209:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ae1bd lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095375 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584283:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ae1c4 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584297:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ae1c4 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584311:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ae1c4 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584325:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ae1cb lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xe1386388 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584332:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ae1cb lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xe1386388 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584338:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ae1cb lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xe1386388 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584396:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ae1d2 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584404:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ae1d2 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x21:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584439:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ae1d2 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x21:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584453:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ae1c4 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584460:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ae1c4 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584466:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ae1c4 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584472:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ae1c4 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584477:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739071.584481:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ae1c4 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584488:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ae1c4 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584500:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5400) 00010000:00010000:0.0:1787739071.584503:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739071.584507:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ae1c4 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584512:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ae1c4 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584526:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ae1cb lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xe1386388 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584532:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ae1cb lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xe1386388 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584538:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ae1cb lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xe1386388 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584544:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609be00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739071.584546:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ae1cb lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xe1386388 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584553:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ae1cb lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xe1386388 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584560:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xe1386388 (ffffa07305ca2700) 00010000:00010000:0.0:1787739071.584562:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739071.584564:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ae1cb lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xe1386388 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584569:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ae1cb lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xe1386388 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584600:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ae1bd lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095375 expref: 46 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584613:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ae1d2 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x21:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095375 expref: 45 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.584624:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x21:0x0].0x0 (ffffa07305ca2d00) 00010000:00010000:0.0:1787739071.584626:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739071.584628:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730609a200, rc 0) 00000100:00100000:0.0:1787739071.584696:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060d2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+45:17725:x1874579652518272:12345-192.168.203.15@tcp:101:ls.0 Request processed in 533us (827us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739071.584704:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23628 00000100:00100000:0.0:1787739071.593878:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739071.594030:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652518528 00000100:00100000:3.0:1787739071.594040:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652518528 00000100:00100000:3.0:1787739071.594104:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23629 00000100:00100000:3.0:1787739071.594114:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306390f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+23:17725:x1874579652518528:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:3.0:1787739071.594133:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739071.594155:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae1d9 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09537c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739071.594327:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x21:0x0]: returning size 4096 00010000:00010000:3.0:1787739071.594464:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae1d9 lrc: 1/0,0 mode: --/CR res: [0x240000402:0x21:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09537c expref: 25 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.594478:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x21:0x0].0x0 (ffffa0730608b200) 00010000:00010000:3.0:1787739071.594481:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739071.594486:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae1d9 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x21:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09537c expref: 25 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.594503:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab200, rc 0) 00000100:00100000:3.0:1787739071.594560:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306390f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+24:17725:x1874579652518528:12345-192.168.203.15@tcp:101:ls.0 Request processed in 448us (693us total) trans 4294968436 rc 301/301 00000100:00100000:3.0:1787739071.594572:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23629 00000100:00100000:3.0:1787739071.600271:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.600460:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652518656 00000100:00100000:1.0:1787739071.600469:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652518656 00000100:00100000:1.0:1787739071.600520:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23630 00000100:00100000:1.0:1787739071.600530:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+24:17725:x1874579652518656:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.600555:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.600577:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae1e7 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095383 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.600662:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ae1ee lrc: 3/1,0 mode: --/PR res: [0x240000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.600677:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ae1ee lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.600688:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ae1ee lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739071.600790:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x21:0x0]: returning size 4096 00000004:00200000:1.0:1787739071.600795:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:1.0:1787739071.601044:0:13793:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ae1ee lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.601075:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae1e7 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095383 expref: 26 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.601098:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ae1ee lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095383 expref: 25 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.601107:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x21:0x0].0x0 (ffffa0730713ad00) 00010000:00010000:1.0:1787739071.601110:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739071.601113:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1da00, rc 0) 00000100:00100000:1.0:1787739071.601217:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+25:17725:x1874579652518656:12345-192.168.203.15@tcp:101:ls.0 Request processed in 689us (955us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.601225:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23630 00000100:00100000:0.0:1787739071.608167:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.608277:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652518784 00000100:00100000:1.0:1787739071.608286:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652518784 00000100:00100000:1.0:1787739071.608348:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23631 00000100:00100000:1.0:1787739071.608359:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306392d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+25:17725:x1874579652518784:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.608378:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.608402:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ae1f5 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x14:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09538a expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.608490:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ae1fc lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.608514:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ae1fc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.608525:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ae1fc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739071.608567:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x14:0x0]: returning size 4096 00000004:00200000:1.0:1787739071.608572:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739071.608618:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ae1f5 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x14:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09538a expref: 27 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.608637:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ae1fc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09538a expref: 26 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.608646:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x14:0x0].0x0 (ffffa0730713ac00) 00010000:00010000:1.0:1787739071.608649:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739071.608652:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1dc00, rc 0) 00000100:00100000:1.0:1787739071.608706:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306392d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+26:17725:x1874579652518784:12345-192.168.203.15@tcp:101:ls.0 Request processed in 350us (550us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.608713:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23631 00000100:00100000:0.0:1787739071.619580:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.619671:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652518912 00000100:00100000:2.0:1787739071.619681:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652518912 00000100:00100000:2.0:1787739071.619741:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23632 00000100:00100000:2.0:1787739071.619752:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073040030c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+45:17725:x1874579652518912:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:2.0:1787739071.619771:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739071.619795:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae203 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x14:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095391 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.619870:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ae20a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x14:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.619888:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ae20a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.619899:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ae20a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739071.619948:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x14:0x0]: returning size 4096 00000004:00200000:2.0:1787739071.619952:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739071.619999:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae203 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x14:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095391 expref: 47 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.620017:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ae20a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x14:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095391 expref: 46 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.620025:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x14:0x0].0x0 (ffffa07403eeb700) 00010000:00010000:2.0:1787739071.620028:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739071.620031:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633de00, rc 0) 00000100:00100000:2.0:1787739071.620097:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073040030c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+46:17725:x1874579652518912:12345-192.168.203.15@tcp:101:ls.0 Request processed in 347us (527us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.620103:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23632 00000100:00100000:2.0:1787739071.629653:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739071.629901:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652519040 00000100:00100000:0.0:1787739071.629910:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652519040 00000100:00100000:0.0:1787739071.629977:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5039 00000100:00100000:0.0:1787739071.629988:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074332a8850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+26:17725:x1874579652519040:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:0.0:1787739071.635808:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074332a8850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+26:17725:x1874579652519040:12345-192.168.203.15@tcp:37:ls.0 Request processed in 5803us (6144us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739071.635823:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5039 00000100:00100000:3.0:1787739071.641360:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.641453:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652519168 00000100:00100000:2.0:1787739071.641463:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652519168 00000100:00100000:2.0:1787739071.641561:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5040 00000100:00100000:2.0:1787739071.641571:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+46:17725:x1874579652519168:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:0.0:1787739071.651785:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+46:17725:x1874579652519168:12345-192.168.203.15@tcp:37:ls.0 Request processed in 10204us (10423us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739071.651922:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5040 00000100:00100000:3.0:1787739071.657516:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739071.657682:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652519424 00000100:00100000:0.0:1787739071.657691:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652519424 00000100:00100000:0.0:1787739071.657761:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5041 00000100:00100000:0.0:1787739071.657772:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+26:17725:x1874579652519424:12345-192.168.203.15@tcp:35:ls.0 00000040:00080000:0.0:1787739071.658040:0:6526:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].372, 344 off563064 00000040:00080000:0.0:1787739071.658088:0:6526:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].372, 424 off614400 00000004:00080000:0.0:1787739071.658094:0:6526:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07306234180 with version 711 00000004:00080000:3.0:1787739071.658448:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 711 rpc_version 711 00000020:00100000:3.0:1787739071.658477:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739071.658487:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743c41e580 x1874579665749120/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739071.658509:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c41e580 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665749120:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739071.658582:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739071.658664:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410cf6d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739071.659264:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665749120 00000100:00100000:0.0:1787739071.659273:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665749120 00000100:00100000:0.0:1787739071.659336:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3504 00000100:00100000:0.0:1787739071.659344:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410b08850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+8:7467:x1874579665749120:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739071.659969:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410b08850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665749120:12345-0@lo:1000:osp_up0-1.0 Request processed in 614us (1374us total) trans 4294983451 rc 0/0 00000100:00100000:0.0:1787739071.659979:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3504 00000100:00100000:2.0:1787739071.660272:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c41e580 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665749120:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739071.661078:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+27:17725:x1874579652519424:12345-192.168.203.15@tcp:35:ls.0 Request processed in 3304us (3567us total) trans 4294968437 rc 0/0 00000100:00100000:0.0:1787739071.661090:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5041 00000100:00100000:3.0:1787739071.668214:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.668300:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652519680 00000100:00100000:1.0:1787739071.668315:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652519680 00000100:00100000:1.0:1787739071.668376:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23633 00000100:00100000:1.0:1787739071.668387:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+46:17725:x1874579652519680:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.668449:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.668470:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ae211 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095398 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.668543:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae218 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.668567:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae218 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.668578:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae218 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.668593:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae21f lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.668599:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae21f lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.668604:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae21f lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739071.668726:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14e6:0x0]: returning size 4096 00010000:00010000:1.0:1787739071.669037:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae218 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.669045:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae218 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.669051:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae218 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.669057:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae218 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.669062:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1de00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739071.669066:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae218 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.669074:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae218 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.669086:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5400) 00010000:00010000:1.0:1787739071.669089:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739071.669094:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae218 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.669100:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae218 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.669115:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae21f lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.669122:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae21f lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.669127:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae21f lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.669132:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739071.669135:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae21f lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.669141:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae21f lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.669148:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x3fb0b00e (ffffa0730713a500) 00010000:00010000:1.0:1787739071.669151:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739071.669153:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae21f lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.669159:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae21f lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.669200:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ae211 lrc: 1/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095398 expref: 48 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.669210:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5400) 00010000:00010000:1.0:1787739071.669212:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739071.669215:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ae211 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095398 expref: 48 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.669225:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c600, rc 0) 00000100:00100000:1.0:1787739071.669285:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+47:17725:x1874579652519680:12345-192.168.203.15@tcp:101:ls.0 Request processed in 900us (1081us total) trans 4294983452 rc 301/301 00000100:00100000:1.0:1787739071.669293:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23633 00000100:00100000:2.0:1787739071.677045:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739071.677148:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652519808 00000100:00100000:3.0:1787739071.677159:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652519808 00000100:00100000:2.0:1787739071.677239:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23634 00000100:00100000:2.0:1787739071.677250:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+47:17725:x1874579652519808:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:2.0:1787739071.677271:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739071.677291:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ae22d lrc: 2/0,0 mode: --/CR res: [0x200000402:0x14e6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09539f expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.677581:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ae234 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14e6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.677595:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ae234 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14e6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.677672:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ae234 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14e6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739071.677816:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14e6:0x0]: returning size 4096 00000004:00200000:2.0:1787739071.677868:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:2.0:1787739071.678011:0:7944:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ae234 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14e6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.678043:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ae22d lrc: 0/0,0 mode: --/CR res: [0x200000402:0x14e6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09539f expref: 49 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.678063:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ae234 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14e6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09539f expref: 48 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.678072:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14e6:0x0].0x0 (ffffa07403eeb100) 00010000:00010000:2.0:1787739071.678075:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787739071.678079:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d200, rc 0) 00000100:00100000:2.0:1787739071.678551:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+48:17725:x1874579652519808:12345-192.168.203.15@tcp:101:ls.0 Request processed in 1300us (1512us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.678560:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23634 00000100:00100000:0.0:1787739071.690200:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.690340:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652519936 00000100:00100000:2.0:1787739071.690351:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652519936 00000100:00100000:3.0:1787739071.690565:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23635 00000100:00100000:3.0:1787739071.690578:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304001e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+48:17725:x1874579652519936:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:3.0:1787739071.690712:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739071.690799:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae23b lrc: 2/0,0 mode: --/CR res: [0x200000400:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0953a6 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.690874:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae242 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.690889:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae242 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.690900:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae242 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739071.690950:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xb0:0x0]: returning size 4096 00000004:00200000:3.0:1787739071.690963:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739071.691000:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae23b lrc: 0/0,0 mode: --/CR res: [0x200000400:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953a6 expref: 50 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.691020:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae242 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953a6 expref: 49 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.691028:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb0:0x0].0x0 (ffffa0730608b200) 00010000:00010000:3.0:1787739071.691031:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739071.691034:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aaa00, rc 0) 00000100:00100000:3.0:1787739071.691101:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304001e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+49:17725:x1874579652519936:12345-192.168.203.15@tcp:101:ls.0 Request processed in 526us (911us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739071.691109:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23635 00000100:00100000:3.0:1787739071.700489:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.700693:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652520064 00000100:00100000:1.0:1787739071.700703:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652520064 00000100:00100000:1.0:1787739071.700761:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23636 00000100:00100000:1.0:1787739071.700771:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+27:17725:x1874579652520064:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.700796:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.700821:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ae249 lrc: 2/0,0 mode: --/CR res: [0x240000401:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0953ad expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.700891:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae250 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.700904:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae250 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.700915:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae250 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739071.700976:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xb0:0x0]: returning size 4096 00000004:00200000:1.0:1787739071.700981:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739071.701018:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ae249 lrc: 0/0,0 mode: --/CR res: [0x240000401:0xb0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953ad expref: 29 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.701038:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae250 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953ad expref: 28 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.701046:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb0:0x0].0x0 (ffffa0730713a500) 00010000:00010000:1.0:1787739071.701049:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739071.701051:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d600, rc 0) 00000100:00100000:1.0:1787739071.701208:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+28:17725:x1874579652520064:12345-192.168.203.15@tcp:101:ls.0 Request processed in 438us (729us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.701215:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23636 00000100:00100000:2.0:1787739071.717100:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739071.717194:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652520192 00000100:00100000:0.0:1787739071.717204:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652520192 00000100:00100000:0.0:1787739071.717263:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5042 00000100:00100000:0.0:1787739071.717272:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+49:17725:x1874579652520192:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:2.0:1787739071.723922:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+49:17725:x1874579652520192:12345-192.168.203.15@tcp:37:ls.0 Request processed in 6637us (6819us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.723948:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5042 00000100:00100000:2.0:1787739071.730334:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739071.730465:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652520320 00000100:00100000:0.0:1787739071.730476:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652520320 00000100:00100000:0.0:1787739071.730527:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5043 00000100:00100000:0.0:1787739071.730538:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+28:17725:x1874579652520320:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:0.0:1787739071.736905:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+28:17725:x1874579652520320:12345-192.168.203.15@tcp:37:ls.0 Request processed in 6358us (6569us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739071.736919:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5043 00000100:00100000:3.0:1787739071.741480:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.742087:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652520576 00000100:00100000:2.0:1787739071.742100:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652520576 00000100:00100000:2.0:1787739071.742165:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5044 00000100:00100000:2.0:1787739071.742175:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074332aa850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+49:17725:x1874579652520576:12345-192.168.203.15@tcp:35:ls.0 00000100:00100000:2.0:1787739071.742563:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074332aa850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+50:17725:x1874579652520576:12345-192.168.203.15@tcp:35:ls.0 Request processed in 388us (1090us total) trans 4294983453 rc 0/0 00000100:00100000:2.0:1787739071.742572:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5044 00000100:00100000:2.0:1787739071.749231:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.750811:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652520832 00000100:00100000:1.0:1787739071.750825:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652520832 00000100:00100000:1.0:1787739071.750965:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23637 00000100:00100000:1.0:1787739071.750978:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+28:17725:x1874579652520832:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.751001:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.751022:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ae257 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0953b4 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751097:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae25e lrc: 3/1,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751111:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae25e lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751123:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae25e lrc: 3/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751140:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ae265 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751148:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ae265 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751154:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ae265 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751213:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd600/0x7ee4453e4a1ae26c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1518:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751220:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd600/0x7ee4453e4a1ae26c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751227:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd600/0x7ee4453e4a1ae26c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751243:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae25e lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751248:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae25e lrc: 4/1,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751255:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae25e lrc: 3/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751261:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae25e lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751266:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1de00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739071.751270:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae25e lrc: 4/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751277:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae25e lrc: 5/0,0 mode: CR/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751288:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:1.0:1787739071.751291:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739071.751296:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae25e lrc: 3/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751303:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae25e lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751317:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ae265 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751322:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ae265 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751328:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ae265 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751333:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07441a1d000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739071.751335:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ae265 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751340:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ae265 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751347:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xda636067 (ffffa0730713a700) 00010000:00010000:1.0:1787739071.751349:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739071.751352:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ae265 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751356:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ae265 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751390:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ae257 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953b4 expref: 30 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751458:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd600/0x7ee4453e4a1ae26c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953b4 expref: 29 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.751465:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1518:0x0].0x0 (ffffa0730713a000) 00010000:00010000:1.0:1787739071.751468:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739071.751470:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fd600, rc 0) 00000100:00100000:1.0:1787739071.751534:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+29:17725:x1874579652520832:12345-192.168.203.15@tcp:101:ls.0 Request processed in 560us (2313us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.751545:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23637 00000100:00100000:3.0:1787739071.756250:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.756334:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652521088 00000100:00100000:1.0:1787739071.756346:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652521088 00000100:00100000:1.0:1787739071.756454:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23638 00000100:00100000:1.0:1787739071.756466:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+50:17725:x1874579652521088:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.756489:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.756510:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1ae273 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1518:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0953bb expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739071.756676:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1518:0x0]: returning size 4096 00010000:00010000:1.0:1787739071.756775:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1ae273 lrc: 1/0,0 mode: --/CR res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953bb expref: 52 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.756790:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1518:0x0].0x0 (ffffa0730713a700) 00010000:00010000:1.0:1787739071.756794:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739071.756799:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1ae273 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953bb expref: 52 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.756816:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fd000, rc 0) 00000100:00100000:1.0:1787739071.756882:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+51:17725:x1874579652521088:12345-192.168.203.15@tcp:101:ls.0 Request processed in 418us (641us total) trans 4294983454 rc 301/301 00000100:00100000:1.0:1787739071.756889:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23638 00000100:00100000:0.0:1787739071.762510:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.762622:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652521216 00000100:00100000:1.0:1787739071.762632:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652521216 00000100:00100000:1.0:1787739071.762683:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23639 00000100:00100000:1.0:1787739071.762693:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073040003c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+51:17725:x1874579652521216:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.762717:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.762748:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1ae281 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0953c2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.762817:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ae288 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1518:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.762831:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ae288 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1518:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.762844:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ae288 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1518:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739071.762929:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1518:0x0]: returning size 4096 00000004:00200000:1.0:1787739071.762933:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:1.0:1787739071.762953:0:13793:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ae288 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1518:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.762987:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1ae281 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953c2 expref: 53 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.763010:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ae288 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1518:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953c2 expref: 52 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.763018:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1518:0x0].0x0 (ffffa07305eaeb00) 00010000:00010000:1.0:1787739071.763021:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739071.763024:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1c600, rc 0) 00000100:00100000:1.0:1787739071.763088:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073040003c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+52:17725:x1874579652521216:12345-192.168.203.15@tcp:101:ls.0 Request processed in 395us (587us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.763095:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23639 00000100:00100000:2.0:1787739071.768470:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.768541:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652521344 00000100:00100000:1.0:1787739071.768549:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652521344 00000100:00100000:1.0:1787739071.768596:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23640 00000100:00100000:1.0:1787739071.768613:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+52:17725:x1874579652521344:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.768632:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.768653:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ae28f lrc: 2/0,0 mode: --/CR res: [0x200000400:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0953c9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.768708:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae296 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.768721:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae296 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.768741:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae296 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739071.768784:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xc6:0x0]: returning size 4096 00000004:00200000:1.0:1787739071.768789:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739071.768819:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ae28f lrc: 0/0,0 mode: --/CR res: [0x200000400:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953c9 expref: 54 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.768838:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae296 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953c9 expref: 53 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.768847:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc6:0x0].0x0 (ffffa07305eaea00) 00010000:00010000:1.0:1787739071.768849:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739071.768852:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1de00, rc 0) 00000100:00100000:1.0:1787739071.768898:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+53:17725:x1874579652521344:12345-192.168.203.15@tcp:101:ls.0 Request processed in 295us (437us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.768908:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23640 00000100:00100000:3.0:1787739071.776340:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.776441:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652521472 00000100:00100000:1.0:1787739071.776451:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652521472 00000100:00100000:1.0:1787739071.776512:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23641 00000100:00100000:1.0:1787739071.776522:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+29:17725:x1874579652521472:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.776541:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.776563:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ae29d lrc: 2/0,0 mode: --/CR res: [0x240000401:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0953d0 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.776633:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc800/0x7ee4453e4a1ae2a4 lrc: 3/1,0 mode: --/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.776647:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc800/0x7ee4453e4a1ae2a4 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.776657:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc800/0x7ee4453e4a1ae2a4 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739071.776700:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000401:0xc6:0x0]: returning size 4096 00000004:00200000:1.0:1787739071.776704:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739071.776746:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ae29d lrc: 0/0,0 mode: --/CR res: [0x240000401:0xc6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953d0 expref: 31 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.776764:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc800/0x7ee4453e4a1ae2a4 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953d0 expref: 30 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.776771:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc6:0x0].0x0 (ffffa07305eaee00) 00010000:00010000:1.0:1787739071.776774:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739071.776777:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fc800, rc 0) 00000100:00100000:1.0:1787739071.776836:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:17725:x1874579652521472:12345-192.168.203.15@tcp:101:ls.0 Request processed in 316us (516us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.776844:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23641 00000100:00100000:0.0:1787739071.784837:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.784940:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652521600 00000100:00100000:2.0:1787739071.784950:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652521600 00000100:00100000:2.0:1787739071.785018:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5045 00000100:00100000:2.0:1787739071.785029:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304002d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+53:17725:x1874579652521600:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:2.0:1787739071.789723:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304002d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+53:17725:x1874579652521600:12345-192.168.203.15@tcp:37:ls.0 Request processed in 4686us (4886us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.793503:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5045 00000100:00100000:3.0:1787739071.794844:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.795073:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652521728 00000100:00100000:2.0:1787739071.795083:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652521728 00000100:00100000:2.0:1787739071.795137:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5046 00000100:00100000:2.0:1787739071.795147:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:17725:x1874579652521728:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:2.0:1787739071.800368:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41d2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:17725:x1874579652521728:12345-192.168.203.15@tcp:37:ls.0 Request processed in 5213us (5519us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.800382:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5046 00000100:00100000:0.0:1787739071.805386:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.805624:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652521984 00000100:00100000:2.0:1787739071.805634:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652521984 00000100:00100000:2.0:1787739071.805691:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5047 00000100:00100000:2.0:1787739071.805701:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304000b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+53:17725:x1874579652521984:12345-192.168.203.15@tcp:35:ls.0 00000100:00100000:2.0:1787739071.805934:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304000b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+54:17725:x1874579652521984:12345-192.168.203.15@tcp:35:ls.0 Request processed in 234us (557us total) trans 4294983455 rc 0/0 00000100:00100000:2.0:1787739071.805943:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5047 00010000:00010000:2.0:1787739071.810702:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739071.812752:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.812962:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652522240 00000100:00100000:2.0:1787739071.812972:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652522240 00000100:00100000:2.0:1787739071.813031:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23642 00000100:00100000:2.0:1787739071.813041:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+54:17725:x1874579652522240:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:2.0:1787739071.813064:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739071.813084:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ae2ab lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0953d7 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813153:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae2b2 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813168:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae2b2 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813183:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae2b2 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813200:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031200/0x7ee4453e4a1ae2b9 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813208:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031200/0x7ee4453e4a1ae2b9 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813215:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031200/0x7ee4453e4a1ae2b9 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813268:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031400/0x7ee4453e4a1ae2c0 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x17c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813276:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031400/0x7ee4453e4a1ae2c0 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813283:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031400/0x7ee4453e4a1ae2c0 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813296:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae2b2 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813302:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae2b2 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813307:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae2b2 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813313:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae2b2 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813324:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633d600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739071.813328:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae2b2 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813335:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae2b2 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813346:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5400) 00010000:00010000:2.0:1787739071.813348:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739071.813352:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae2b2 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813358:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae2b2 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813372:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031200/0x7ee4453e4a1ae2b9 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813377:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031200/0x7ee4453e4a1ae2b9 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813383:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031200/0x7ee4453e4a1ae2b9 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813388:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07400031200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739071.813391:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031200/0x7ee4453e4a1ae2b9 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813409:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031200/0x7ee4453e4a1ae2b9 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813475:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd1c5f96a (ffffa07403eebc00) 00010000:00010000:2.0:1787739071.813477:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739071.813480:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031200/0x7ee4453e4a1ae2b9 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813486:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031200/0x7ee4453e4a1ae2b9 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813519:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ae2ab lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953d7 expref: 56 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813532:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031400/0x7ee4453e4a1ae2c0 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953d7 expref: 55 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739071.813539:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17c:0x0].0x0 (ffffa07403eebf00) 00010000:00010000:2.0:1787739071.813541:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739071.813543:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07400031400, rc 0) 00000100:00100000:2.0:1787739071.813594:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+55:17725:x1874579652522240:12345-192.168.203.15@tcp:101:ls.0 Request processed in 555us (847us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.813612:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23642 00000100:00100000:0.0:1787739071.822624:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739071.823233:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652522496 00000100:00100000:3.0:1787739071.823243:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652522496 00000100:00100000:3.0:1787739071.823658:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23643 00000100:00100000:3.0:1787739071.823669:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304000780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:17725:x1874579652522496:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:3.0:1787739071.823691:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739071.823713:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae2c7 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x17c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0953de expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739071.824056:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17c:0x0]: returning size 4096 00010000:00010000:3.0:1787739071.824226:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae2c7 lrc: 1/0,0 mode: --/CR res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953de expref: 32 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.824243:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17c:0x0].0x0 (ffffa0730608bc00) 00010000:00010000:3.0:1787739071.824246:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739071.824251:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae2c7 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953de expref: 32 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.824269:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab200, rc 0) 00000100:00100000:3.0:1787739071.824388:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304000780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+31:17725:x1874579652522496:12345-192.168.203.15@tcp:101:ls.0 Request processed in 721us (1785us total) trans 4294968438 rc 301/301 00000100:00100000:3.0:1787739071.824396:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23643 00000100:00100000:0.0:1787739071.835288:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.836845:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652522624 00000100:00100000:1.0:1787739071.836858:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652522624 00000100:00100000:1.0:1787739071.836927:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23644 00000100:00100000:1.0:1787739071.836937:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304000000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+31:17725:x1874579652522624:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.836969:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.836991:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd000/0x7ee4453e4a1ae2d5 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0953e5 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.837061:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ae2dc lrc: 3/1,0 mode: --/PR res: [0x240000402:0x17c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.837074:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ae2dc lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.837085:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ae2dc lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739071.837177:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17c:0x0]: returning size 4096 00000004:00200000:1.0:1787739071.837183:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:1.0:1787739071.837203:0:13793:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ae2dc lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x17c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.837230:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd000/0x7ee4453e4a1ae2d5 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953e5 expref: 33 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.837254:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ae2dc lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x17c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953e5 expref: 32 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.837263:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17c:0x0].0x0 (ffffa07305eae500) 00010000:00010000:1.0:1787739071.837266:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739071.837270:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07441a1d000, rc 0) 00000100:00100000:1.0:1787739071.837337:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304000000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+32:17725:x1874579652522624:12345-192.168.203.15@tcp:101:ls.0 Request processed in 401us (2058us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.837344:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23644 00000100:00100000:2.0:1787739071.846573:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.850495:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652522752 00000100:00100000:1.0:1787739071.850509:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652522752 00000100:00100000:3.0:1787739071.850626:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23645 00000100:00100000:3.0:1787739071.850639:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+32:17725:x1874579652522752:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:3.0:1787739071.850661:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739071.850689:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae2e3 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0953ec expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.850753:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae2ea lrc: 3/1,0 mode: --/PR res: [0x240000400:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.850768:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae2ea lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.850779:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae2ea lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739071.850825:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xc5:0x0]: returning size 4096 00000004:00200000:3.0:1787739071.850830:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739071.850867:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae2e3 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953ec expref: 34 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.850886:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae2ea lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953ec expref: 33 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.850895:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc5:0x0].0x0 (ffffa0730608bc00) 00010000:00010000:3.0:1787739071.850898:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739071.850901:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa200, rc 0) 00000100:00100000:3.0:1787739071.851889:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+33:17725:x1874579652522752:12345-192.168.203.15@tcp:101:ls.0 Request processed in 1249us (5322us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739071.851899:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23645 00000100:00100000:3.0:1787739071.859329:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.859469:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652522880 00000100:00100000:1.0:1787739071.859479:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652522880 00000100:00100000:1.0:1787739071.859543:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23646 00000100:00100000:1.0:1787739071.859553:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060043c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+55:17725:x1874579652522880:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.859574:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.859598:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1ae2f1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0953f3 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.859658:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc5800/0x7ee4453e4a1ae2f8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.859671:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc5800/0x7ee4453e4a1ae2f8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.859682:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc5800/0x7ee4453e4a1ae2f8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739071.859742:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xc5:0x0]: returning size 4096 00000004:00200000:1.0:1787739071.859747:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739071.859783:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1ae2f1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xc5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953f3 expref: 57 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.859801:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc5800/0x7ee4453e4a1ae2f8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953f3 expref: 56 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.859810:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc5:0x0].0x0 (ffffa07305eae100) 00010000:00010000:1.0:1787739071.859812:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739071.859815:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07404cc5800, rc 0) 00000100:00100000:1.0:1787739071.859876:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060043c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+56:17725:x1874579652522880:12345-192.168.203.15@tcp:101:ls.0 Request processed in 324us (557us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.859883:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23646 00000100:00100000:2.0:1787739071.878108:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739071.878215:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652523008 00000100:00100000:0.0:1787739071.878224:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652523008 00000100:00100000:0.0:1787739071.878282:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5048 00000100:00100000:0.0:1787739071.878292:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+33:17725:x1874579652523008:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:2.0:1787739071.887523:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+33:17725:x1874579652523008:12345-192.168.203.15@tcp:37:ls.0 Request processed in 9222us (9416us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.887536:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5048 00000100:00100000:3.0:1787739071.896386:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.898508:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652523136 00000100:00100000:1.0:1787739071.898533:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652523136 00000100:00100000:2.0:1787739071.898858:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5049 00000100:00100000:2.0:1787739071.898870:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074332a8050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+56:17725:x1874579652523136:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:3.0:1787739071.904674:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074332a8050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+56:17725:x1874579652523136:12345-192.168.203.15@tcp:37:ls.0 Request processed in 5795us (8287us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739071.904687:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5049 00000100:00100000:3.0:1787739071.910515:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.910641:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652523392 00000100:00100000:2.0:1787739071.910650:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652523392 00000100:00100000:2.0:1787739071.910700:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5050 00000100:00100000:2.0:1787739071.910708:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060070c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+33:17725:x1874579652523392:12345-192.168.203.15@tcp:35:ls.0 00000100:00100000:2.0:1787739071.911153:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060070c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+34:17725:x1874579652523392:12345-192.168.203.15@tcp:35:ls.0 Request processed in 444us (645us total) trans 4294968439 rc 0/0 00000100:00100000:2.0:1787739071.911162:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5050 00000100:00100000:2.0:1787739071.916687:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739071.917301:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652523648 00000100:00100000:0.0:1787739071.917311:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652523648 00000100:00100000:0.0:1787739071.917509:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23647 00000100:00100000:0.0:1787739071.917521:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+56:17725:x1874579652523648:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:0.0:1787739071.917542:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739071.917562:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ae2ff lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0953fa expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.917647:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ae306 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.917663:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ae306 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.917674:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ae306 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.917689:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ae30d lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.917697:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ae30d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.917708:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ae30d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.918132:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07409d95200/0x7ee4453e4a1ae314 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x39:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.918142:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07409d95200/0x7ee4453e4a1ae314 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.918149:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07409d95200/0x7ee4453e4a1ae314 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.918162:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ae306 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.918168:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ae306 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.918174:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ae306 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.918180:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ae306 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.918186:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609ac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739071.918190:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ae306 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.918197:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ae306 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.918208:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5400) 00010000:00010000:0.0:1787739071.918212:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739071.918217:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ae306 lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.918224:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ae306 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.918240:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ae30d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.918245:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ae30d lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.918251:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ae30d lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.918256:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739071.918259:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ae30d lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.918264:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ae30d lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.918271:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xce9d12a2 (ffffa07305ca2700) 00010000:00010000:0.0:1787739071.918274:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739071.918276:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ae30d lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.918281:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ae30d lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.918315:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ae2ff lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953fa expref: 58 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.918347:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07409d95200/0x7ee4453e4a1ae314 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953fa expref: 57 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739071.918355:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x39:0x0].0x0 (ffffa07305ca2100) 00010000:00010000:0.0:1787739071.918357:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739071.918360:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07409d95200, rc 0) 00000100:00100000:0.0:1787739071.918717:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+57:17725:x1874579652523648:12345-192.168.203.15@tcp:101:ls.0 Request processed in 1196us (2039us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739071.918736:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23647 00000100:00100000:2.0:1787739071.923073:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739071.923349:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652523904 00000100:00100000:3.0:1787739071.923359:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652523904 00000100:00100000:3.0:1787739071.923443:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23648 00000100:00100000:3.0:1787739071.923453:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+34:17725:x1874579652523904:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:3.0:1787739071.923478:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739071.923502:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae31b lrc: 2/0,0 mode: --/CR res: [0x240000402:0x39:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095401 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739071.923720:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x39:0x0]: returning size 4096 00010000:00010000:3.0:1787739071.924009:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae31b lrc: 1/0,0 mode: --/CR res: [0x240000402:0x39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095401 expref: 36 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.924167:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x39:0x0].0x0 (ffffa0730608bd00) 00010000:00010000:3.0:1787739071.924171:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739071.924177:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae31b lrc: 0/0,0 mode: --/CR res: [0x240000402:0x39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095401 expref: 36 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739071.924193:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739071.924292:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:17725:x1874579652523904:12345-192.168.203.15@tcp:101:ls.0 Request processed in 840us (1227us total) trans 4294968440 rc 301/301 00000100:00100000:3.0:1787739071.924301:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23648 00000100:00100000:3.0:1787739071.930344:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.930443:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652524032 00000100:00100000:1.0:1787739071.930452:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652524032 00000100:00100000:1.0:1787739071.930511:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23649 00000100:00100000:1.0:1787739071.930521:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306006d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:17725:x1874579652524032:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.930543:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.930566:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07404cc4600/0x7ee4453e4a1ae329 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095408 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.930641:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07404cc5400/0x7ee4453e4a1ae330 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x39:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.930654:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07404cc5400/0x7ee4453e4a1ae330 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x39:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.930665:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07404cc5400/0x7ee4453e4a1ae330 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x39:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739071.930777:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x39:0x0]: returning size 4096 00000004:00200000:1.0:1787739071.930782:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:1.0:1787739071.930814:0:6525:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa07404cc5400/0x7ee4453e4a1ae330 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x39:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.930842:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07404cc4600/0x7ee4453e4a1ae329 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095408 expref: 37 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.930867:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07404cc5400/0x7ee4453e4a1ae330 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x39:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095408 expref: 36 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.930876:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x39:0x0].0x0 (ffffa07305eaec00) 00010000:00010000:1.0:1787739071.930879:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739071.930882:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07404cc5400, rc 0) 00000100:00100000:1.0:1787739071.930964:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306006d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+36:17725:x1874579652524032:12345-192.168.203.15@tcp:101:ls.0 Request processed in 434us (619us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.930972:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23649 00000100:00100000:2.0:1787739071.936512:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.936653:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652524160 00000100:00100000:1.0:1787739071.936662:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652524160 00000100:00100000:1.0:1787739071.936721:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23650 00000100:00100000:1.0:1787739071.936731:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+36:17725:x1874579652524160:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.936762:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.936783:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07404cc5e00/0x7ee4453e4a1ae337 lrc: 2/0,0 mode: --/CR res: [0x240000400:0x22:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09540f expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.936851:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07404cc4200/0x7ee4453e4a1ae33e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.936869:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07404cc4200/0x7ee4453e4a1ae33e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.936880:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07404cc4200/0x7ee4453e4a1ae33e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739071.936923:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x22:0x0]: returning size 4096 00000004:00200000:1.0:1787739071.936928:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739071.936970:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07404cc5e00/0x7ee4453e4a1ae337 lrc: 0/0,0 mode: --/CR res: [0x240000400:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09540f expref: 38 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.936990:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07404cc4200/0x7ee4453e4a1ae33e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09540f expref: 37 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.936998:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x22:0x0].0x0 (ffffa07305eae300) 00010000:00010000:1.0:1787739071.937001:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739071.937004:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07404cc4200, rc 0) 00000100:00100000:1.0:1787739071.937381:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+37:17725:x1874579652524160:12345-192.168.203.15@tcp:101:ls.0 Request processed in 651us (882us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.937390:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23650 00000100:00100000:2.0:1787739071.938850:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3650:1874579665749376:0@lo:400:kworker.0 00000100:00100000:2.0:1787739071.938913:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739071.938979:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665749888:0@lo:400:kworker.0 00000100:00100000:2.0:1787739071.939007:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739071.939040:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665750400:0@lo:400:kworker.0 00000100:00100000:2.0:1787739071.939062:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739071.939087:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:2.0:1787739071.939093:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3650:1874579665749504:0@lo:400:kworker.0 00000100:00100000:2.0:1787739071.939114:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739071.939129:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665750016:0@lo:400:kworker.0 00000100:00100000:2.0:1787739071.939150:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739071.939170:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665750528:0@lo:400:kworker.0 00000100:00100000:2.0:1787739071.939191:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739071.939225:0:14663:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665749888 00000100:00100000:2.0:1787739071.939233:0:14663:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665749888 00000100:00100000:2.0:1787739071.939279:0:14663:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665750400 00000100:00100000:2.0:1787739071.939282:0:14663:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665750400 00000100:00100000:2.0:1787739071.939300:0:14663:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665750016 00000100:00100000:2.0:1787739071.939302:0:14663:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665750016 00000100:00100000:2.0:1787739071.939322:0:14663:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665750528 00000100:00100000:2.0:1787739071.939339:0:14663:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665750528 00000100:00100000:2.0:1787739071.939359:0:14663:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 357 00000100:00100000:2.0:1787739071.939366:0:14663:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f7840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3650:x1874579665749888:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739071.939438:0:14663:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f7840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3650:x1874579665749888:12345-0@lo:400:kworker.0 Request processed in 72us (430us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.939446:0:14663:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 357 00000100:00100000:2.0:1787739071.939462:0:14663:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 358 00000100:00100000:2.0:1787739071.939468:0:14663:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f52c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3650:x1874579665750400:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739071.939497:0:14663:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f52c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3650:x1874579665750400:12345-0@lo:400:kworker.0 Request processed in 30us (434us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.939503:0:14663:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 358 00000100:00100000:2.0:1787739071.939516:0:14663:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 359 00000100:00100000:2.0:1787739071.939522:0:14663:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f6d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3650:x1874579665750016:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739071.939550:0:14663:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f6d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3650:x1874579665750016:12345-0@lo:400:kworker.0 Request processed in 28us (398us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.939555:0:14663:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 359 00000100:00100000:2.0:1787739071.939566:0:14663:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 360 00000100:00100000:2.0:1787739071.939572:0:14663:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f5a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3650:x1874579665750528:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739071.939597:0:14663:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f5a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3650:x1874579665750528:12345-0@lo:400:kworker.0 Request processed in 26us (405us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.939602:0:14663:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 360 00000100:00080000:2.0:1787739071.939625:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1787739071.939630:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787739071.939668:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f6580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665750016:0@lo:400:kworker.0 00000100:00100000:2.0:1787739071.939680:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665750528:0@lo:400:kworker.0 00000100:00100000:2.0:1787739071.939687:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665749888:0@lo:400:kworker.0 00000100:00100000:2.0:1787739071.939695:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f43c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665750400:0@lo:400:kworker.0 00000100:00100000:0.0:1787739071.940090:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665749376 00000100:00100000:0.0:1787739071.940095:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665749376 00000100:00100000:0.0:1787739071.940133:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665749504 00000100:00100000:0.0:1787739071.940137:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665749504 00000100:00100000:0.0:1787739071.940161:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 668 00000100:00100000:0.0:1787739071.940167:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f5e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1874579665749376:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787739071.940214:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f5e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1874579665749376:12345-0@lo:400:kworker.0 Request processed in 47us (1301us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739071.940220:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 668 00000100:00100000:0.0:1787739071.940249:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432fc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3650:1874579665749376:0@lo:400:kworker.0 00000100:00100000:0.0:1787739071.940269:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 669 00000100:00100000:0.0:1787739071.940276:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3650:x1874579665749504:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787739071.940311:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3650:x1874579665749504:12345-0@lo:400:kworker.0 Request processed in 35us (1195us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739071.940316:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 669 00000100:00100000:0.0:1787739071.940359:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3650:1874579665749504:0@lo:400:kworker.0 00000100:00100000:3.0:1787739071.940752:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2:3653:1874579665749248:0@lo:400:kworker.0 00000100:00100000:3.0:1787739071.940803:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739071.940843:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3653:1874579665749760:0@lo:400:kworker.0 00000100:00100000:3.0:1787739071.940868:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739071.940886:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3653:1874579665750272:0@lo:400:kworker.0 00000100:00100000:3.0:1787739071.940906:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739071.940922:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:3.0:1787739071.940928:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3653:1874579665749632:0@lo:400:kworker.0 00000100:00100000:3.0:1787739071.940945:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739071.940973:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3653:1874579665750144:0@lo:400:kworker.0 00000100:00100000:3.0:1787739071.940994:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739071.941008:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073040012c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665750656:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787739071.941215:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739071.941679:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665749760 00000100:00100000:1.0:1787739071.941687:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665749760 00000100:00100000:1.0:1787739071.941727:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665750272 00000100:00100000:1.0:1787739071.941730:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665750272 00000100:00100000:1.0:1787739071.941756:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665749632 00000100:00100000:1.0:1787739071.941759:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665749632 00000100:00100000:1.0:1787739071.941774:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665750144 00000100:00100000:1.0:1787739071.941776:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665750144 00000100:00100000:1.0:1787739071.941967:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 670 00000100:00100000:1.0:1787739071.941980:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306005e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3653:x1874579665749760:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787739071.942047:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306005e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3653:x1874579665749760:12345-0@lo:400:kworker.0 Request processed in 71us (1178us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.942054:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 670 00000100:00100000:1.0:1787739071.942070:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 671 00000100:00100000:1.0:1787739071.942077:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306005680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874579665750272:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787739071.942105:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306005680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3653:x1874579665750272:12345-0@lo:400:kworker.0 Request processed in 29us (1197us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.942110:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 671 00000100:00100000:1.0:1787739071.942121:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 672 00000100:00100000:1.0:1787739071.942127:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874579665749632:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787739071.942153:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3653:x1874579665749632:12345-0@lo:400:kworker.0 Request processed in 27us (1206us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.942158:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 672 00000100:00100000:1.0:1787739071.942171:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 673 00000100:00100000:1.0:1787739071.942176:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306006940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3653:x1874579665750144:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787739071.942198:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306006940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3653:x1874579665750144:12345-0@lo:400:kworker.0 Request processed in 24us (1204us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.942203:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 673 00000100:00080000:1.0:1787739071.942218:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1787739071.942222:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787739071.942382:0:6516:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665749248 00000100:00100000:0.0:1787739071.942387:0:6516:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665749248 00000100:00100000:0.0:1787739071.942449:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 452 00000100:00100000:0.0:1787739071.942456:0:6516:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0740ea11450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2+24:3653:x1874579665749248:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787739071.942490:0:6516:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0740ea11450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2+24:3653:x1874579665749248:12345-0@lo:400:kworker.0 Request processed in 35us (1697us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739071.942496:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 452 00000100:00100000:0.0:1787739071.942527:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665750656 00000100:00100000:0.0:1787739071.942531:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665750656 00000100:00100000:0.0:1787739071.942557:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 364 00000100:00100000:0.0:1787739071.942564:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874579665750656:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787739071.942583:0:8441:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 25431488 pending 0 free 3923496960 avail 3691974656 00000100:00100000:0.0:1787739071.942628:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3653:x1874579665750656:12345-0@lo:13:osp-pre-1-0.0 Request processed in 65us (1412us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739071.942634:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 364 00000100:00100000:3.0:1787739071.944233:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306007c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665750784:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1787739071.944284:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739071.944346:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432e580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3653:1874579665749632:0@lo:400:kworker.0 00000100:00100000:3.0:1787739071.944359:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f4b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3653:1874579665750144:0@lo:400:kworker.0 00000100:00100000:3.0:1787739071.944377:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073040012c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665750656:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787739071.944388:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2:3653:1874579665749248:0@lo:400:kworker.0 00000100:00100000:3.0:1787739071.944399:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3653:1874579665749760:0@lo:400:kworker.0 00000100:00100000:3.0:1787739071.944408:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f4780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3653:1874579665750272:0@lo:400:kworker.0 00000100:00100000:2.0:1787739071.944511:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665750784 00000100:00100000:2.0:1787739071.944516:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665750784 00000100:00100000:2.0:1787739071.944551:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 365 00000100:00100000:2.0:1787739071.944557:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874579665750784:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1787739071.944572:0:8440:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 25431488 pending 0 free 3923496960 avail 3691974656 00000100:00100000:3.0:1787739071.944709:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306007c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665750784:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1787739071.944752:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874579665750784:12345-0@lo:13:osp-pre-1-1.0 Request processed in 195us (467us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.944759:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 365 00000100:00100000:0.0:1787739071.953814:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739071.953891:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652524288 00000100:00100000:1.0:1787739071.953901:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652524288 00000100:00100000:1.0:1787739071.953970:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23651 00000100:00100000:1.0:1787739071.953980:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304003c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+57:17725:x1874579652524288:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:1.0:1787739071.953999:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739071.954020:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc5e00/0x7ee4453e4a1ae345 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095416 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.954080:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4600/0x7ee4453e4a1ae34c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.954094:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4600/0x7ee4453e4a1ae34c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.954105:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4600/0x7ee4453e4a1ae34c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739071.954154:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x22:0x0]: returning size 4096 00000004:00200000:1.0:1787739071.954159:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:1.0:1787739071.954196:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc5e00/0x7ee4453e4a1ae345 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095416 expref: 59 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.954214:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4600/0x7ee4453e4a1ae34c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095416 expref: 58 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739071.954223:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffffa07305eaed00) 00010000:00010000:1.0:1787739071.954226:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739071.954229:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07404cc4600, rc 0) 00000100:00100000:1.0:1787739071.954338:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304003c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+58:17725:x1874579652524288:12345-192.168.203.15@tcp:101:ls.0 Request processed in 360us (532us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739071.954347:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23651 00000100:00100000:3.0:1787739071.962194:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.962293:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652524416 00000100:00100000:2.0:1787739071.962303:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652524416 00000100:00100000:2.0:1787739071.962357:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5051 00000100:00100000:2.0:1787739071.962368:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306007c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+37:17725:x1874579652524416:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:2.0:1787739071.972343:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306007c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+37:17725:x1874579652524416:12345-192.168.203.15@tcp:37:ls.0 Request processed in 9967us (10148us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.972357:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5051 00000100:00100000:1.0:1787739071.980157:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.980253:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652524544 00000100:00100000:2.0:1787739071.980264:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652524544 00000100:00100000:2.0:1787739071.980320:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5052 00000100:00100000:2.0:1787739071.980331:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+58:17725:x1874579652524544:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:2.0:1787739071.984309:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+58:17725:x1874579652524544:12345-192.168.203.15@tcp:37:ls.0 Request processed in 3969us (4150us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739071.984322:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5052 00000100:00100000:3.0:1787739071.989248:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739071.989402:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652524800 00000100:00100000:2.0:1787739071.989445:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652524800 00000100:00100000:2.0:1787739071.989494:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5053 00000100:00100000:2.0:1787739071.989504:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306006580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+37:17725:x1874579652524800:12345-192.168.203.15@tcp:35:ls.0 00000040:00080000:2.0:1787739071.989890:0:14475:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].373, 344 off563408 00000040:00080000:2.0:1787739071.989930:0:14475:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].373, 424 off614824 00000004:00080000:2.0:1787739071.989934:0:14475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07435539a80 with version 712 00000004:00080000:3.0:1787739071.990187:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 712 rpc_version 712 00000020:00100000:3.0:1787739071.990216:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739071.990225:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306004780 x1874579665751040/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1304/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739071.990247:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306004780 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665751040:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739071.990298:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739071.990374:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410cf69c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739071.991555:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665751040 00000100:00100000:2.0:1787739071.991565:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665751040 00000100:00100000:2.0:1787739071.991675:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3505 00000100:00100000:2.0:1787739071.991684:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665751040:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739071.992216:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665751040:12345-0@lo:1000:osp_up0-1.0 Request processed in 532us (1915us total) trans 4294983456 rc 0/0 00000100:00100000:2.0:1787739071.992225:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3505 00000100:00100000:3.0:1787739071.992351:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306004780 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665751040:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739071.993568:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306006580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+38:17725:x1874579652524800:12345-192.168.203.15@tcp:35:ls.0 Request processed in 4059us (4323us total) trans 4294968441 rc 0/0 00000100:00100000:2.0:1787739071.993580:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5053 00000100:00100000:0.0:1787739072.003133:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.003269:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652525056 00000100:00100000:2.0:1787739072.003279:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652525056 00000100:00100000:2.0:1787739072.003354:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23652 00000100:00100000:2.0:1787739072.003363:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304001680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+58:17725:x1874579652525056:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:2.0:1787739072.003384:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739072.003406:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031e00/0x7ee4453e4a1ae353 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09541d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.003518:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031200/0x7ee4453e4a1ae35a lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.003532:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031200/0x7ee4453e4a1ae35a lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.003543:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031200/0x7ee4453e4a1ae35a lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.003556:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ae361 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.003562:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ae361 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.003568:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ae361 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787739072.003754:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1787739072.004088:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4c00/0x7ee4453e4a1ae368 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x4e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.004102:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4c00/0x7ee4453e4a1ae368 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.004111:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4c00/0x7ee4453e4a1ae368 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.004124:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031200/0x7ee4453e4a1ae35a lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.004131:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031200/0x7ee4453e4a1ae35a lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.004138:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031200/0x7ee4453e4a1ae35a lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.004144:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031200/0x7ee4453e4a1ae35a lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.004151:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07400031200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739072.004155:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031200/0x7ee4453e4a1ae35a lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.004166:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031200/0x7ee4453e4a1ae35a lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.004177:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5400) 00010000:00010000:1.0:1787739072.004179:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739072.004184:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031200/0x7ee4453e4a1ae35a lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.004190:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031200/0x7ee4453e4a1ae35a lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.004207:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ae361 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.004213:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ae361 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.004219:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ae361 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.004224:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730633dc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739072.004227:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ae361 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.004237:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ae361 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.004244:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x6ae86992 (ffffa07403eebc00) 00010000:00010000:1.0:1787739072.004246:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739072.004249:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ae361 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.004255:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ae361 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.004293:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031e00/0x7ee4453e4a1ae353 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09541d expref: 60 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.004310:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4c00/0x7ee4453e4a1ae368 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09541d expref: 59 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.004321:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4e:0x0].0x0 (ffffa07305eae600) 00010000:00010000:1.0:1787739072.004324:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739072.004326:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07404cc4c00, rc 0) 00000100:00100000:1.0:1787739072.004388:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304001680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+59:17725:x1874579652525056:12345-192.168.203.15@tcp:101:ls.0 Request processed in 1026us (1264us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.004395:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23652 00000100:00100000:1.0:1787739072.010106:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.010187:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652525312 00000100:00100000:2.0:1787739072.010196:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652525312 00000100:00100000:2.0:1787739072.010266:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23653 00000100:00100000:2.0:1787739072.010276:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+38:17725:x1874579652525312:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:2.0:1787739072.010298:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739072.010318:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae36f lrc: 2/0,0 mode: --/CR res: [0x240000402:0x4e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095424 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739072.010496:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x4e:0x0]: returning size 4096 00010000:00010000:2.0:1787739072.010582:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae36f lrc: 1/0,0 mode: --/CR res: [0x240000402:0x4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095424 expref: 40 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.010598:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4e:0x0].0x0 (ffffa07403eebe00) 00010000:00010000:2.0:1787739072.010825:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739072.010833:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae36f lrc: 0/0,0 mode: --/CR res: [0x240000402:0x4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095424 expref: 40 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.010852:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633d600, rc 0) 00000100:00100000:2.0:1787739072.010949:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+39:17725:x1874579652525312:12345-192.168.203.15@tcp:101:ls.0 Request processed in 674us (853us total) trans 4294968442 rc 301/301 00000100:00100000:2.0:1787739072.010967:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23653 00000100:00080000:1.0:1787739072.011938:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787739072.021541:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.021726:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652525440 00000100:00100000:2.0:1787739072.021736:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652525440 00000100:00100000:2.0:1787739072.021890:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23654 00000100:00100000:2.0:1787739072.021900:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073060061c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+39:17725:x1874579652525440:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:2.0:1787739072.021925:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739072.021946:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae37d lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09542b expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.022021:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ae384 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x4e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.022035:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ae384 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x4e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.022044:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ae384 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x4e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739072.022130:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x4e:0x0]: returning size 4096 00000004:00200000:2.0:1787739072.022136:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:2.0:1787739072.022156:0:13793:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ae384 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x4e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.022182:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae37d lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09542b expref: 41 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.022203:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ae384 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x4e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09542b expref: 40 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.022212:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4e:0x0].0x0 (ffffa07403eeba00) 00010000:00010000:2.0:1787739072.022215:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739072.022218:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa0730633dc00, rc 0) 00000100:00100000:2.0:1787739072.022289:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073060061c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+40:17725:x1874579652525440:12345-192.168.203.15@tcp:101:ls.0 Request processed in 391us (757us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.022297:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23654 00000100:00100000:0.0:1787739072.037862:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.037994:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652525568 00000100:00100000:2.0:1787739072.038002:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652525568 00000100:00100000:2.0:1787739072.038059:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23655 00000100:00100000:2.0:1787739072.038068:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073040012c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+40:17725:x1874579652525568:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:2.0:1787739072.038087:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739072.038108:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae38b lrc: 2/0,0 mode: --/CR res: [0x240000400:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095432 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.038169:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae392 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.038182:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae392 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.038197:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae392 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739072.038243:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0x2d:0x0]: returning size 4096 00000004:00200000:2.0:1787739072.038248:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739072.038282:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d600/0x7ee4453e4a1ae38b lrc: 0/0,0 mode: --/CR res: [0x240000400:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095432 expref: 42 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.038302:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae392 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095432 expref: 41 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.038311:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x2d:0x0].0x0 (ffffa07403eebe00) 00010000:00010000:2.0:1787739072.038314:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739072.038317:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa1000, rc 0) 00000100:00100000:2.0:1787739072.038655:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073040012c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+41:17725:x1874579652525568:12345-192.168.203.15@tcp:101:ls.0 Request processed in 587us (804us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.038663:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23655 00000100:00100000:0.0:1787739072.049199:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.049932:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652525696 00000100:00100000:2.0:1787739072.050100:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652525696 00000100:00100000:2.0:1787739072.050359:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23656 00000100:00100000:2.0:1787739072.050371:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304002d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+59:17725:x1874579652525696:12345-192.168.203.15@tcp:101:ls.0 00010000:00010000:2.0:1787739072.050392:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739072.050436:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1ae399 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095439 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.050574:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1ae3a0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.050593:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1ae3a0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.050607:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1ae3a0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739072.050654:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2d:0x0]: returning size 4096 00000004:00200000:2.0:1787739072.053378:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:2.0:1787739072.053480:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1ae399 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095439 expref: 61 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.053506:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1ae3a0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095439 expref: 60 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.053517:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2d:0x0].0x0 (ffffa07403eebc00) 00010000:00010000:2.0:1787739072.053520:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739072.053523:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa0200, rc 0) 00000100:00100000:2.0:1787739072.053604:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304002d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:17725:x1874579652525696:12345-192.168.203.15@tcp:101:ls.0 Request processed in 3235us (4413us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.053613:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23656 00000100:00100000:0.0:1787739072.064918:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739072.064974:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739072.065013:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739072.065102:0:8885:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652525952 00000100:00100000:1.0:1787739072.065112:0:8885:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652525952 00000100:00100000:2.0:1787739072.065170:0:6518:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652525824 00000100:00100000:2.0:1787739072.065174:0:6518:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652525824 00000100:00100000:1.0:1787739072.065177:0:8885:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652526080 00000100:00100000:1.0:1787739072.065179:0:8885:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652526080 00000100:00100000:2.0:1787739072.065204:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 453 00000100:00100000:1.0:1787739072.065204:0:8885:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 361 00000100:00100000:2.0:1787739072.065213:0:6518:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0740ea10850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:53427813-ebb7-48fc-94b6-ef03330d7ee5+10:2407:x1874579652525824:12345-192.168.203.15@tcp:400:kworker.0 00000100:00100000:1.0:1787739072.065213:0:8885:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304000b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2407:x1874579652525952:12345-192.168.203.15@tcp:400:kworker.0 00000100:00100000:2.0:1787739072.065284:0:6518:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0740ea10850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:53427813-ebb7-48fc-94b6-ef03330d7ee5+10:2407:x1874579652525824:12345-192.168.203.15@tcp:400:kworker.0 Request processed in 73us (310us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.065292:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 453 00000100:00100000:1.0:1787739072.065298:0:8885:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304000b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2407:x1874579652525952:12345-192.168.203.15@tcp:400:kworker.0 Request processed in 88us (391us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.065304:0:8885:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 361 00000100:00100000:1.0:1787739072.065317:0:8885:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 362 00000100:00100000:1.0:1787739072.065323:0:8885:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432cb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2407:x1874579652526080:12345-192.168.203.15@tcp:400:kworker.0 00000100:00100000:1.0:1787739072.065347:0:8885:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432cb40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2407:x1874579652526080:12345-192.168.203.15@tcp:400:kworker.0 Request processed in 25us (335us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.065353:0:8885:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 362 00000100:00080000:0.0:1787739072.065612:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787739072.065617:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787739072.070346:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.071007:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652526208 00000100:00100000:2.0:1787739072.071018:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652526208 00000100:00100000:2.0:1787739072.071077:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5054 00000100:00100000:2.0:1787739072.071087:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074332a9c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+41:17725:x1874579652526208:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:2.0:1787739072.083656:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074332a9c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+41:17725:x1874579652526208:12345-192.168.203.15@tcp:37:ls.0 Request processed in 12561us (13310us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.083670:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5054 00000100:00100000:0.0:1787739072.096853:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.097050:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652526336 00000100:00100000:2.0:1787739072.097060:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652526336 00000100:00100000:2.0:1787739072.097119:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5055 00000100:00100000:2.0:1787739072.097130:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:17725:x1874579652526336:12345-192.168.203.15@tcp:37:ls.0 00000100:00100000:2.0:1787739072.103228:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:17725:x1874579652526336:12345-192.168.203.15@tcp:37:ls.0 Request processed in 6090us (6374us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.103242:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5055 00000100:00100000:1.0:1787739072.120035:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.120454:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652526592 00000100:00100000:2.0:1787739072.120465:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652526592 00000100:00100000:2.0:1787739072.120516:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5056 00000100:00100000:2.0:1787739072.120526:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+41:17725:x1874579652526592:12345-192.168.203.15@tcp:35:ls.0 00000040:00080000:2.0:1787739072.121075:0:6527:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].374, 344 off563752 00000040:00080000:2.0:1787739072.121127:0:6527:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].374, 424 off615248 00000004:00080000:2.0:1787739072.121133:0:6527:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07302e89f00 with version 713 00000004:00080000:3.0:1787739072.121445:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 713 rpc_version 713 00000020:00100000:3.0:1787739072.121476:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739072.121485:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073060070c0 x1874579665751296/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 1560/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739072.121508:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073060070c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665751296:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739072.121571:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739072.122373:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410cf69c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.122541:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665751296 00000100:00100000:0.0:1787739072.122548:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665751296 00000100:00100000:0.0:1787739072.122734:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3506 00000100:00100000:0.0:1787739072.122745:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306007c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665751296:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739072.123034:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306007c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:7467:x1874579665751296:12345-0@lo:1000:osp_up0-1.0 Request processed in 290us (1461us total) trans 4294983457 rc 0/0 00000100:00100000:0.0:1787739072.123042:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3506 00000100:00100000:2.0:1787739072.123151:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073060070c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665751296:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739072.123692:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+42:17725:x1874579652526592:12345-192.168.203.15@tcp:35:ls.0 Request processed in 3165us (3665us total) trans 4294968443 rc 0/0 00000100:00100000:0.0:1787739072.123702:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5056 00000100:00100000:0.0:1787739072.180861:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.182902:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652526720 00000100:00100000:2.0:1787739072.182919:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652526720 00000100:00100000:2.0:1787739072.183013:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23657 00000100:00100000:2.0:1787739072.183025:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:17726:x1874579652526720:12345-192.168.203.15@tcp:101:rm.0 00010000:00010000:2.0:1787739072.183046:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739072.183069:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1ae3a7 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095440 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.183137:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae3ae lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.183152:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae3ae lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.183164:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae3ae lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.183181:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae3b5 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0xac50785c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.183189:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae3b5 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.183196:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae3b5 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.183250:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae3bc lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14c4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.183259:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae3bc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14c4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.183265:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae3bc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14c4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739072.184702:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14c4:0x0]: returning size 4096 00000004:00200000:2.0:1787739072.184711:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:2.0:1787739072.184741:0:13793:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae3bc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14c4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.184759:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae3ae lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.184766:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae3ae lrc: 4/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.184772:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae3ae lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.184779:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae3ae lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.184786:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739072.184790:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae3ae lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.184798:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae3ae lrc: 5/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.184814:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5400) 00010000:00010000:2.0:1787739072.184817:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739072.184821:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae3ae lrc: 3/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.184829:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae3ae lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.184845:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae3b5 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.184857:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae3b5 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.184862:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae3b5 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.184867:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739072.184870:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae3b5 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.184875:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae3b5 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.184883:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xac50785c (ffffa07403eebb00) 00010000:00010000:2.0:1787739072.184885:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739072.184888:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae3b5 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.184893:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae3b5 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.184931:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1ae3a7 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095440 expref: 62 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.184944:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae3bc lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14c4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095440 expref: 61 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.184963:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14c4:0x0].0x0 (ffffa0730608ba00) 00010000:00010000:2.0:1787739072.184966:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1787739072.184969:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa1a00, rc 0) 00000100:00100000:2.0:1787739072.185127:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:17726:x1874579652526720:12345-192.168.203.15@tcp:101:rm.0 Request processed in 2092us (4265us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.185137:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23657 00000100:00100000:1.0:1787739072.200789:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.201837:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652526848 00000100:00100000:2.0:1787739072.201848:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652526848 00000100:00100000:2.0:1787739072.201927:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1364 00000100:00100000:2.0:1787739072.201938:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be61c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:2409:x1874579652526848:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739072.201962:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739072.201989:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae3bc lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14c4:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095440 expref: 61 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.202008:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14c4:0x0].0x0 (ffffa0730608ba00) 00010000:00010000:2.0:1787739072.202013:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739072.202076:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be61c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:2409:x1874579652526848:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 140us (1295us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.202083:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1364 00000100:00100000:1.0:1787739072.212333:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739072.212389:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739072.212531:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652527232 00000100:00100000:0.0:1787739072.212541:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652527232 00000100:00100000:0.0:1787739072.212609:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 3507 00000100:00100000:0.0:1787739072.212620:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:2408:x1874579652527232:12345-192.168.203.15@tcp:63:ll_sa.0 00010000:00010000:0.0:1787739072.212658:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739072.212689:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ae3c3 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09544e expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213054:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ae3ca lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213071:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ae3ca lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213083:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ae3ca lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213101:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ae3d1 lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213114:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ae3d1 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213120:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ae3d1 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213174:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1ae3d8 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14e6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213183:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1ae3d8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14e6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213191:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1ae3d8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14e6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739072.213264:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14e6:0x0]: returning size 4096 00000004:00200000:0.0:1787739072.213269:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:0.0:1787739072.213313:0:6528:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1ae3d8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14e6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213326:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ae3ca lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213333:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ae3ca lrc: 2/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213340:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ae3ca lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213347:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609b800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739072.213351:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ae3ca lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213360:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ae3ca lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213372:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5400) 00010000:00010000:0.0:1787739072.213375:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739072.213380:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ae3ca lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213387:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b800/0x7ee4453e4a1ae3ca lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213401:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ae3d1 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213407:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ae3d1 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213500:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ae3d1 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213507:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0730609be00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739072.213510:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ae3d1 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213516:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ae3d1 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213524:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x3fb0b00e (ffffa07305ca2700) 00010000:00010000:0.0:1787739072.213527:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739072.213530:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ae3d1 lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213540:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609be00/0x7ee4453e4a1ae3d1 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213571:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ac00/0x7ee4453e4a1ae3c3 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09544e expref: 62 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213584:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1ae3d8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14e6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09544e expref: 61 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.213592:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14e6:0x0].0x0 (ffffa07403eeb100) 00010000:00010000:0.0:1787739072.213595:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787739072.213609:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306088c00, rc 0) 00000100:00100000:0.0:1787739072.213689:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:2408:x1874579652527232:12345-192.168.203.15@tcp:63:ll_sa.0 Request processed in 1071us (1298us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.213698:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 3507 00000100:00100000:2.0:1787739072.215469:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652527104 00000100:00100000:2.0:1787739072.215483:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652527104 00000100:00100000:2.0:1787739072.215547:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23658 00000100:00100000:2.0:1787739072.215557:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:17726:x1874579652527104:12345-192.168.203.15@tcp:101:rm.0 00010000:00010000:2.0:1787739072.215576:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739072.215596:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae3df lrc: 2/0,0 mode: --/CR res: [0x200000402:0x14c4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095447 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739072.215769:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14c4:0x0]: returning size 4096 00010000:00010000:2.0:1787739072.215866:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae3df lrc: 1/0,0 mode: --/CR res: [0x200000402:0x14c4:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095447 expref: 63 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.215880:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14c4:0x0].0x0 (ffffa0730608ba00) 00010000:00010000:2.0:1787739072.215882:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739072.215887:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae3df lrc: 0/0,0 mode: --/CR res: [0x200000402:0x14c4:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095447 expref: 63 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.215899:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa1a00, rc 0) 00000100:00100000:2.0:1787739072.215970:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+62:17726:x1874579652527104:12345-192.168.203.15@tcp:101:rm.0 Request processed in 414us (3648us total) trans 4294983458 rc 301/301 00000100:00100000:2.0:1787739072.215977:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23658 00000100:00100000:0.0:1787739072.229656:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.229768:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652527488 00000100:00100000:2.0:1787739072.229779:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652527488 00000100:00100000:2.0:1787739072.229836:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5057 00000100:00100000:2.0:1787739072.229847:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+62:17726:x1874579652527488:12345-192.168.203.15@tcp:35:rm.0 00000100:00100000:2.0:1787739072.230140:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+63:17726:x1874579652527488:12345-192.168.203.15@tcp:35:rm.0 Request processed in 294us (493us total) trans 4294983459 rc 0/0 00000100:00100000:2.0:1787739072.230149:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5057 00000100:00100000:0.0:1787739072.234580:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.234849:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652527744 00000100:00100000:2.0:1787739072.234858:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652527744 00000100:00100000:2.0:1787739072.234912:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23659 00000100:00100000:2.0:1787739072.234921:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+63:17726:x1874579652527744:12345-192.168.203.15@tcp:36:rm.0 00010000:00010000:2.0:1787739072.234978:0:7944:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1787739072.235003:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae146 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14c4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09533d expref: 63 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.235021:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14c4:0x0].0x0 (ffffa0730608ba00) 00010000:00010000:2.0:1787739072.235034:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d000/0x7ee4453e4a1adee5 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095241 expref: 62 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.235043:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5400) 00010000:00010000:2.0:1787739072.235048:0:7944:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739072.235067:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae3ed lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.235078:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae3ed lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.235086:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae3ed lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.235097:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1ae3f4 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xac50785c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.235103:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1ae3f4 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.235108:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1ae3f4 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.235157:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae3fb lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14c4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.235163:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae3fb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14c4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.235168:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae3fb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14c4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.235183:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae402 lrc: 3/0,1 mode: --/EX res: [0x200000400:0x9f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.235192:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ae154 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095344 expref: 61 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.235200:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae402 lrc: 4/0,1 mode: --/EX res: [0x200000400:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.235231:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ae154 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095344 expref: 61 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.235244:0:7944:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ae154 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095344 expref: 61 pid: 6524 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.235258:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f43c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665751424:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739072.236226:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074118e0d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.242616:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739072.242700:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652527872 00000100:00100000:3.0:1787739072.242710:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652527872 00010000:00010000:3.0:1787739072.242767:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ae154 lrc: 5/0,0 mode: PR/PR res: [0x200000400:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095344 expref: 62 pid: 6524 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.242813:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1569 00000100:00100000:3.0:1787739072.242823:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be43c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+62:2409:x1874579652527872:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739072.242839:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739072.242845:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ae154 lrc: 5/0,0 mode: PR/PR res: [0x200000400:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095344 expref: 62 pid: 6524 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.242858:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ae154 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095344 expref: 62 pid: 6524 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.242883:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x9f:0x0].0x0 (ffffa07305ca2a00) 00010000:00010000:3.0:1787739072.242888:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae402 lrc: 3/0,1 mode: --/EX res: [0x200000400:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.242898:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae402 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.242907:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae402 lrc: 4/0,1 mode: EX/EX res: [0x200000400:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.242928:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739072.242992:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be43c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+62:2409:x1874579652527872:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 170us (393us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739072.243000:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1569 00010000:00010000:2.0:1787739072.243386:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609b600/0x7ee4453e4a1ae154 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095344 expref: 61 pid: 6524 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.243404:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f43c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665751424:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739072.243466:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae402 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.243476:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae402 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.243482:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae402 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.243521:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1200/0x7ee4453e4a1ae409 lrc: 3/0,1 mode: --/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.243529:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1200/0x7ee4453e4a1ae409 lrc: 3/0,1 mode: --/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.243539:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1200/0x7ee4453e4a1ae409 lrc: 3/0,1 mode: --/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.243557:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f43c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665751552:0@lo:101:mdt00_003.0 00000100:00100000:2.0:1787739072.243597:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.243627:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074118e0d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739072.244286:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665751552 00000100:00100000:3.0:1787739072.244294:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665751552 00000100:00100000:3.0:1787739072.244350:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3508 00000100:00100000:3.0:1787739072.244358:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:7944:x1874579665751552:12345-0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739072.244378:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739072.244395:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae410 lrc: 2/0,0 mode: --/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae409 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.244447:0:6529:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ae162 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09534b expref: 41 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.244457:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae410 lrc: 4/0,0 mode: --/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae409 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.244483:0:6529:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ae162 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09534b expref: 41 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.244496:0:6529:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ae162 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09534b expref: 41 pid: 7944 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.244513:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306006580 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665751680:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739072.244586:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410cf69c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739072.249725:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306006580 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665751680:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739072.249836:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae410 lrc: 3/0,0 mode: --/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae409 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.250007:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x9f:0x0].0x0 (ffffa07403eeb900) 00010000:00010000:3.0:1787739072.250011:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739072.250015:0:6529:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae410 lrc: 3/0,0 mode: --/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae409 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.250026:0:6529:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae410 lrc: 3/0,0 mode: --/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae409 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.250034:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739072.250145:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+11:7944:x1874579665751552:12345-0@lo:101:mdt00_003.0 Request processed in 5787us (6547us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739072.250155:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3508 00000100:00100000:2.0:1787739072.251229:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f43c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665751552:0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739072.251253:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1200/0x7ee4453e4a1ae409 lrc: 4/0,1 mode: --/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae410 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.251265:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1200/0x7ee4453e4a1ae409 lrc: 4/0,1 mode: --/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae410 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.254638:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739072.254963:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652528000 00000100:00100000:3.0:1787739072.254973:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652528000 00010000:00010000:3.0:1787739072.255011:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ae162 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09534b expref: 42 pid: 7944 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.255052:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1570 00000100:00100000:3.0:1787739072.255063:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+42:2408:x1874579652528000:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739072.255080:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739072.255086:0:6510:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ae162 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09534b expref: 42 pid: 7944 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.255099:0:6510:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ae162 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09534b expref: 42 pid: 7944 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.255119:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633d800/0x7ee4453e4a1ae162 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x9f:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09534b expref: 42 pid: 7944 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.255135:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x9f:0x0].0x0 (ffffa07403eeb900) 00010000:00010000:3.0:1787739072.255141:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae410 lrc: 2/0,0 mode: --/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae409 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.255152:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae410 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae409 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.255161:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae410 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae409 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.255193:0:6510:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae410 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae409 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.255208:0:6510:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306006580 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6510:1874579665751808:0@lo:105: 00000100:00100000:3.0:1787739072.255255:0:6510:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739072.255311:0:6510:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410cf69c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.255396:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665751808 00000100:00100000:0.0:1787739072.255401:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665751808 00000100:00100000:0.0:1787739072.255474:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1200 00000100:00100000:0.0:1787739072.255482:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6510:x1874579665751808:12345-0@lo:105: 00010000:00010000:0.0:1787739072.255494:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1200/0x7ee4453e4a1ae409 lrc: 5/0,1 mode: --/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae410 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.255503:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1200/0x7ee4453e4a1ae409 lrc: 5/0,1 mode: --/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae410 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.255516:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1200/0x7ee4453e4a1ae409 lrc: 5/0,1 mode: --/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae410 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.255523:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1200/0x7ee4453e4a1ae409 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae410 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.255626:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1200/0x7ee4453e4a1ae409 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae410 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.255636:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1200/0x7ee4453e4a1ae409 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ae410 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.255701:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07306aa1200) 00000100:00100000:0.0:1787739072.255741:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:6510:x1874579665751808:12345-0@lo:105: Request processed in 260us (484us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.255748:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1200 00000100:00100000:1.0:1787739072.256028:0:6510:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306006580 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001_UUID:6510:1874579665751808:0@lo:105: 00010000:00010000:1.0:1787739072.256044:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739072.256080:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+41:2408:x1874579652528000:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 1019us (1451us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.256087:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1570 00010000:00010000:2.0:1787739072.258226:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1200/0x7ee4453e4a1ae409 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae410 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.258244:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1200/0x7ee4453e4a1ae409 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae410 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.259317:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f43c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665751936:0@lo:403:mdt00_003.0 00000100:00100000:2.0:1787739072.259840:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.259926:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074118e0d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.260002:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665751936 00000100:00100000:0.0:1787739072.260009:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665751936 00000100:00100000:0.0:1787739072.260085:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3509 00000100:00100000:0.0:1787739072.260094:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+11:7944:x1874579665751936:12345-0@lo:403:mdt00_003.0 00000100:00100000:0.0:1787739072.262276:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+11:7944:x1874579665751936:12345-0@lo:403:mdt00_003.0 Request processed in 2175us (2430us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.262289:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3509 00000100:00100000:2.0:1787739072.264944:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f43c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665751936:0@lo:403:mdt00_003.0 00000040:00080000:2.0:1787739072.266255:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].392, 952 off594848 00000040:00080000:2.0:1787739072.267073:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].391, 1032 off644192 00000004:00080000:2.0:1787739072.267082:0:7944:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa074118e0d80 with version 745 00000004:00080000:1.0:1787739072.267165:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 745 rpc_version 745 00000020:00100000:1.0:1787739072.267201:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739072.267211:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306006580 x1874579665752192/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2248/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739072.267234:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306006580 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665752192:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739072.267305:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739072.267338:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46cd80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739072.267389:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665752192 00000100:00100000:3.0:1787739072.267397:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665752192 00000100:00100000:3.0:1787739072.267679:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3510 00000100:00100000:3.0:1787739072.267689:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c383c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+11:7740:x1874579665752192:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739072.267912:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c383c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+12:7740:x1874579665752192:12345-0@lo:1000:osp_up1-0.0 Request processed in 224us (606us total) trans 4294968444 rc 0/0 00000100:00100000:3.0:1787739072.267920:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3510 00000100:00100000:1.0:1787739072.268008:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306006580 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665752192:0@lo:1000:osp_up1-0.0 00000004:00200000:2.0:1787739072.268115:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14c4:0x0]: returning size 4096 00010000:00010000:2.0:1787739072.268135:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae402 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268145:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae402 lrc: 2/0,0 mode: EX/EX res: [0x200000400:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268153:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae402 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268159:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739072.268163:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae402 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268171:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae402 lrc: 4/0,0 mode: EX/EX res: [0x200000400:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268185:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x9f:0x0].0x0 (ffffa07305ca2a00) 00010000:00010000:2.0:1787739072.268187:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739072.268191:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae402 lrc: 2/0,0 mode: --/EX res: [0x200000400:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268198:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae402 lrc: 0/0,0 mode: --/EX res: [0x200000400:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268218:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1200/0x7ee4453e4a1ae409 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae410 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268227:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1200/0x7ee4453e4a1ae409 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae410 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268239:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1200/0x7ee4453e4a1ae409 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae410 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268260:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739072.268269:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x9f:0x0] 00010000:00010000:2.0:1787739072.268278:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae3fb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14c4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268283:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae3fb lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x14c4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268289:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae3fb lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14c4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268293:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739072.268296:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae3fb lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14c4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268302:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae3fb lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x14c4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268310:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14c4:0x0].0x0 (ffffa074419d5100) 00010000:00010000:2.0:1787739072.268312:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739072.268315:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae3fb lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14c4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268321:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae3fb lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14c4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268349:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae3ed lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268356:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae3ed lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268361:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae3ed lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268367:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae3ed lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268372:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1a00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739072.268375:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae3ed lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268380:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae3ed lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268387:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07403eebb00) 00010000:00010000:2.0:1787739072.268389:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739072.268392:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae3ed lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268397:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae3ed lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739072.268409:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000926552a6 reply state 00000000e08fffe8 transno 4294983460 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1ae3f4 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268440:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1ae3f4 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.268447:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xac50785c (ffffa074419d5000) 00010000:00010000:2.0:1787739072.268450:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739072.268452:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787739072.268507:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+62:17726:x1874579652527744:12345-192.168.203.15@tcp:36:rm.0 Request processed in 33585us (33941us total) trans 4294983460 rc 0/0 00000100:00100000:2.0:1787739072.268514:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23659 00010000:00010000:1.0:1787739072.268903:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739072.268909:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1200/0x7ee4453e4a1ae409 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae410 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.268918:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739072.268922:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1200/0x7ee4453e4a1ae409 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae410 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739072.268930:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1200/0x7ee4453e4a1ae409 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae410 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.268968:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1200/0x7ee4453e4a1ae409 lrc: 2/0,0 mode: --/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae410 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.268975:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739072.269004:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1200/0x7ee4453e4a1ae409 lrc: 1/0,0 mode: --/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae410 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.269012:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1200/0x7ee4453e4a1ae409 lrc: 0/0,0 mode: --/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae410 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.269025:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739072.269027:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739072.269028:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739072.269035:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787739072.269046:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c380780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665752320:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739072.269084:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.269524:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665752320 00000100:00100000:2.0:1787739072.269541:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665752320 00000100:00100000:2.0:1787739072.269589:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1365 00000100:00100000:2.0:1787739072.269605:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c381a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+12:3651:x1874579665752320:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739072.269618:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739072.269637:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae410 lrc: 0/0,0 mode: --/EX res: [0x240000401:0x9f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae409 expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.269650:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x9f:0x0].0x0 (ffffa07403eeb900) 00010000:00010000:2.0:1787739072.269656:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739072.269700:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c381a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+11:3651:x1874579665752320:12345-0@lo:103:ldlm_bl.0 Request processed in 104us (616us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.269706:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1365 00000100:00100000:1.0:1787739072.269919:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c380780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665752320:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739072.283490:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.283542:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652528256 00000100:00100000:2.0:1787739072.283552:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652528256 00000100:00100000:1.0:1787739072.283593:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.283620:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23660 00000100:00100000:2.0:1787739072.283631:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c380780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+41:17726:x1874579652528256:12345-192.168.203.15@tcp:101:rm.0 00010000:00010000:2.0:1787739072.283654:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1787739072.283677:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652528384 00000100:00100000:0.0:1787739072.283683:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652528384 00010000:00010000:2.0:1787739072.283686:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1400/0x7ee4453e4a1ae417 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x39:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095455 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.283726:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 3511 00000100:00100000:0.0:1787739072.283735:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c3830c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+62:2408:x1874579652528384:12345-192.168.203.15@tcp:63:ll_sa.0 00010000:00010000:0.0:1787739072.283760:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739072.283775:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1ae41e lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09545c expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.283811:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae425 lrc: 3/1,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.283826:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae425 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.283837:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae425 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.283851:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088400/0x7ee4453e4a1ae42c lrc: 3/1,0 mode: --/PR res: [0x200000400:0x5:0x0].0x649afea4 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.283858:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088400/0x7ee4453e4a1ae42c lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.283865:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088400/0x7ee4453e4a1ae42c lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.283911:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae433 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x13a5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.283920:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae433 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x13a5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739072.283923:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x39:0x0]: returning size 4096 00010000:00010000:0.0:1787739072.283931:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae433 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x13a5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739072.283997:0:6528:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x13a5:0x0]: returning size 4096 00000004:00200000:0.0:1787739072.284001:0:6528:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:0.0:1787739072.284020:0:6528:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae433 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x13a5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.284032:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae425 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.284038:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae425 lrc: 2/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.284044:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae425 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.284047:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1400/0x7ee4453e4a1ae417 lrc: 1/0,0 mode: --/CR res: [0x240000402:0x39:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095455 expref: 43 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.284052:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306088a00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739072.284064:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x39:0x0].0x0 (ffffa07305eaec00) 00010000:00010000:2.0:1787739072.284069:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739072.284070:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae425 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.284075:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1400/0x7ee4453e4a1ae417 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x39:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095455 expref: 43 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.284082:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae425 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.284092:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa1400, rc 0) 00010000:00010000:0.0:1787739072.284094:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305ca2700) 00010000:00010000:0.0:1787739072.284098:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739072.284101:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae425 lrc: 2/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.284107:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae425 lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.284118:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088400/0x7ee4453e4a1ae42c lrc: 3/1,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.284127:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088400/0x7ee4453e4a1ae42c lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.284132:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088400/0x7ee4453e4a1ae42c lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.284137:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306088400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739072.284140:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088400/0x7ee4453e4a1ae42c lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.284145:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088400/0x7ee4453e4a1ae42c lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.284151:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x649afea4 (ffffa07305ca2f00) 00010000:00010000:0.0:1787739072.284154:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739072.284157:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088400/0x7ee4453e4a1ae42c lrc: 2/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.284163:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c380780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+42:17726:x1874579652528256:12345-192.168.203.15@tcp:101:rm.0 Request processed in 534us (681us total) trans 4294968445 rc 301/301 00010000:00010000:0.0:1787739072.284165:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088400/0x7ee4453e4a1ae42c lrc: 0/0,0 mode: --/PR res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.284177:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23660 00010000:00010000:0.0:1787739072.284196:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1ae41e lrc: 0/0,0 mode: --/CR res: [0x200000400:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09545c expref: 64 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.284380:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae433 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x13a5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09545c expref: 63 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.284388:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13a5:0x0].0x0 (ffffa0730608b800) 00010000:00010000:0.0:1787739072.284390:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787739072.284393:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306089a00, rc 0) 00000100:00100000:0.0:1787739072.284449:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c3830c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+63:2408:x1874579652528384:12345-192.168.203.15@tcp:63:ll_sa.0 Request processed in 715us (855us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.284456:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 3511 00000100:00100000:1.0:1787739072.293320:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739072.293519:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652528640 00000100:00100000:0.0:1787739072.293529:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652528640 00000100:00100000:0.0:1787739072.293585:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5058 00000100:00100000:0.0:1787739072.293595:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+42:17726:x1874579652528640:12345-192.168.203.15@tcp:35:rm.0 00000100:00100000:0.0:1787739072.293835:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+43:17726:x1874579652528640:12345-192.168.203.15@tcp:35:rm.0 Request processed in 242us (532us total) trans 4294968446 rc 0/0 00000100:00100000:0.0:1787739072.293844:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5058 00000100:00100000:0.0:1787739072.301518:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739072.301550:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739072.301641:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652528768 00000100:00100000:3.0:1787739072.301650:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652528768 00000100:00100000:3.0:1787739072.301714:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1366 00000100:00100000:3.0:1787739072.301735:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+63:2408:x1874579652528768:12345-192.168.203.15@tcp:103:rm.0 00010000:00010000:3.0:1787739072.301752:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739072.301777:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07409d95200/0x7ee4453e4a1ae314 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x39:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953fa expref: 63 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.301797:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x39:0x0].0x0 (ffffa07305ca2100) 00010000:00010000:3.0:1787739072.301808:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739072.301872:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+62:2408:x1874579652528768:12345-192.168.203.15@tcp:103:rm.0 Request processed in 150us (321us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739072.301880:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1366 00000100:00100000:1.0:1787739072.303193:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652529024 00000100:00100000:1.0:1787739072.303203:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652529024 00000100:00100000:1.0:1787739072.303253:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23661 00000100:00100000:1.0:1787739072.303262:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+43:17726:x1874579652529024:12345-192.168.203.15@tcp:36:rm.0 00010000:00010000:1.0:1787739072.303330:0:6523:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739072.303353:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07404cc5400/0x7ee4453e4a1ae330 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x39:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095408 expref: 43 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.303371:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x39:0x0].0x0 (ffffa07305eaec00) 00010000:00010000:1.0:1787739072.303381:0:6523:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739072.303467:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5200/0x7ee4453e4a1ae441 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.303477:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5200/0x7ee4453e4a1ae441 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.303487:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5200/0x7ee4453e4a1ae441 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.303508:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356acb40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665752448:0@lo:101:mdt00_000.0 00000100:00100000:1.0:1787739072.303560:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739072.303589:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46cd80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739072.303707:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665752448 00000100:00100000:2.0:1787739072.303712:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665752448 00000100:00100000:2.0:1787739072.303755:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3512 00000100:00100000:2.0:1787739072.303766:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+11:6523:x1874579665752448:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739072.303780:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739072.303797:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1400/0x7ee4453e4a1ae448 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae441 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.303820:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1400/0x7ee4453e4a1ae448 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae441 expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.303845:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1400/0x7ee4453e4a1ae448 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae441 expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.303852:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5100) 00010000:00010000:2.0:1787739072.303855:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739072.303857:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa1400, rc 0) 00000100:00100000:2.0:1787739072.303903:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:6523:x1874579665752448:12345-0@lo:101:mdt00_000.0 Request processed in 140us (343us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.303910:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3512 00000100:00100000:1.0:1787739072.304885:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356acb40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665752448:0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739072.304906:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5200/0x7ee4453e4a1ae441 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae448 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.304918:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5200/0x7ee4453e4a1ae441 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae448 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739072.304938:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739072.304978:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5e00/0x7ee4453e4a1ae44f lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.304984:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5e00/0x7ee4453e4a1ae44f lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.304992:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5e00/0x7ee4453e4a1ae44f lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.305007:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356acb40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665752576:0@lo:101:mdt00_000.0 00000100:00100000:1.0:1787739072.305044:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739072.305161:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665752576 00000100:00100000:0.0:1787739072.305166:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665752576 00000100:00100000:0.0:1787739072.305212:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3513 00000100:00100000:0.0:1787739072.305219:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:6523:x1874579665752576:12345-0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739072.305230:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739072.305244:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088400/0x7ee4453e4a1ae456 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae44f expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.305262:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088400/0x7ee4453e4a1ae456 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae44f expref: 13 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.305273:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088400/0x7ee4453e4a1ae456 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae44f expref: 13 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.305286:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xce9d12a2 (ffffa07305ca2700) 00010000:00010000:0.0:1787739072.305290:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739072.305293:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306088400, rc 0) 00000100:00100000:0.0:1787739072.305837:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:6523:x1874579665752576:12345-0@lo:101:mdt00_000.0 Request processed in 617us (791us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.305846:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3513 00000100:00100000:1.0:1787739072.305903:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46cd80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.305920:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356acb40 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665752576:0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739072.305932:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5e00/0x7ee4453e4a1ae44f lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae456 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.305942:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5e00/0x7ee4453e4a1ae44f lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae456 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739072.305964:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:0.0:1787739072.306132:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252a1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665752704:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739072.306160:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739072.306183:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074118e0480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739072.306463:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665752704 00000100:00100000:2.0:1787739072.306469:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665752704 00000100:00100000:2.0:1787739072.306523:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3514 00000100:00100000:2.0:1787739072.306531:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+13:6523:x1874579665752704:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739072.306619:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+13:6523:x1874579665752704:12345-0@lo:1000:mdt00_000.0 Request processed in 88us (457us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.306626:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3514 00000100:00100000:0.0:1787739072.306696:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252a1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665752704:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739072.306841:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252a1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665752832:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739072.306868:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739072.306919:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074118e0300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.306961:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665752832 00000100:00100000:1.0:1787739072.306965:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665752832 00000100:00100000:1.0:1787739072.306998:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3515 00000100:00100000:1.0:1787739072.307004:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:6523:x1874579665752832:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739072.307179:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:6523:x1874579665752832:12345-0@lo:1000:mdt00_000.0 Request processed in 174us (309us total) trans 0 rc 1/1 00000100:00100000:1.0:1787739072.307186:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3515 00000100:00100000:0.0:1787739072.307187:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252a1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665752832:0@lo:1000:mdt00_000.0 00010000:00010000:0.0:1787739072.307235:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088a00/0x7ee4453e4a1ae45d lrc: 3/0,1 mode: --/EX res: [0x240000402:0x39:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.307243:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088a00/0x7ee4453e4a1ae45d lrc: 3/0,1 mode: --/EX res: [0x240000402:0x39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.307253:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088a00/0x7ee4453e4a1ae45d lrc: 3/0,1 mode: --/EX res: [0x240000402:0x39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.307264:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252a1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665752960:0@lo:101:mdt00_000.0 00000100:00100000:0.0:1787739072.307288:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739072.307472:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739072.307518:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665752960 00000100:00100000:3.0:1787739072.307523:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665752960 00000100:00100000:3.0:1787739072.307551:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3516 00000100:00100000:3.0:1787739072.307559:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+13:6523:x1874579665752960:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739072.307568:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739072.307583:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae464 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x39:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae45d expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.307618:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae464 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae45d expref: 14 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.307632:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae464 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae45d expref: 14 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.307639:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x39:0x0].0x0 (ffffa0730608bd00) 00010000:00010000:3.0:1787739072.307642:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739072.307645:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab200, rc 0) 00000100:00100000:3.0:1787739072.307690:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:6523:x1874579665752960:12345-0@lo:101:mdt00_000.0 Request processed in 133us (401us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739072.307696:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3516 00000100:00100000:0.0:1787739072.307850:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252a1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665752960:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739072.307862:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088a00/0x7ee4453e4a1ae45d lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae464 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.307872:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088a00/0x7ee4453e4a1ae45d lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae464 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.307894:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088800/0x7ee4453e4a1ae46b lrc: 3/0,1 mode: --/EX res: [0x240000402:0x39:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.307903:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088800/0x7ee4453e4a1ae46b lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x39:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.307910:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088800/0x7ee4453e4a1ae46b lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x39:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.307927:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae472 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.307939:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07404cc4200/0x7ee4453e4a1ae33e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09540f expref: 42 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.307946:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae472 lrc: 4/0,1 mode: --/EX res: [0x240000400:0x22:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.307979:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07404cc4200/0x7ee4453e4a1ae33e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09540f expref: 42 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.307992:0:6523:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07404cc4200/0x7ee4453e4a1ae33e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09540f expref: 42 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.308009:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252a1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665753088:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739072.308122:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739072.312366:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252a1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665753088:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739072.312395:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae472 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x22:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.315075:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739072.317106:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652529152 00000100:00100000:1.0:1787739072.317121:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652529152 00010000:00010000:0.0:1787739072.317256:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07404cc4200/0x7ee4453e4a1ae33e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09540f expref: 43 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.317338:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1571 00000100:00100000:0.0:1787739072.317349:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252bc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+43:2409:x1874579652529152:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739072.317374:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739072.317381:0:6510:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07404cc4200/0x7ee4453e4a1ae33e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09540f expref: 43 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.317394:0:6510:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07404cc4200/0x7ee4453e4a1ae33e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09540f expref: 43 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.317437:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07404cc4200/0x7ee4453e4a1ae33e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x22:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09540f expref: 43 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.317462:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x22:0x0].0x0 (ffffa07305eae300) 00010000:00010000:0.0:1787739072.317467:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae472 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x22:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.317475:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae472 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x22:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.317483:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae472 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0x22:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.317525:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739072.317578:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae472 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.317589:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae472 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.317615:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252bc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+42:2409:x1874579652529152:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 267us (2549us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.317623:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1571 00010000:00010000:3.0:1787739072.317643:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae479 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.317652:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae479 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.317662:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae479 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.317681:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d861c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665753216:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739072.317739:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739072.317788:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410cf66c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.319246:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665753216 00000100:00100000:1.0:1787739072.319257:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665753216 00000100:00100000:1.0:1787739072.319311:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3517 00000100:00100000:1.0:1787739072.319319:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d87c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+14:6523:x1874579665753216:12345-0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739072.319336:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739072.319355:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4800/0x7ee4453e4a1ae480 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae479 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.319381:0:12410:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4600/0x7ee4453e4a1ae34c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095416 expref: 61 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.319390:0:12410:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4800/0x7ee4453e4a1ae480 lrc: 4/0,0 mode: --/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae479 expref: 15 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.319434:0:12410:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4600/0x7ee4453e4a1ae34c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095416 expref: 61 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.319447:0:12410:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4600/0x7ee4453e4a1ae34c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095416 expref: 61 pid: 6525 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.319460:0:12410:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356acb40 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:12410:1874579665753344:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739072.319943:0:12410:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46c0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.334457:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739072.334580:0:12410:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356acb40 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:12410:1874579665753344:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739072.334626:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4800/0x7ee4453e4a1ae480 lrc: 3/0,0 mode: --/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae479 expref: 15 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.334639:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffffa07305eaed00) 00010000:00010000:1.0:1787739072.334642:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739072.334646:0:12410:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4800/0x7ee4453e4a1ae480 lrc: 3/0,0 mode: --/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae479 expref: 15 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.334655:0:12410:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4800/0x7ee4453e4a1ae480 lrc: 3/0,0 mode: --/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae479 expref: 15 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.334667:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07404cc4800, rc 0) 00000100:00100000:1.0:1787739072.334721:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d87c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+15:6523:x1874579665753216:12345-0@lo:101:mdt00_000.0 Request processed in 15402us (16982us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.334729:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3517 00000100:00100000:2.0:1787739072.336469:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d861c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665753216:0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739072.336497:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae479 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae480 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.336510:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae479 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae480 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.338158:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652529280 00000100:00100000:3.0:1787739072.338173:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652529280 00010000:00010000:3.0:1787739072.338220:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4600/0x7ee4453e4a1ae34c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095416 expref: 62 pid: 6525 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.338265:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1572 00000100:00100000:3.0:1787739072.338274:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba5450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+62:2408:x1874579652529280:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739072.338289:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739072.338295:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4600/0x7ee4453e4a1ae34c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095416 expref: 62 pid: 6525 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.338308:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4600/0x7ee4453e4a1ae34c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095416 expref: 62 pid: 6525 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.338330:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4600/0x7ee4453e4a1ae34c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095416 expref: 62 pid: 6525 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.338347:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffffa07305eaed00) 00010000:00010000:3.0:1787739072.338353:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4800/0x7ee4453e4a1ae480 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae479 expref: 14 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.338364:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4800/0x7ee4453e4a1ae480 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae479 expref: 14 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.338372:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4800/0x7ee4453e4a1ae480 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae479 expref: 14 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.338410:0:6509:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4800/0x7ee4453e4a1ae480 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae479 expref: 14 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.338471:0:6509:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d86d00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665753472:0@lo:105: 00000100:00100000:3.0:1787739072.338551:0:6509:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739072.339060:0:6509:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410cf66c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.339324:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665753472 00000100:00100000:1.0:1787739072.339330:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665753472 00000100:00100000:1.0:1787739072.339380:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1201 00000100:00100000:1.0:1787739072.339388:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665753472:12345-0@lo:105: 00010000:00010000:1.0:1787739072.339401:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae479 lrc: 5/0,1 mode: --/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae480 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.339410:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae479 lrc: 5/0,1 mode: --/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae480 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.339691:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae479 lrc: 5/0,1 mode: --/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae480 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.339700:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae479 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae480 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.339708:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae479 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae480 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.339716:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae479 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ae480 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.339749:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069aac00) 00000100:00100000:1.0:1787739072.339848:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665753472:12345-0@lo:105: Request processed in 461us (1296us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.339855:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1201 00010000:00010000:2.0:1787739072.339921:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae479 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae480 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.339931:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae479 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae480 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.340993:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f43c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665753600:0@lo:403:mdt00_000.0 00000100:00100000:2.0:1787739072.341060:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.341098:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.341234:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665753600 00000100:00100000:1.0:1787739072.341241:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665753600 00000100:00100000:1.0:1787739072.341287:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3518 00000100:00100000:1.0:1787739072.341295:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+15:6523:x1874579665753600:12345-0@lo:403:mdt00_000.0 00000100:00100000:1.0:1787739072.345136:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+15:6523:x1874579665753600:12345-0@lo:403:mdt00_000.0 Request processed in 3829us (4063us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.345151:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3518 00000100:00100000:3.0:1787739072.345323:0:6509:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d86d00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665753472:0@lo:105: 00010000:00010000:3.0:1787739072.345344:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739072.345392:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba5450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:2408:x1874579652529280:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 7116us (10978us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739072.345401:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1572 00000100:00100000:0.0:1787739072.346260:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f43c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665753600:0@lo:403:mdt00_000.0 00000040:00080000:0.0:1787739072.348200:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].375, 952 off564704 00000040:00080000:0.0:1787739072.349165:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].375, 1032 off616280 00000004:00080000:0.0:1787739072.349176:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07306887e40 with version 714 00000004:00080000:2.0:1787739072.349274:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 714 rpc_version 714 00000020:00100000:2.0:1787739072.349363:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739072.349374:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074074f4000 x1874579665753856/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739072.349397:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f4000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665753856:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739072.349727:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.349768:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.349836:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665753856 00000100:00100000:1.0:1787739072.349843:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665753856 00000100:00100000:1.0:1787739072.349931:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3519 00000100:00100000:1.0:1787739072.349940:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+15:7467:x1874579665753856:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739072.350319:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+16:7467:x1874579665753856:12345-0@lo:1000:osp_up0-1.0 Request processed in 379us (591us total) trans 4294983461 rc 0/0 00000100:00100000:1.0:1787739072.350327:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3519 00000100:00100000:3.0:1787739072.350442:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f4000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665753856:0@lo:1000:osp_up0-1.0 00000004:00200000:0.0:1787739072.350733:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x39:0x0]: returning size 4096 00010000:00010000:0.0:1787739072.350755:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae472 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.350765:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae472 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.350773:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae472 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.350779:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306088600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739072.350783:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae472 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.350791:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae472 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.350806:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x22:0x0].0x0 (ffffa07305eae300) 00010000:00010000:0.0:1787739072.350809:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739072.350815:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae472 lrc: 2/0,0 mode: --/EX res: [0x240000400:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.350821:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae472 lrc: 0/0,0 mode: --/EX res: [0x240000400:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.350839:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae479 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae480 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.350847:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae479 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae480 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.350854:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae479 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae480 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.350868:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739072.350875:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x22:0x0] 00010000:00010000:0.0:1787739072.350884:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088800/0x7ee4453e4a1ae46b lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x39:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.350890:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088800/0x7ee4453e4a1ae46b lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x39:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.350896:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088800/0x7ee4453e4a1ae46b lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x39:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.350901:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306088800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739072.350904:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088800/0x7ee4453e4a1ae46b lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x39:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.350909:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088800/0x7ee4453e4a1ae46b lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x39:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.350920:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x39:0x0].0x0 (ffffa07305ca2500) 00010000:00010000:0.0:1787739072.350923:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739072.350925:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088800/0x7ee4453e4a1ae46b lrc: 2/0,0 mode: --/EX res: [0x240000402:0x39:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.350931:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088800/0x7ee4453e4a1ae46b lrc: 0/0,0 mode: --/EX res: [0x240000402:0x39:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.350943:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088a00/0x7ee4453e4a1ae45d lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae464 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.350966:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088a00/0x7ee4453e4a1ae45d lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae464 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.350973:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088a00/0x7ee4453e4a1ae45d lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae464 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.350987:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:0.0:1787739072.350995:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739072.351000:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5200/0x7ee4453e4a1ae441 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae448 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.351006:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5200/0x7ee4453e4a1ae441 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae448 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.351012:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5200/0x7ee4453e4a1ae441 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae448 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.351029:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:0.0:1787739072.351033:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739072.351038:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5e00/0x7ee4453e4a1ae44f lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae456 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.351045:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5e00/0x7ee4453e4a1ae44f lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae456 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.351112:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+42:17726:x1874579652529024:12345-192.168.203.15@tcp:36:rm.0 Request processed in 47849us (49603us total) trans 4294968447 rc 0/0 00000100:00100000:0.0:1787739072.351120:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23661 00010000:00010000:3.0:1787739072.351300:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1787739072.351307:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae479 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae480 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.351314:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aac00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739072.351319:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae479 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae480 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739072.351325:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae479 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae480 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.351356:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae479 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae480 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.351369:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739072.351398:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae479 lrc: 1/0,0 mode: --/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae480 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.351406:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae479 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae480 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.351443:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787739072.351456:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088a00/0x7ee4453e4a1ae45d lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae464 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.351463:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306088a00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739072.351467:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088a00/0x7ee4453e4a1ae45d lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae464 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739072.351474:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088a00/0x7ee4453e4a1ae45d lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae464 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.351490:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088a00/0x7ee4453e4a1ae45d lrc: 2/0,0 mode: --/EX res: [0x240000402:0x39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae464 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.351496:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739072.351512:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088a00/0x7ee4453e4a1ae45d lrc: 1/0,0 mode: --/EX res: [0x240000402:0x39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae464 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.351519:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088a00/0x7ee4453e4a1ae45d lrc: 0/0,0 mode: --/EX res: [0x240000402:0x39:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae464 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.351547:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739072.351550:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5200/0x7ee4453e4a1ae441 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae448 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.351557:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07404cc5200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739072.351561:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5200/0x7ee4453e4a1ae441 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae448 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739072.351567:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5200/0x7ee4453e4a1ae441 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae448 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.351583:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5200/0x7ee4453e4a1ae441 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae448 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.351589:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739072.351594:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5200/0x7ee4453e4a1ae441 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae448 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.351868:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5200/0x7ee4453e4a1ae441 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae448 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.351883:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739072.351885:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739072.351887:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739072.351895:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739072.351902:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787739072.351917:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d86d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665753984:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739072.351983:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739072.352031:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d85e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665754112:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739072.352053:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739072.352084:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787739072.352089:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d84000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665754240:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739072.352114:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739072.352145:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665753984 00000100:00100000:3.0:1787739072.352151:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665753984 00000100:00100000:3.0:1787739072.352188:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665754112 00000100:00100000:3.0:1787739072.352191:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665754112 00000100:00100000:3.0:1787739072.352206:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665754240 00000100:00100000:3.0:1787739072.352209:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665754240 00000100:00100000:3.0:1787739072.352223:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1367 00000100:00100000:3.0:1787739072.352230:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d85a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+18:3650:x1874579665753984:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739072.352242:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739072.352259:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4800/0x7ee4453e4a1ae480 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae479 expref: 18 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.352272:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffffa07305eaed00) 00010000:00010000:3.0:1787739072.352279:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739072.352323:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d85a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+17:3650:x1874579665753984:12345-0@lo:103:ldlm_bl.0 Request processed in 94us (340us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739072.352330:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1367 00000100:00100000:3.0:1787739072.352345:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1368 00000100:00100000:3.0:1787739072.352350:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d86580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+16:3652:x1874579665754112:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739072.352358:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739072.352369:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae464 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x39:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae45d expref: 16 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.352384:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x39:0x0].0x0 (ffffa0730608bd00) 00010000:00010000:3.0:1787739072.352390:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739072.352440:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d86580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+15:3652:x1874579665754112:12345-0@lo:103:ldlm_bl.0 Request processed in 89us (385us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739072.352459:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1368 00000100:00100000:3.0:1787739072.352472:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1369 00000100:00100000:3.0:1787739072.352478:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d86940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+14:3652:x1874579665754240:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739072.352486:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739072.352496:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1400/0x7ee4453e4a1ae448 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae441 expref: 14 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.352506:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5100) 00010000:00010000:3.0:1787739072.352512:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739072.352549:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d86940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+13:3652:x1874579665754240:12345-0@lo:103:ldlm_bl.0 Request processed in 71us (434us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739072.352555:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1369 00000100:00100000:3.0:1787739072.352592:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d84000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665754240:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739072.353226:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d85e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665754112:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739072.353263:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d86d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665753984:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739072.353770:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739072.364839:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.364930:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652529536 00000100:00100000:2.0:1787739072.364940:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652529536 00000100:00100000:2.0:1787739072.365013:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23662 00000100:00100000:2.0:1787739072.365024:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356acb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+42:17726:x1874579652529536:12345-192.168.203.15@tcp:101:rm.0 00010000:00010000:2.0:1787739072.365048:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739072.365068:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae487 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095463 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739072.365474:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x21:0x0]: returning size 4096 00010000:00010000:2.0:1787739072.365735:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae487 lrc: 1/0,0 mode: --/CR res: [0x240000402:0x21:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095463 expref: 44 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.365750:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x21:0x0].0x0 (ffffa0730713ad00) 00010000:00010000:2.0:1787739072.365753:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739072.365759:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae487 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x21:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095463 expref: 44 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.365772:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa0000, rc 0) 00000100:00100000:2.0:1787739072.365830:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356acb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+43:17726:x1874579652529536:12345-192.168.203.15@tcp:101:rm.0 Request processed in 807us (1000us total) trans 4294968448 rc 301/301 00000100:00100000:2.0:1787739072.365838:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23662 00000100:00100000:1.0:1787739072.386842:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739072.387387:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652529792 00000100:00100000:0.0:1787739072.387396:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652529792 00000100:00100000:0.0:1787739072.387488:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5059 00000100:00100000:0.0:1787739072.387499:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ada40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+43:17726:x1874579652529792:12345-192.168.203.15@tcp:35:rm.0 00000100:00100000:1.0:1787739072.388054:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ada40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+44:17726:x1874579652529792:12345-192.168.203.15@tcp:35:rm.0 Request processed in 555us (1219us total) trans 4294968449 rc 0/0 00000100:00100000:1.0:1787739072.388063:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5059 00000100:00100000:3.0:1787739072.393529:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739072.393564:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.393690:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652529920 00000100:00100000:2.0:1787739072.393699:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652529920 00000100:00100000:2.0:1787739072.393789:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1370 00000100:00100000:2.0:1787739072.393799:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d85e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:2407:x1874579652529920:12345-192.168.203.15@tcp:103:rm.0 00010000:00010000:2.0:1787739072.393817:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739072.393840:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a200/0x7ee4453e4a1ae1d2 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x21:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095375 expref: 61 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.393859:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x21:0x0].0x0 (ffffa07305ca2d00) 00010000:00010000:2.0:1787739072.393870:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739072.393931:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d85e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:2407:x1874579652529920:12345-192.168.203.15@tcp:103:rm.0 Request processed in 134us (366us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.393946:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1370 00000100:00100000:0.0:1787739072.394831:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652530176 00000100:00100000:0.0:1787739072.394840:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652530176 00000100:00100000:0.0:1787739072.394887:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23663 00000100:00100000:0.0:1787739072.394896:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d86d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+44:17726:x1874579652530176:12345-192.168.203.15@tcp:36:rm.0 00010000:00010000:0.0:1787739072.394947:0:6524:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739072.394974:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1da00/0x7ee4453e4a1ae1ee lrc: 0/0,0 mode: --/PR res: [0x240000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095383 expref: 44 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.394991:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x21:0x0].0x0 (ffffa0730713ad00) 00010000:00010000:0.0:1787739072.395000:0:6524:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739072.395044:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1ae495 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.395052:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1ae495 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.395063:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1ae495 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.395083:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252a580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665754368:0@lo:101:mdt00_001.0 00000100:00100000:0.0:1787739072.395132:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739072.395168:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803780 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.395258:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665754368 00000100:00100000:1.0:1787739072.395262:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665754368 00000100:00100000:1.0:1787739072.395295:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3520 00000100:00100000:1.0:1787739072.395301:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+13:6524:x1874579665754368:12345-0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739072.395314:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739072.395329:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031e00/0x7ee4453e4a1ae49c lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae495 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.395362:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031e00/0x7ee4453e4a1ae49c lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae495 expref: 14 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.395376:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031e00/0x7ee4453e4a1ae49c lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae495 expref: 14 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.395383:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eaec00) 00010000:00010000:1.0:1787739072.395385:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739072.395388:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07400031e00, rc 0) 00000100:00100000:1.0:1787739072.395463:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+14:6524:x1874579665754368:12345-0@lo:101:mdt00_001.0 Request processed in 162us (331us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.395469:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3520 00000100:00100000:0.0:1787739072.395835:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252a580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665754368:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739072.395851:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1ae495 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae49c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.395862:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1ae495 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae49c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739072.395880:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739072.395903:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089800/0x7ee4453e4a1ae4a3 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xe1386388 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.395909:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089800/0x7ee4453e4a1ae4a3 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xe1386388 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.395917:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089800/0x7ee4453e4a1ae4a3 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xe1386388 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.395929:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252a580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665754496:0@lo:101:mdt00_001.0 00000100:00100000:0.0:1787739072.395967:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739072.395993:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803780 going to sleep for 11 seconds 00000100:00100000:2.0:1787739072.396179:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665754496 00000100:00100000:2.0:1787739072.396184:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665754496 00000100:00100000:2.0:1787739072.396221:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3521 00000100:00100000:2.0:1787739072.396229:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:6524:x1874579665754496:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739072.396239:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739072.396252:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae4aa lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xe1386388 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae4a3 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.396269:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae4aa lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xe1386388 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae4a3 expref: 15 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.396285:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae4aa lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xe1386388 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae4a3 expref: 15 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.396292:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xe1386388 (ffffa074419d5500) 00010000:00010000:2.0:1787739072.396295:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739072.396298:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa0000, rc 0) 00000100:00100000:2.0:1787739072.396331:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+15:6524:x1874579665754496:12345-0@lo:101:mdt00_001.0 Request processed in 103us (363us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.396337:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3521 00000100:00100000:0.0:1787739072.397150:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252a580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665754496:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739072.397167:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089800/0x7ee4453e4a1ae4a3 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xe1386388 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae4aa expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.397178:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089800/0x7ee4453e4a1ae4a3 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xe1386388 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae4aa expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739072.397193:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:0.0:1787739072.397231:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252a580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665754624:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739072.397266:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739072.397291:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.397308:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665754624 00000100:00100000:0.0:1787739072.397312:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665754624 00000100:00100000:0.0:1787739072.397349:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3522 00000100:00100000:0.0:1787739072.397364:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+15:6524:x1874579665754624:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739072.397445:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+15:6524:x1874579665754624:12345-0@lo:1000:mdt00_001.0 Request processed in 81us (178us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.397451:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3522 00000100:00100000:1.0:1787739072.397683:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252a580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665754624:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739072.397734:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356af480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665754752:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739072.397761:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739072.397784:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a459000 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.397934:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665754752 00000100:00100000:0.0:1787739072.397938:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665754752 00000100:00100000:0.0:1787739072.397978:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3523 00000100:00100000:0.0:1787739072.397984:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356aed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+15:6524:x1874579665754752:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739072.398065:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356aed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+15:6524:x1874579665754752:12345-0@lo:1000:mdt00_001.0 Request processed in 80us (302us total) trans 0 rc 1/1 00000100:00100000:0.0:1787739072.398071:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3523 00000100:00100000:1.0:1787739072.398161:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356af480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665754752:0@lo:1000:mdt00_001.0 00010000:00010000:1.0:1787739072.398210:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fd000/0x7ee4453e4a1ae4b1 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.398219:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fd000/0x7ee4453e4a1ae4b1 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x21:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.398229:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fd000/0x7ee4453e4a1ae4b1 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x21:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.398240:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356af480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665754880:0@lo:101:mdt00_001.0 00000100:00100000:1.0:1787739072.398263:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739072.398284:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a459000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739072.398326:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665754880 00000100:00100000:2.0:1787739072.398330:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665754880 00000100:00100000:2.0:1787739072.398365:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3524 00000100:00100000:2.0:1787739072.398371:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+15:6524:x1874579665754880:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739072.398379:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739072.398394:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae4b8 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae4b1 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.398434:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae4b8 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x21:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae4b1 expref: 16 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.398446:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae4b8 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x21:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae4b1 expref: 16 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.398453:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x21:0x0].0x0 (ffffa074419d5900) 00010000:00010000:2.0:1787739072.398456:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739072.398460:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa1800, rc 0) 00000100:00100000:2.0:1787739072.398497:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+16:6524:x1874579665754880:12345-0@lo:101:mdt00_001.0 Request processed in 126us (233us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.398503:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3524 00000100:00100000:1.0:1787739072.398623:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356af480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665754880:0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739072.398634:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fd000/0x7ee4453e4a1ae4b1 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x21:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae4b8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.398643:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fd000/0x7ee4453e4a1ae4b1 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x21:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae4b8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.398674:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840600/0x7ee4453e4a1ae4bf lrc: 3/0,1 mode: --/EX res: [0x240000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.398682:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840600/0x7ee4453e4a1ae4bf lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.398689:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840600/0x7ee4453e4a1ae4bf lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.398708:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae4c6 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x14:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.398869:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ae1fc lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09538a expref: 43 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.398879:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae4c6 lrc: 4/0,1 mode: --/EX res: [0x240000400:0x14:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.398902:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ae1fc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09538a expref: 43 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.398914:0:6524:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ae1fc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09538a expref: 43 pid: 6525 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.398925:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356af480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665755008:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739072.399002:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a459000 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.403610:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356af480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665755008:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739072.403648:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae4c6 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x14:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.406576:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739072.406887:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652530304 00000100:00100000:3.0:1787739072.406896:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652530304 00010000:00010000:3.0:1787739072.406932:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ae1fc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09538a expref: 44 pid: 6525 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.407029:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1573 00000100:00100000:3.0:1787739072.407039:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356af480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+44:2408:x1874579652530304:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739072.407053:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739072.407061:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ae1fc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09538a expref: 44 pid: 6525 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.407074:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ae1fc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09538a expref: 44 pid: 6525 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.407095:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1dc00/0x7ee4453e4a1ae1fc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09538a expref: 44 pid: 6525 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.407112:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x14:0x0].0x0 (ffffa0730713ac00) 00010000:00010000:3.0:1787739072.407117:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae4c6 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x14:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.407126:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae4c6 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x14:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.407132:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae4c6 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0x14:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.407161:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739072.407216:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356af480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+43:2408:x1874579652530304:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 177us (646us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739072.407223:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1573 00010000:00010000:1.0:1787739072.407275:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae4c6 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.407283:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae4c6 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.407340:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306841800/0x7ee4453e4a1ae4cd lrc: 3/0,1 mode: --/EX res: [0x200000401:0x14:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.407348:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306841800/0x7ee4453e4a1ae4cd lrc: 3/0,1 mode: --/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.407364:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306841800/0x7ee4453e4a1ae4cd lrc: 3/0,1 mode: --/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.407384:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665755136:0@lo:101:mdt00_001.0 00000100:00100000:1.0:1787739072.407453:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739072.407482:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a459000 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.407497:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665755136 00000100:00100000:1.0:1787739072.407500:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665755136 00000100:00100000:1.0:1787739072.407525:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3525 00000100:00100000:1.0:1787739072.407531:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+16:6524:x1874579665755136:12345-0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739072.407544:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739072.407555:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1ae4d4 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x14:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae4cd expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.407574:0:12410:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ae20a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095391 expref: 59 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.407583:0:12410:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1ae4d4 lrc: 4/0,0 mode: --/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae4cd expref: 17 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.407613:0:12410:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ae20a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095391 expref: 59 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.407624:0:12410:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ae20a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095391 expref: 59 pid: 7944 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.407633:0:12410:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ac780 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:12410:1874579665755264:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739072.407676:0:12410:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a459180 going to sleep for 11 seconds 00000100:00100000:2.0:1787739072.413440:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739072.413521:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652530432 00000100:00100000:3.0:1787739072.413529:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652530432 00010000:00010000:3.0:1787739072.413569:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ae20a lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095391 expref: 60 pid: 7944 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.413616:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1574 00000100:00100000:3.0:1787739072.413625:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f7c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:2409:x1874579652530432:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739072.413640:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739072.413674:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ae20a lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x14:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095391 expref: 60 pid: 7944 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.413689:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ae20a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x14:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095391 expref: 60 pid: 7944 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.413713:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x14:0x0].0x0 (ffffa07403eeb700) 00010000:00010000:3.0:1787739072.413731:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1ae4d4 lrc: 3/0,0 mode: --/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae4cd expref: 17 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.413743:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1ae4d4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae4cd expref: 17 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.413759:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1ae4d4 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae4cd expref: 17 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.413800:0:6509:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1ae4d4 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae4cd expref: 17 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.413823:0:6509:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d84000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665755392:0@lo:105: 00000100:00100000:3.0:1787739072.413884:0:6509:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739072.413960:0:6509:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410cf69c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.414148:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665755392 00000100:00100000:0.0:1787739072.414153:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665755392 00000100:00100000:0.0:1787739072.414304:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1202 00000100:00100000:0.0:1787739072.414313:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d843c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665755392:12345-0@lo:105: 00010000:00010000:0.0:1787739072.414324:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306841800/0x7ee4453e4a1ae4cd lrc: 4/0,1 mode: --/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae4d4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.414335:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306841800/0x7ee4453e4a1ae4cd lrc: 4/0,1 mode: --/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae4d4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.414342:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306841800/0x7ee4453e4a1ae4cd lrc: 4/0,1 mode: --/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae4d4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.414350:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306841800/0x7ee4453e4a1ae4cd lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae4d4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.414358:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306841800/0x7ee4453e4a1ae4cd lrc: 5/0,1 mode: EX/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae4d4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.414366:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306841800/0x7ee4453e4a1ae4cd lrc: 5/0,1 mode: EX/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ae4d4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.414379:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07306841800) 00000100:00100000:0.0:1787739072.414449:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d843c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6509:x1874579665755392:12345-0@lo:105: Request processed in 137us (563us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.414456:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1202 00000100:00100000:3.0:1787739072.414538:0:6509:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d84000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6509:1874579665755392:0@lo:105: 00010000:00010000:3.0:1787739072.414554:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739072.414609:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f7c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:2409:x1874579652530432:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 984us (1223us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739072.414616:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1574 00010000:00010000:1.0:1787739072.414913:0:12410:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633de00/0x7ee4453e4a1ae20a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095391 expref: 59 pid: 7944 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.414931:0:12410:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ac780 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:12410:1874579665755264:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739072.414960:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1ae4d4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae4cd expref: 17 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.414969:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x14:0x0].0x0 (ffffa07403eeb700) 00010000:00010000:1.0:1787739072.414972:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739072.414975:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841000, rc 0) 00000100:00100000:1.0:1787739072.415059:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+17:6524:x1874579665755136:12345-0@lo:101:mdt00_001.0 Request processed in 7528us (7604us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.415066:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3525 00000100:00100000:3.0:1787739072.416966:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665755136:0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787739072.416994:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306841800/0x7ee4453e4a1ae4cd lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae4d4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.418190:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665755520:0@lo:403:mdt00_001.0 00000100:00100000:2.0:1787739072.418257:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.418293:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404c00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.418563:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665755520 00000100:00100000:0.0:1787739072.418572:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665755520 00000100:00100000:0.0:1787739072.418623:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3526 00000100:00100000:0.0:1787739072.418631:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f43c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+17:6524:x1874579665755520:12345-0@lo:403:mdt00_001.0 00000100:00100000:0.0:1787739072.421965:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f43c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+17:6524:x1874579665755520:12345-0@lo:403:mdt00_001.0 Request processed in 3319us (3692us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.421979:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3526 00000100:00100000:1.0:1787739072.422640:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-mdtlov_UUID:6524:1874579665755520:0@lo:403:mdt00_001.0 00000040:00080000:1.0:1787739072.424199:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].376, 952 off565656 00000040:00080000:1.0:1787739072.424379:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].376, 1032 off617312 00000004:00080000:1.0:1787739072.424386:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0740a459180 with version 715 00000004:00080000:3.0:1787739072.424558:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 715 rpc_version 715 00000020:00100000:3.0:1787739072.424591:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739072.424664:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304d84000 x1874579665755776/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739072.424853:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d84000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665755776:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739072.424923:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739072.425009:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410cf69c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.425050:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665755776 00000100:00100000:1.0:1787739072.425056:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665755776 00000100:00100000:1.0:1787739072.425109:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3527 00000100:00100000:1.0:1787739072.425118:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d870c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+17:7467:x1874579665755776:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739072.425611:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d870c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+18:7467:x1874579665755776:12345-0@lo:1000:osp_up0-1.0 Request processed in 492us (686us total) trans 4294983462 rc 0/0 00000100:00100000:1.0:1787739072.425620:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3527 00000100:00100000:3.0:1787739072.425966:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d84000 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665755776:0@lo:1000:osp_up0-1.0 00000004:00200000:1.0:1787739072.426033:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x21:0x0]: returning size 4096 00010000:00010000:1.0:1787739072.426055:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae4c6 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.426065:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae4c6 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.426072:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae4c6 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.426270:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840a00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739072.426277:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae4c6 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.426287:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae4c6 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.426302:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x14:0x0].0x0 (ffffa0730713ac00) 00010000:00010000:1.0:1787739072.426305:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739072.426309:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae4c6 lrc: 2/0,0 mode: --/EX res: [0x240000400:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.426315:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae4c6 lrc: 0/0,0 mode: --/EX res: [0x240000400:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.426336:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306841800/0x7ee4453e4a1ae4cd lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae4d4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.426344:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306841800/0x7ee4453e4a1ae4cd lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae4d4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.426351:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306841800/0x7ee4453e4a1ae4cd lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae4d4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.426385:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739072.426393:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x14:0x0] 00010000:00010000:1.0:1787739072.426402:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840600/0x7ee4453e4a1ae4bf lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.426408:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840600/0x7ee4453e4a1ae4bf lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.426541:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840600/0x7ee4453e4a1ae4bf lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.426547:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739072.426550:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840600/0x7ee4453e4a1ae4bf lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.426556:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840600/0x7ee4453e4a1ae4bf lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.426564:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x21:0x0].0x0 (ffffa07305eaed00) 00010000:00010000:1.0:1787739072.426567:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739072.426570:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840600/0x7ee4453e4a1ae4bf lrc: 2/0,0 mode: --/EX res: [0x240000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.426576:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840600/0x7ee4453e4a1ae4bf lrc: 0/0,0 mode: --/EX res: [0x240000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.426588:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fd000/0x7ee4453e4a1ae4b1 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x21:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae4b8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.426659:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fd000/0x7ee4453e4a1ae4b1 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x21:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae4b8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.426832:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739072.426839:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306841800/0x7ee4453e4a1ae4cd lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae4d4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.426846:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306841800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739072.426851:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306841800/0x7ee4453e4a1ae4cd lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae4d4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739072.426858:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306841800/0x7ee4453e4a1ae4cd lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae4d4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.426889:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306841800/0x7ee4453e4a1ae4cd lrc: 2/0,0 mode: --/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae4d4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.426895:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739072.426928:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306841800/0x7ee4453e4a1ae4cd lrc: 1/0,0 mode: --/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae4d4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.426935:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306841800/0x7ee4453e4a1ae4cd lrc: 0/0,0 mode: --/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae4d4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.426944:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665755904:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739072.426958:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739072.426960:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739072.426961:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739072.427013:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.427070:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665755904 00000100:00100000:2.0:1787739072.427077:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665755904 00010000:00010000:1.0:1787739072.427112:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fd000/0x7ee4453e4a1ae4b1 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x21:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae4b8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.427122:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1371 00000100:00100000:2.0:1787739072.427130:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba4850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+18:3650:x1874579665755904:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739072.427141:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739072.427157:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1ae4d4 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x14:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae4cd expref: 18 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.427169:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x14:0x0].0x0 (ffffa07403eeb700) 00010000:00010000:1.0:1787739072.427179:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739072.427192:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739072.427196:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1ae495 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae49c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.427205:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1ae495 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae49c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.427219:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1ae495 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae49c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.427241:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1787739072.427245:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739072.427247:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fd000/0x7ee4453e4a1ae4b1 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x21:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae4b8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.427253:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fd000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739072.427257:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fd000/0x7ee4453e4a1ae4b1 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x21:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae4b8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739072.427265:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fd000/0x7ee4453e4a1ae4b1 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x21:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae4b8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.427287:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba4850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+17:3650:x1874579665755904:12345-0@lo:103:ldlm_bl.0 Request processed in 159us (274us total) trans 0 rc 0/0 00010000:00010000:3.0:1787739072.427292:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fd000/0x7ee4453e4a1ae4b1 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x21:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae4b8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.427294:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1371 00010000:00010000:3.0:1787739072.427298:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739072.427315:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fd000/0x7ee4453e4a1ae4b1 lrc: 1/0,0 mode: --/EX res: [0x240000402:0x21:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae4b8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.427322:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073063fd000/0x7ee4453e4a1ae4b1 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x21:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae4b8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.427334:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739072.427338:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1ae495 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae49c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.427344:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306088600 already unused, calling callback (ffffffffc18e6ee0) 00000100:00100000:2.0:1787739072.427344:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d85680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665756032:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739072.427348:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1ae495 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae49c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739072.427366:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1ae495 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae49c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.427367:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787739072.427382:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1ae495 lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae49c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.427388:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739072.427405:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1ae495 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae49c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.427428:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739072.427430:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739072.427430:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d87480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665756160:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739072.427431:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739072.427444:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665756032 00000100:00100000:3.0:1787739072.427448:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665756032 00000100:00100000:2.0:1787739072.427453:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.427479:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665756160 00000100:00100000:2.0:1787739072.427482:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665756160 00000100:00100000:3.0:1787739072.427494:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1372 00000100:00100000:3.0:1787739072.427500:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f7c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+18:3651:x1874579665756032:12345-0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739072.427501:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1373 00000100:00100000:2.0:1787739072.427506:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f4780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+18:3652:x1874579665756160:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739072.427508:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739072.427512:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739072.427519:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae4b8 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x21:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae4b1 expref: 18 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.427520:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031e00/0x7ee4453e4a1ae49c lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae495 expref: 18 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.427530:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x21:0x0].0x0 (ffffa074419d5900) 00010000:00010000:3.0:1787739072.427535:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739072.427542:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eaec00) 00010000:00010000:2.0:1787739072.427548:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739072.427585:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f7c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+16:3651:x1874579665756032:12345-0@lo:103:ldlm_bl.0 Request processed in 85us (217us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739072.427592:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1372 00000100:00100000:0.0:1787739072.427622:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252a1c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665755904:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739072.428040:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:1.0:1787739072.428047:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1ae495 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae49c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739072.428071:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:1.0:1787739072.428078:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089800/0x7ee4453e4a1ae4a3 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xe1386388 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae4aa expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.428085:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089800/0x7ee4453e4a1ae4a3 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xe1386388 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae4aa expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.428179:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d86d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+43:17726:x1874579652530176:12345-192.168.203.15@tcp:36:rm.0 Request processed in 33283us (34657us total) trans 4294968450 rc 0/0 00000100:00100000:1.0:1787739072.428194:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23663 00010000:00010000:3.0:1787739072.428248:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739072.428985:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f4780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+15:3652:x1874579665756160:12345-0@lo:103:ldlm_bl.0 Request processed in 1475us (1527us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.428994:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1373 00000100:00100000:2.0:1787739072.429039:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d85680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665756032:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739072.429070:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d87480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665756160:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739072.435583:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739072.435647:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652530688 00000100:00100000:0.0:1787739072.435656:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652530688 00000100:00100000:0.0:1787739072.435717:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23664 00000100:00100000:0.0:1787739072.435727:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+43:17726:x1874579652530688:12345-192.168.203.15@tcp:101:rm.0 00010000:00010000:0.0:1787739072.435866:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739072.435887:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089200/0x7ee4453e4a1ae4db lrc: 2/0,0 mode: --/CR res: [0x240000402:0x4e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09546a expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739072.436037:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x4e:0x0]: returning size 4096 00010000:00010000:0.0:1787739072.436377:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089200/0x7ee4453e4a1ae4db lrc: 1/0,0 mode: --/CR res: [0x240000402:0x4e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09546a expref: 45 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.436392:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4e:0x0].0x0 (ffffa07403eeba00) 00010000:00010000:0.0:1787739072.436395:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739072.436400:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089200/0x7ee4453e4a1ae4db lrc: 0/0,0 mode: --/CR res: [0x240000402:0x4e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09546a expref: 45 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.436499:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306089200, rc 0) 00000100:00100000:0.0:1787739072.436608:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+44:17726:x1874579652530688:12345-192.168.203.15@tcp:101:rm.0 Request processed in 882us (1034us total) trans 4294968451 rc 301/301 00000100:00100000:0.0:1787739072.436615:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23664 00000100:00100000:0.0:1787739072.442350:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.442462:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652530944 00000100:00100000:2.0:1787739072.442471:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652530944 00000100:00100000:2.0:1787739072.442525:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5060 00000100:00100000:2.0:1787739072.442533:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074332aa450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+44:17726:x1874579652530944:12345-192.168.203.15@tcp:35:rm.0 00000100:00100000:2.0:1787739072.442764:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074332aa450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+45:17726:x1874579652530944:12345-192.168.203.15@tcp:35:rm.0 Request processed in 230us (421us total) trans 4294968452 rc 0/0 00000100:00100000:2.0:1787739072.442771:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5060 00000100:00100000:1.0:1787739072.452848:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739072.452899:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739072.452981:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652531072 00000100:00100000:3.0:1787739072.452991:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652531072 00000100:00100000:3.0:1787739072.453068:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1374 00000100:00100000:3.0:1787739072.453078:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356afc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+59:2406:x1874579652531072:12345-192.168.203.15@tcp:103:rm.0 00010000:00010000:3.0:1787739072.453094:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739072.453120:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc4c00/0x7ee4453e4a1ae368 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x4e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09541d expref: 59 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.453149:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4e:0x0].0x0 (ffffa07305eae600) 00010000:00010000:3.0:1787739072.453160:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739072.453225:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356afc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+58:2406:x1874579652531072:12345-192.168.203.15@tcp:103:rm.0 Request processed in 149us (330us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739072.453232:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1374 00000100:00100000:2.0:1787739072.453606:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652531328 00000100:00100000:2.0:1787739072.453611:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652531328 00000100:00100000:2.0:1787739072.453649:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23665 00000100:00100000:2.0:1787739072.453656:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+45:17726:x1874579652531328:12345-192.168.203.15@tcp:36:rm.0 00010000:00010000:2.0:1787739072.453707:0:7944:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739072.453722:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633dc00/0x7ee4453e4a1ae384 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x4e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09542b expref: 45 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.453878:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4e:0x0].0x0 (ffffa07403eeba00) 00010000:00010000:2.0:1787739072.453887:0:7944:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739072.453938:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa0400/0x7ee4453e4a1ae4e9 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.453945:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa0400/0x7ee4453e4a1ae4e9 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.453964:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa0400/0x7ee4453e4a1ae4e9 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.453986:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665756288:0@lo:101:mdt00_003.0 00000100:00100000:2.0:1787739072.454032:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.454118:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.454855:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665756288 00000100:00100000:1.0:1787739072.454863:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665756288 00000100:00100000:1.0:1787739072.454905:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3528 00000100:00100000:1.0:1787739072.454916:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+15:7944:x1874579665756288:12345-0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739072.454936:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739072.455164:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae4f0 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae4e9 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.455191:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae4f0 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae4e9 expref: 16 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.455206:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae4f0 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae4e9 expref: 16 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.455214:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eaed00) 00010000:00010000:1.0:1787739072.455217:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739072.455221:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306840c00, rc 0) 00000100:00100000:1.0:1787739072.455322:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+16:7944:x1874579665756288:12345-0@lo:101:mdt00_003.0 Request processed in 408us (1288us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.455328:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3528 00000100:00100000:3.0:1787739072.455527:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f61c0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665756288:0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739072.455545:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa0400/0x7ee4453e4a1ae4e9 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae4f0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.455557:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa0400/0x7ee4453e4a1ae4e9 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae4f0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739072.455580:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739072.455607:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ae4f7 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.455614:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ae4f7 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.455622:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ae4f7 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.455636:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d84780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665756416:0@lo:101:mdt00_003.0 00000100:00100000:3.0:1787739072.455664:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739072.455737:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740133a780 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.455768:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665756416 00000100:00100000:1.0:1787739072.455772:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665756416 00000100:00100000:1.0:1787739072.455797:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3529 00000100:00100000:1.0:1787739072.455802:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d852c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+16:7944:x1874579665756416:12345-0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739072.455811:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739072.455822:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841200/0x7ee4453e4a1ae4fe lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae4f7 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.455837:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841200/0x7ee4453e4a1ae4fe lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae4f7 expref: 17 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.455847:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841200/0x7ee4453e4a1ae4fe lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae4f7 expref: 17 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.455855:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x6ae86992 (ffffa07305eae300) 00010000:00010000:1.0:1787739072.455858:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739072.455860:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841200, rc 0) 00000100:00100000:1.0:1787739072.455922:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d852c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+17:7944:x1874579665756416:12345-0@lo:101:mdt00_003.0 Request processed in 118us (256us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.455928:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3529 00000100:00100000:3.0:1787739072.456194:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d84780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665756416:0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739072.456207:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ae4f7 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae4fe expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.456216:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ae4f7 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae4fe expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739072.456229:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:3.0:1787739072.456259:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d84780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665756544:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739072.456286:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739072.456499:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410cf69c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739072.456629:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665756544 00000100:00100000:2.0:1787739072.456634:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665756544 00000100:00100000:2.0:1787739072.456710:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3530 00000100:00100000:2.0:1787739072.456718:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+17:7944:x1874579665756544:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739072.456791:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+17:7944:x1874579665756544:12345-0@lo:1000:mdt00_003.0 Request processed in 74us (503us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.456798:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3530 00000100:00100000:3.0:1787739072.456845:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d84780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665756544:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739072.456887:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d84780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665756672:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739072.456909:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739072.457067:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e600 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.457261:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665756672 00000100:00100000:1.0:1787739072.457266:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665756672 00000100:00100000:1.0:1787739072.457288:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3531 00000100:00100000:1.0:1787739072.457293:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d87840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+17:7944:x1874579665756672:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739072.457384:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d87840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+17:7944:x1874579665756672:12345-0@lo:1000:mdt00_003.0 Request processed in 90us (474us total) trans 0 rc 1/1 00000100:00100000:1.0:1787739072.457390:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3531 00000100:00100000:3.0:1787739072.457655:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d84780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665756672:0@lo:1000:mdt00_003.0 00010000:00010000:3.0:1787739072.457704:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae505 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.457714:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae505 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.457723:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae505 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.457735:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d84780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665756800:0@lo:101:mdt00_003.0 00000100:00100000:3.0:1787739072.457760:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739072.457787:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.457825:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665756800 00000100:00100000:1.0:1787739072.457830:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665756800 00000100:00100000:1.0:1787739072.457851:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3532 00000100:00100000:1.0:1787739072.457856:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d87480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+17:7944:x1874579665756800:12345-0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739072.457866:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739072.457881:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1ae50c lrc: 2/0,0 mode: --/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae505 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.457902:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1ae50c lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae505 expref: 18 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.457915:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1ae50c lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae505 expref: 18 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.457927:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4e:0x0].0x0 (ffffa07305eaec00) 00010000:00010000:1.0:1787739072.457929:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739072.457933:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306840e00, rc 0) 00000100:00100000:1.0:1787739072.458292:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d87480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+18:7944:x1874579665756800:12345-0@lo:101:mdt00_003.0 Request processed in 433us (529us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.458299:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3532 00000100:00100000:3.0:1787739072.458619:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d84780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665756800:0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739072.458634:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae505 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae50c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.458646:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae505 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae50c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.458674:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae513 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.458683:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae513 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.458691:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae513 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.458711:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae51a lrc: 3/0,1 mode: --/EX res: [0x240000400:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.458723:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae392 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095432 expref: 44 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.458730:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae51a lrc: 4/0,1 mode: --/EX res: [0x240000400:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.458752:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae392 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095432 expref: 44 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.458766:0:7944:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae392 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095432 expref: 44 pid: 6524 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.458776:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d84780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665756928:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739072.459162:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739072.466695:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d84780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001_UUID:7944:1874579665756928:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739072.466722:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae51a lrc: 3/0,1 mode: --/EX res: [0x240000400:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.471232:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739072.471650:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652531456 00000100:00100000:3.0:1787739072.471660:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652531456 00010000:00010000:3.0:1787739072.471711:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae392 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095432 expref: 45 pid: 6524 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.471766:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1575 00000100:00100000:3.0:1787739072.471776:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ae940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+45:2409:x1874579652531456:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739072.471792:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739072.471799:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae392 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095432 expref: 45 pid: 6524 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.471812:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae392 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095432 expref: 45 pid: 6524 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.471845:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae392 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095432 expref: 45 pid: 6524 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.471862:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x2d:0x0].0x0 (ffffa07403eebe00) 00010000:00010000:3.0:1787739072.471867:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae51a lrc: 3/0,1 mode: --/EX res: [0x240000400:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.471876:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae51a lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.471883:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae51a lrc: 4/0,1 mode: EX/EX res: [0x240000400:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.471914:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739072.471962:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae51a lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.471973:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae51a lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.471979:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ae940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+44:2409:x1874579652531456:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 203us (755us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739072.471995:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1575 00010000:00010000:2.0:1787739072.472096:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1c00/0x7ee4453e4a1ae521 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.472106:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1c00/0x7ee4453e4a1ae521 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.472121:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1c00/0x7ee4453e4a1ae521 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.472146:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665757056:0@lo:101:mdt00_003.0 00000100:00100000:2.0:1787739072.472201:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.472238:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739072.472350:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665757056 00000100:00100000:3.0:1787739072.472365:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665757056 00000100:00100000:3.0:1787739072.472405:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3533 00000100:00100000:3.0:1787739072.472445:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+18:7944:x1874579665757056:12345-0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739072.472460:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739072.472474:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae528 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae521 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.472495:0:12410:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1ae3a0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095439 expref: 57 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.472506:0:12410:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae528 lrc: 4/0,0 mode: --/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae521 expref: 19 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.472527:0:12410:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1ae3a0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095439 expref: 57 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.472539:0:12410:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1ae3a0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095439 expref: 57 pid: 7944 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.472550:0:12410:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d84780 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:12410:1874579665757184:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739072.472589:0:12410:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739072.480037:0:12410:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d84780 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:12410:1874579665757184:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739072.480083:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae528 lrc: 3/0,0 mode: --/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae521 expref: 19 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.480098:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2d:0x0].0x0 (ffffa07403eebc00) 00010000:00010000:3.0:1787739072.480101:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739072.480105:0:12410:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae528 lrc: 3/0,0 mode: --/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae521 expref: 19 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.480117:0:12410:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae528 lrc: 3/0,0 mode: --/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae521 expref: 19 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.480124:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abe00, rc 0) 00000100:00100000:3.0:1787739072.480206:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+19:7944:x1874579665757056:12345-0@lo:101:mdt00_003.0 Request processed in 7786us (7995us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739072.480216:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3533 00000100:00100000:2.0:1787739072.482116:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665757056:0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739072.482141:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1c00/0x7ee4453e4a1ae521 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae528 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.482155:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1c00/0x7ee4453e4a1ae521 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae528 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.486514:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.487021:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652531584 00000100:00100000:2.0:1787739072.487031:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652531584 00010000:00010000:2.0:1787739072.487076:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1ae3a0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095439 expref: 58 pid: 7944 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.487122:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1576 00000100:00100000:2.0:1787739072.487131:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356af840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+58:2408:x1874579652531584:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739072.487146:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739072.487153:0:6510:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1ae3a0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095439 expref: 58 pid: 7944 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.487166:0:6510:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1ae3a0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095439 expref: 58 pid: 7944 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.487187:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1ae3a0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095439 expref: 58 pid: 7944 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.487202:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2d:0x0].0x0 (ffffa07403eebc00) 00010000:00010000:2.0:1787739072.487209:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae528 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae521 expref: 18 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.487219:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae528 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae521 expref: 18 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.487228:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae528 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae521 expref: 18 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.487263:0:6510:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae528 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae521 expref: 18 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.487277:0:6510:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a30b40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6510:1874579665757312:0@lo:105: 00000100:00100000:2.0:1787739072.487342:0:6510:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.487382:0:6510:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.487963:0:6506:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787739072.487979:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665757312 00000100:00100000:1.0:1787739072.487983:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665757312 00000100:00100000:1.0:1787739072.488018:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1203 00000100:00100000:1.0:1787739072.488025:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306b99450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6510:x1874579665757312:12345-0@lo:105: 00010000:00010000:1.0:1787739072.488036:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1c00/0x7ee4453e4a1ae521 lrc: 5/0,1 mode: --/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae528 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.488047:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1c00/0x7ee4453e4a1ae521 lrc: 5/0,1 mode: --/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae528 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.488067:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1c00/0x7ee4453e4a1ae521 lrc: 5/0,1 mode: --/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae528 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.488079:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1c00/0x7ee4453e4a1ae521 lrc: 5/0,1 mode: EX/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae528 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.488085:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1c00/0x7ee4453e4a1ae521 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae528 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.488093:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1c00/0x7ee4453e4a1ae521 lrc: 6/0,1 mode: EX/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ae528 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.488112:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07306aa1c00) 00010000:00010000:3.0:1787739072.488128:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1c00/0x7ee4453e4a1ae521 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae528 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.488141:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1c00/0x7ee4453e4a1ae521 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae528 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.488172:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306b99450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0001-mdtlov_UUID+4:6510:x1874579665757312:12345-0@lo:105: Request processed in 147us (828us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.488180:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1203 00000100:00100000:3.0:1787739072.488883:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d84780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665757440:0@lo:403:mdt00_003.0 00000100:00100000:3.0:1787739072.488927:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739072.488968:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.489050:0:6510:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a30b40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6510:1874579665757312:0@lo:105: 00010000:00010000:0.0:1787739072.489065:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739072.489111:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356af840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+57:2408:x1874579652531584:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 1980us (2606us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.489120:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1576 00000100:00100000:1.0:1787739072.489887:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665757440 00000100:00100000:1.0:1787739072.489894:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665757440 00000100:00100000:1.0:1787739072.489934:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3534 00000100:00100000:1.0:1787739072.489941:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d85680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+19:7944:x1874579665757440:12345-0@lo:403:mdt00_003.0 00000100:00100000:1.0:1787739072.493403:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d85680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+19:7944:x1874579665757440:12345-0@lo:403:mdt00_003.0 Request processed in 3446us (4458us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.493444:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3534 00000100:00100000:3.0:1787739072.493759:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d84780 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:7944:1874579665757440:0@lo:403:mdt00_003.0 00000040:00080000:3.0:1787739072.494778:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].377, 952 off566608 00000040:00080000:3.0:1787739072.494925:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].377, 1032 off618344 00000004:00080000:3.0:1787739072.494931:0:7944:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730610e6c0 with version 716 00000004:00080000:2.0:1787739072.495006:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 716 rpc_version 716 00000020:00100000:2.0:1787739072.495040:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739072.495049:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303a31a40 x1874579665757696/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739072.495071:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a31a40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665757696:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739072.495132:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.495165:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739072.495210:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665757696 00000100:00100000:3.0:1787739072.495218:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665757696 00000100:00100000:3.0:1787739072.495267:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3535 00000100:00100000:3.0:1787739072.495274:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+19:7467:x1874579665757696:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739072.495847:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+20:7467:x1874579665757696:12345-0@lo:1000:osp_up0-1.0 Request processed in 571us (713us total) trans 4294983463 rc 0/0 00000100:00100000:3.0:1787739072.495855:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3535 00000100:00100000:2.0:1787739072.496223:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a31a40 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665757696:0@lo:1000:osp_up0-1.0 00000004:00200000:3.0:1787739072.496271:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x4e:0x0]: returning size 4096 00010000:00010000:3.0:1787739072.496296:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae51a lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.496306:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae51a lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.496314:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae51a lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.496319:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aba00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739072.496323:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae51a lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.496330:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae51a lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.496345:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x2d:0x0].0x0 (ffffa07403eebe00) 00010000:00010000:3.0:1787739072.496348:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739072.496358:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae51a lrc: 2/0,0 mode: --/EX res: [0x240000400:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.496364:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae51a lrc: 0/0,0 mode: --/EX res: [0x240000400:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.496382:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1c00/0x7ee4453e4a1ae521 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae528 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.496401:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1c00/0x7ee4453e4a1ae521 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae528 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.496408:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1c00/0x7ee4453e4a1ae521 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae528 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.496655:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739072.496664:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x2d:0x0] 00010000:00010000:3.0:1787739072.496673:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae513 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.496679:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae513 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.496685:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae513 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.496689:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739072.496692:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae513 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.496697:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae513 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.496706:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4e:0x0].0x0 (ffffa0730608b900) 00010000:00010000:3.0:1787739072.496708:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739072.496712:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae513 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.496718:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae513 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.496730:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae505 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae50c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.496736:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae505 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae50c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.496742:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae505 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae50c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.496793:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1787739072.496799:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1c00/0x7ee4453e4a1ae521 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae528 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.496806:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1c00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739072.496810:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1c00/0x7ee4453e4a1ae521 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae528 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739072.496818:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1c00/0x7ee4453e4a1ae521 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae528 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.496847:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1c00/0x7ee4453e4a1ae521 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae528 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.496853:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739072.496890:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1c00/0x7ee4453e4a1ae521 lrc: 1/0,0 mode: --/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae528 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.496897:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1c00/0x7ee4453e4a1ae521 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae528 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.496910:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739072.496913:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae505 lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae50c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.496919:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aac00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739072.496922:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae505 lrc: 5/0,0 mode: EX/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae50c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739072.496929:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae505 lrc: 5/0,0 mode: EX/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae50c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.496944:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae505 lrc: 4/0,0 mode: --/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae50c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.496961:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739072.496977:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae505 lrc: 3/0,0 mode: --/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae50c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.496985:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739072.496986:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739072.496988:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739072.496996:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787739072.497008:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665757952:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739072.497050:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739072.497129:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a30780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665757824:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739072.497158:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739072.497197:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665757952 00000100:00100000:0.0:1787739072.497202:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665757952 00000100:00100000:0.0:1787739072.497235:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665757824 00000100:00100000:0.0:1787739072.497237:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665757824 00000100:00100000:0.0:1787739072.497258:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1375 00000100:00100000:0.0:1787739072.497264:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a330c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+21:3651:x1874579665757952:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739072.497274:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739072.497291:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1ae50c lrc: 0/0,0 mode: --/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae505 expref: 21 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.497304:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x4e:0x0].0x0 (ffffa07305eaec00) 00010000:00010000:0.0:1787739072.497311:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739072.497336:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a330c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+20:3651:x1874579665757952:12345-0@lo:103:ldlm_bl.0 Request processed in 72us (286us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.497343:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1375 00000100:00100000:0.0:1787739072.497358:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1376 00000100:00100000:0.0:1787739072.497364:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba6850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+19:3653:x1874579665757824:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739072.497371:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739072.497380:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae528 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x2d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae521 expref: 19 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.497402:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2d:0x0].0x0 (ffffa07403eebc00) 00010000:00010000:0.0:1787739072.497408:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739072.497463:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba6850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+18:3653:x1874579665757824:12345-0@lo:103:ldlm_bl.0 Request processed in 91us (295us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.497470:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1376 00000100:00100000:0.0:1787739072.497519:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a30780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665757824:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739072.497690:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665757952:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739072.497794:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:3.0:1787739072.497801:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae505 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x4e:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae50c expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739072.497822:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739072.497828:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa0400/0x7ee4453e4a1ae4e9 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae4f0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.497836:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa0400/0x7ee4453e4a1ae4e9 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae4f0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.497843:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa0400/0x7ee4453e4a1ae4e9 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae4f0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.497860:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739072.497864:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739072.497868:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ae4f7 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae4fe expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.497881:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ae4f7 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae4fe expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.497939:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ac780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+44:17726:x1874579652531328:12345-192.168.203.15@tcp:36:rm.0 Request processed in 44282us (45101us total) trans 4294968453 rc 0/0 00000100:00100000:3.0:1787739072.497946:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23665 00010000:00010000:2.0:1787739072.498500:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739072.498508:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa0400/0x7ee4453e4a1ae4e9 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae4f0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.498516:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739072.498521:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa0400/0x7ee4453e4a1ae4e9 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae4f0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739072.498529:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa0400/0x7ee4453e4a1ae4e9 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae4f0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.498556:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa0400/0x7ee4453e4a1ae4e9 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae4f0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.498563:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739072.498582:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa0400/0x7ee4453e4a1ae4e9 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae4f0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.498588:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa0400/0x7ee4453e4a1ae4e9 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae4f0 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.498601:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739072.498602:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739072.498604:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739072.498839:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787739072.498853:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665758080:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739072.498893:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1787739072.499280:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739072.499366:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665758080 00000100:00100000:2.0:1787739072.499374:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665758080 00000100:00100000:2.0:1787739072.499634:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1377 00000100:00100000:2.0:1787739072.499643:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356acf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+18:3650:x1874579665758080:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739072.499654:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739072.499672:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae4f0 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae4e9 expref: 18 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.499687:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305eaed00) 00010000:00010000:2.0:1787739072.499694:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739072.499738:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356acf00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+17:3650:x1874579665758080:12345-0@lo:103:ldlm_bl.0 Request processed in 97us (845us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.499745:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1377 00000100:00100000:1.0:1787739072.499890:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665758080:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739072.508850:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.508991:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652531840 00000100:00100000:2.0:1787739072.509002:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652531840 00000100:00100000:2.0:1787739072.509062:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23666 00000100:00100000:2.0:1787739072.509072:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+57:17726:x1874579652531840:12345-192.168.203.15@tcp:101:rm.0 00010000:00010000:2.0:1787739072.509092:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739072.509112:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1ae52f lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1470:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095471 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739072.509269:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1470:0x0]: returning size 4096 00010000:00010000:2.0:1787739072.509361:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1ae52f lrc: 1/0,0 mode: --/CR res: [0x200000402:0x1470:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095471 expref: 59 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.509376:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1470:0x0].0x0 (ffffa07403eeb600) 00010000:00010000:2.0:1787739072.509379:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739072.509384:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1ae52f lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1470:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095471 expref: 59 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.509396:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa0400, rc 0) 00000100:00100000:2.0:1787739072.512692:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ac3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+58:17726:x1874579652531840:12345-192.168.203.15@tcp:101:rm.0 Request processed in 3610us (3839us total) trans 4294983464 rc 301/301 00000100:00100000:2.0:1787739072.512706:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23666 00000100:00100000:1.0:1787739072.538958:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.540212:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652532096 00000100:00100000:2.0:1787739072.540225:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652532096 00000100:00100000:2.0:1787739072.540279:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5061 00000100:00100000:2.0:1787739072.540289:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356afc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+58:17726:x1874579652532096:12345-192.168.203.15@tcp:35:rm.0 00000100:00100000:3.0:1787739072.541007:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356afc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+59:17726:x1874579652532096:12345-192.168.203.15@tcp:35:rm.0 Request processed in 715us (2065us total) trans 4294983465 rc 0/0 00000100:00100000:3.0:1787739072.541018:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5061 00000100:00100000:3.0:1787739072.551209:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739072.551255:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739072.551475:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652532224 00000100:00100000:0.0:1787739072.551485:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652532224 00000100:00100000:0.0:1787739072.551555:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1378 00000100:00100000:0.0:1787739072.551566:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+44:2408:x1874579652532224:12345-192.168.203.15@tcp:103:rm.0 00010000:00010000:0.0:1787739072.551581:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739072.551618:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a400/0x7ee4453e4a1adff6 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1470:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952b1 expref: 44 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.551637:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1470:0x0].0x0 (ffffa07305ca2200) 00010000:00010000:0.0:1787739072.551647:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739072.551808:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+43:2408:x1874579652532224:12345-192.168.203.15@tcp:103:rm.0 Request processed in 244us (608us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.551817:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1378 00000100:00100000:2.0:1787739072.552633:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652532480 00000100:00100000:2.0:1787739072.552642:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652532480 00000100:00100000:2.0:1787739072.552687:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23667 00000100:00100000:2.0:1787739072.552696:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d85a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+59:17726:x1874579652532480:12345-192.168.203.15@tcp:36:rm.0 00010000:00010000:2.0:1787739072.552747:0:6523:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739072.552768:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633da00/0x7ee4453e4a1ae012 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1470:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952bf expref: 59 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.552784:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1470:0x0].0x0 (ffffa07403eeb600) 00010000:00010000:2.0:1787739072.552792:0:6523:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739072.552840:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae53d lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.552849:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae53d lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.552859:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae53d lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.552879:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a321c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665758208:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1787739072.552928:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.553035:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062340c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.553156:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665758208 00000100:00100000:1.0:1787739072.553162:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665758208 00000100:00100000:1.0:1787739072.553228:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3536 00000100:00100000:1.0:1787739072.553236:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+11:6523:x1874579665758208:12345-0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739072.553250:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739072.553264:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841a00/0x7ee4453e4a1ae544 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae53d expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.553287:0:6528:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1adef3 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095248 expref: 42 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.553298:0:6528:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841a00/0x7ee4453e4a1ae544 lrc: 4/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae53d expref: 12 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.553321:0:6528:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1adef3 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095248 expref: 42 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.553334:0:6528:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1adef3 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095248 expref: 42 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.553344:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae940 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001_UUID:6528:1874579665758336:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739072.553439:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288c00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.561409:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae940 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001_UUID:6528:1874579665758336:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739072.561472:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841a00/0x7ee4453e4a1ae544 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae53d expref: 12 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.561487:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:0.0:1787739072.561490:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739072.561494:0:6528:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841a00/0x7ee4453e4a1ae544 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae53d expref: 12 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.561506:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841a00, rc 0) 00000100:00100000:0.0:1787739072.561570:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+12:6523:x1874579665758208:12345-0@lo:101:mdt00_000.0 Request processed in 8334us (8641us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.561579:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3536 00000100:00100000:2.0:1787739072.562176:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a321c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665758208:0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739072.562199:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae53d lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae544 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.562210:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae53d lrc: 4/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae544 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.563149:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.563220:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652532608 00000100:00100000:2.0:1787739072.563228:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652532608 00010000:00010000:2.0:1787739072.563265:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1adef3 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095248 expref: 43 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.563309:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1577 00000100:00100000:2.0:1787739072.563320:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d86580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+43:2408:x1874579652532608:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739072.563336:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739072.563343:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1adef3 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095248 expref: 43 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.563372:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1adef3 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095248 expref: 43 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.563396:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609bc00/0x7ee4453e4a1adef3 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095248 expref: 43 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.563443:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:2.0:1787739072.563450:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841a00/0x7ee4453e4a1ae544 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae53d expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.563462:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841a00/0x7ee4453e4a1ae544 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae53d expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.563471:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841a00/0x7ee4453e4a1ae544 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae53d expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.563512:0:6508:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841a00/0x7ee4453e4a1ae544 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae53d expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.563528:0:6508:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a312c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001_UUID:6508:1874579665758464:0@lo:105: 00000100:00100000:2.0:1787739072.563585:0:6508:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.563648:0:6508:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062340c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739072.564980:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665758464 00000100:00100000:3.0:1787739072.564989:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665758464 00000100:00100000:3.0:1787739072.565051:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1204 00000100:00100000:3.0:1787739072.565061:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6508:x1874579665758464:12345-0@lo:105: 00010000:00010000:3.0:1787739072.565075:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae53d lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae544 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.565088:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae53d lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae544 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.565098:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae53d lrc: 5/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae544 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.565107:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae53d lrc: 5/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae544 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.565116:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae53d lrc: 6/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae544 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.565124:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae53d lrc: 6/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ae544 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.565151:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07306aa0400) 00010000:00010000:0.0:1787739072.565184:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae53d lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae544 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.565197:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae53d lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae544 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.565221:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6508:x1874579665758464:12345-0@lo:105: Request processed in 161us (1637us total) trans 0 rc 0/0 00000004:00080000:0.0:1787739072.565225:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:3.0:1787739072.565230:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1204 00010000:00010000:0.0:1787739072.565269:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089200/0x7ee4453e4a1ae54b lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.565277:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089200/0x7ee4453e4a1ae54b lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.565287:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089200/0x7ee4453e4a1ae54b lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.565306:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252a1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665758592:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1787739072.565321:0:6508:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a312c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001_UUID:6508:1874579665758464:0@lo:105: 00000100:00100000:0.0:1787739072.565346:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787739072.565347:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739072.565397:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d86580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+42:2408:x1874579652532608:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 2078us (2253us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.565405:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1577 00000100:00100000:0.0:1787739072.565556:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739072.566035:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665758592 00000100:00100000:3.0:1787739072.566043:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665758592 00000100:00100000:3.0:1787739072.566085:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3537 00000100:00100000:3.0:1787739072.566093:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+12:6523:x1874579665758592:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739072.566108:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739072.566126:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd000/0x7ee4453e4a1ae552 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae54b expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.566148:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd000/0x7ee4453e4a1ae552 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae54b expref: 13 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.566161:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd000/0x7ee4453e4a1ae552 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae54b expref: 13 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.566169:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x4a1b0d34 (ffffa0730608b600) 00010000:00010000:3.0:1787739072.566171:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739072.566175:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fd000, rc 0) 00000100:00100000:3.0:1787739072.566224:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+13:6523:x1874579665758592:12345-0@lo:101:mdt00_000.0 Request processed in 133us (879us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739072.566230:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3537 00000100:00100000:0.0:1787739072.567363:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252a1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665758592:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739072.567387:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089200/0x7ee4453e4a1ae54b lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae552 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.567401:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089200/0x7ee4453e4a1ae54b lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae552 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739072.567440:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:0.0:1787739072.567494:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252a1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665758720:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739072.567543:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739072.567577:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803000 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.567648:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665758720 00000100:00100000:1.0:1787739072.567654:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665758720 00000100:00100000:1.0:1787739072.567704:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3538 00000100:00100000:1.0:1787739072.567712:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f770c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+13:6523:x1874579665758720:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739072.567797:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f770c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+13:6523:x1874579665758720:12345-0@lo:1000:mdt00_000.0 Request processed in 85us (253us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.567803:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3538 00000100:00100000:0.0:1787739072.567903:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252a1c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665758720:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739072.567945:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f743c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665758848:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739072.567983:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739072.568002:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803000 going to sleep for 11 seconds 00000100:00100000:3.0:1787739072.568063:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665758848 00000100:00100000:3.0:1787739072.568068:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665758848 00000100:00100000:3.0:1787739072.568106:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3539 00000100:00100000:3.0:1787739072.568113:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+13:6523:x1874579665758848:12345-0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739072.568205:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+13:6523:x1874579665758848:12345-0@lo:1000:mdt00_000.0 Request processed in 92us (220us total) trans 0 rc 1/1 00000100:00100000:3.0:1787739072.568211:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3539 00000100:00100000:0.0:1787739072.568288:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f743c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665758848:0@lo:1000:mdt00_000.0 00010000:00010000:0.0:1787739072.568341:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088600/0x7ee4453e4a1ae559 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.568350:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088600/0x7ee4453e4a1ae559 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.568360:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088600/0x7ee4453e4a1ae559 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.568371:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f743c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665758976:0@lo:101:mdt00_000.0 00000100:00100000:0.0:1787739072.568396:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739072.568584:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665758976 00000100:00100000:1.0:1787739072.568589:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665758976 00000100:00100000:1.0:1787739072.568613:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3540 00000100:00100000:1.0:1787739072.568619:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+13:6523:x1874579665758976:12345-0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739072.568630:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739072.568645:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841400/0x7ee4453e4a1ae560 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae559 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.568669:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841400/0x7ee4453e4a1ae560 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae559 expref: 14 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.568684:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841400/0x7ee4453e4a1ae560 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae559 expref: 14 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.568692:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1470:0x0].0x0 (ffffa07305eae000) 00010000:00010000:1.0:1787739072.568695:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739072.568698:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841400, rc 0) 00000100:00100000:1.0:1787739072.568728:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+14:6523:x1874579665758976:12345-0@lo:101:mdt00_000.0 Request processed in 109us (331us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.568734:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3540 00000100:00100000:0.0:1787739072.568803:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.568817:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f743c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665758976:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739072.568828:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088600/0x7ee4453e4a1ae559 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae560 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.568839:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088600/0x7ee4453e4a1ae559 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae560 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.568863:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae567 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.568871:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae567 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.568881:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae567 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.568897:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae56e lrc: 3/0,1 mode: --/EX res: [0x200000400:0x73:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.568909:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ae020 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x73:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952c6 expref: 58 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.568918:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae56e lrc: 4/0,1 mode: --/EX res: [0x200000400:0x73:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.568938:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ae020 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x73:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952c6 expref: 58 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.568965:0:6523:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ae020 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x73:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952c6 expref: 58 pid: 6524 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.568976:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f743c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665759104:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739072.569142:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.576676:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f743c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665759104:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739072.576709:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae56e lrc: 3/0,1 mode: --/EX res: [0x200000400:0x73:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.580315:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739072.582674:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652532736 00000100:00100000:0.0:1787739072.582688:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652532736 00010000:00010000:0.0:1787739072.582733:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ae020 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x73:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952c6 expref: 59 pid: 6524 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.582778:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1578 00000100:00100000:0.0:1787739072.582786:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a312c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+59:2408:x1874579652532736:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739072.582799:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739072.582805:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ae020 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x73:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952c6 expref: 59 pid: 6524 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.582816:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ae020 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x73:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952c6 expref: 59 pid: 6524 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.582849:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609a000/0x7ee4453e4a1ae020 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x73:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952c6 expref: 59 pid: 6524 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.582866:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x73:0x0].0x0 (ffffa07305ca2800) 00010000:00010000:0.0:1787739072.582870:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae56e lrc: 3/0,1 mode: --/EX res: [0x200000400:0x73:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.582879:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae56e lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x73:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.582886:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae56e lrc: 4/0,1 mode: EX/EX res: [0x200000400:0x73:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.582920:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739072.582996:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a312c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+58:2408:x1874579652532736:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 209us (2688us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.583004:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1578 00010000:00010000:2.0:1787739072.583151:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae56e lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.583160:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae56e lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.583211:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0200/0x7ee4453e4a1ae575 lrc: 3/0,1 mode: --/EX res: [0x240000401:0x73:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.583220:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0200/0x7ee4453e4a1ae575 lrc: 3/0,1 mode: --/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.583230:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0200/0x7ee4453e4a1ae575 lrc: 3/0,1 mode: --/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.583251:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665759232:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1787739072.583307:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.583353:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062340c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739072.583515:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665759232 00000100:00100000:3.0:1787739072.583521:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665759232 00000100:00100000:3.0:1787739072.583567:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3541 00000100:00100000:3.0:1787739072.583575:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+14:6523:x1874579665759232:12345-0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739072.583593:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739072.583613:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae57c lrc: 2/0,0 mode: --/EX res: [0x240000401:0x73:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae575 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.583637:0:6529:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae02e lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x73:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952cd expref: 41 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.583646:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae57c lrc: 4/0,0 mode: --/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae575 expref: 15 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.583669:0:6529:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae02e lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x73:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952cd expref: 41 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.583687:0:6529:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae02e lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x73:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952cd expref: 41 pid: 6523 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.583703:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d86940 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665759360:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739072.584125:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e780 going to sleep for 11 seconds 00000100:00100000:2.0:1787739072.588390:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d86940 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665759360:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739072.588461:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae57c lrc: 3/0,0 mode: --/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae575 expref: 15 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.588476:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x73:0x0].0x0 (ffffa0730608b400) 00010000:00010000:2.0:1787739072.588478:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739072.588483:0:6529:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae57c lrc: 3/0,0 mode: --/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae575 expref: 15 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.588493:0:6529:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae57c lrc: 3/0,0 mode: --/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae575 expref: 15 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.588500:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abe00, rc 0) 00000100:00100000:2.0:1787739072.588556:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+15:6523:x1874579665759232:12345-0@lo:101:mdt00_000.0 Request processed in 4981us (5247us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.588565:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3541 00000100:00100000:0.0:1787739072.588682:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665759232:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739072.588701:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0200/0x7ee4453e4a1ae575 lrc: 4/0,1 mode: --/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae57c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.588711:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0200/0x7ee4453e4a1ae575 lrc: 4/0,1 mode: --/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae57c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.596785:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739072.601580:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652532864 00000100:00100000:1.0:1787739072.601604:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652532864 00010000:00010000:1.0:1787739072.601658:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae02e lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x73:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952cd expref: 42 pid: 6523 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.601693:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1579 00000100:00100000:1.0:1787739072.601703:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+42:2409:x1874579652532864:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739072.601719:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739072.601726:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae02e lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x73:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952cd expref: 42 pid: 6523 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.601744:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae02e lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x73:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952cd expref: 42 pid: 6523 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.601766:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae02e lrc: 0/0,0 mode: --/PR res: [0x240000401:0x73:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952cd expref: 42 pid: 6523 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.601784:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x73:0x0].0x0 (ffffa0730608b400) 00010000:00010000:1.0:1787739072.601789:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae57c lrc: 2/0,0 mode: --/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae575 expref: 14 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.601800:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae57c lrc: 2/0,0 mode: EX/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae575 expref: 14 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.601809:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae57c lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae575 expref: 14 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.601844:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae57c lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae575 expref: 14 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.601858:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356af840 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665759488:0@lo:105: 00000100:00100000:1.0:1787739072.601923:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739072.601983:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739072.602356:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665759488 00000100:00100000:2.0:1787739072.602361:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665759488 00000100:00100000:2.0:1787739072.602432:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1205 00000100:00100000:2.0:1787739072.602441:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ada40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665759488:12345-0@lo:105: 00010000:00010000:2.0:1787739072.602452:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0200/0x7ee4453e4a1ae575 lrc: 5/0,1 mode: --/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae57c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.602462:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0200/0x7ee4453e4a1ae575 lrc: 5/0,1 mode: --/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae57c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.602471:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0200/0x7ee4453e4a1ae575 lrc: 5/0,1 mode: --/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae57c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.602479:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0200/0x7ee4453e4a1ae575 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae57c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.602485:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0200/0x7ee4453e4a1ae575 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae57c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.602494:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0200/0x7ee4453e4a1ae575 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ae57c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.602511:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07306aa0200) 00000100:00100000:2.0:1787739072.602554:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ada40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665759488:12345-0@lo:105: Request processed in 115us (630us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.602561:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1205 00010000:00010000:0.0:1787739072.602696:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0200/0x7ee4453e4a1ae575 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae57c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.602707:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0200/0x7ee4453e4a1ae575 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae57c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.603472:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f743c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665759616:0@lo:403:mdt00_000.0 00000100:00100000:0.0:1787739072.603525:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739072.603548:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739072.603805:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665759616 00000100:00100000:2.0:1787739072.603812:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665759616 00000100:00100000:2.0:1787739072.603855:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3542 00000100:00100000:2.0:1787739072.603863:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304247c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+15:6523:x1874579665759616:12345-0@lo:403:mdt00_000.0 00000100:00100000:2.0:1787739072.605604:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304247c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+15:6523:x1874579665759616:12345-0@lo:403:mdt00_000.0 Request processed in 1736us (2072us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.605617:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3542 00000100:00100000:0.0:1787739072.605891:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f743c0 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665759616:0@lo:403:mdt00_000.0 00000100:00100000:1.0:1787739072.606172:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356af840 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665759488:0@lo:105: 00010000:00010000:1.0:1787739072.606188:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739072.606239:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+41:2409:x1874579652532864:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 4535us (9643us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.606252:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1579 00000040:00080000:0.0:1787739072.609009:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].393, 952 off595800 00000040:00080000:0.0:1787739072.609831:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].392, 1032 off645224 00000004:00080000:0.0:1787739072.609840:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07303803f00 with version 746 00000004:00080000:1.0:1787739072.610753:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 746 rpc_version 746 00000020:00100000:1.0:1787739072.610792:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739072.610802:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074356af840 x1874579665759872/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739072.610825:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356af840 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665759872:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739072.610893:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739072.610918:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739072.610990:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665759872 00000100:00100000:2.0:1787739072.610997:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665759872 00000100:00100000:2.0:1787739072.611061:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3543 00000100:00100000:2.0:1787739072.611071:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410b0a850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+15:7740:x1874579665759872:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739072.611402:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410b0a850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+16:7740:x1874579665759872:12345-0@lo:1000:osp_up1-0.0 Request processed in 332us (507us total) trans 4294968454 rc 0/0 00000100:00100000:2.0:1787739072.611410:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3543 00000100:00100000:1.0:1787739072.611778:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356af840 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665759872:0@lo:1000:osp_up1-0.0 00000004:00200000:0.0:1787739072.611835:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1470:0x0]: returning size 4096 00010000:00010000:0.0:1787739072.611861:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae56e lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.611871:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae56e lrc: 2/0,0 mode: EX/EX res: [0x200000400:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.611878:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae56e lrc: 3/0,0 mode: EX/EX res: [0x200000400:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.611888:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306088800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739072.611893:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae56e lrc: 3/0,0 mode: EX/EX res: [0x200000400:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.611900:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae56e lrc: 4/0,0 mode: EX/EX res: [0x200000400:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.611914:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x73:0x0].0x0 (ffffa07305ca2800) 00010000:00010000:0.0:1787739072.611917:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739072.611921:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae56e lrc: 2/0,0 mode: --/EX res: [0x200000400:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.611927:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae56e lrc: 0/0,0 mode: --/EX res: [0x200000400:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.611945:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0200/0x7ee4453e4a1ae575 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae57c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.611963:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0200/0x7ee4453e4a1ae575 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae57c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.611969:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0200/0x7ee4453e4a1ae575 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae57c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.611995:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739072.612005:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x73:0x0] 00010000:00010000:0.0:1787739072.612014:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae567 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.612020:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae567 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.612027:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae567 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.612031:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306089000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739072.612034:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae567 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.612048:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae567 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.612056:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1470:0x0].0x0 (ffffa07305ca2500) 00010000:00010000:0.0:1787739072.612059:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739072.612062:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae567 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.612067:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae567 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.612079:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088600/0x7ee4453e4a1ae559 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae560 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.612086:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088600/0x7ee4453e4a1ae559 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae560 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.612093:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088600/0x7ee4453e4a1ae559 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae560 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.612111:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:0.0:1787739072.612119:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739072.612124:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae53d lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae544 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.612130:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae53d lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae544 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.612136:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae53d lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae544 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.612147:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:0.0:1787739072.612150:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739072.612154:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089200/0x7ee4453e4a1ae54b lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae552 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.612161:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089200/0x7ee4453e4a1ae54b lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae552 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.612227:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d85a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+58:17726:x1874579652532480:12345-192.168.203.15@tcp:36:rm.0 Request processed in 59530us (60968us total) trans 4294983466 rc 0/0 00000100:00100000:0.0:1787739072.612235:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23667 00010000:00010000:1.0:1787739072.612696:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1787739072.612703:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0200/0x7ee4453e4a1ae575 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae57c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.612711:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739072.612716:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0200/0x7ee4453e4a1ae575 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae57c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739072.612724:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0200/0x7ee4453e4a1ae575 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae57c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.612753:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0200/0x7ee4453e4a1ae575 lrc: 2/0,0 mode: --/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae57c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.612759:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739072.612781:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0200/0x7ee4453e4a1ae575 lrc: 1/0,0 mode: --/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae57c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.612792:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0200/0x7ee4453e4a1ae575 lrc: 0/0,0 mode: --/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae57c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.612807:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1787739072.612810:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088600/0x7ee4453e4a1ae559 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae560 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.612816:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306088600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739072.612819:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088600/0x7ee4453e4a1ae559 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae560 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739072.612826:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088600/0x7ee4453e4a1ae559 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae560 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.612841:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088600/0x7ee4453e4a1ae559 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae560 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.612847:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739072.612853:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088600/0x7ee4453e4a1ae559 lrc: 1/0,0 mode: --/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae560 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.612860:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088600/0x7ee4453e4a1ae559 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae560 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.612871:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739072.612874:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae53d lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae544 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.612884:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739072.612887:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae53d lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae544 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739072.612894:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae53d lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae544 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.612909:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae53d lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae544 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.612915:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739072.612929:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae53d lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae544 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.612936:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae53d lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae544 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.612959:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739072.612961:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739072.612963:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739072.612977:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665760256:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739072.613022:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739072.613084:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665760256 00000100:00100000:1.0:1787739072.613091:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665760256 00000100:00100000:1.0:1787739072.613131:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1379 00000100:00100000:1.0:1787739072.613139:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ade00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+16:3650:x1874579665760256:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739072.613151:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739072.613173:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841a00/0x7ee4453e4a1ae544 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae53d expref: 16 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.613187:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:1.0:1787739072.613194:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739072.613224:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ade00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+15:3650:x1874579665760256:12345-0@lo:103:ldlm_bl.0 Request processed in 86us (202us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.613231:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1379 00000100:00100000:1.0:1787739072.613264:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ac000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665760256:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739072.613447:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665760128:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739072.613478:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739072.613514:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787739072.613520:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356af480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665760000:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739072.613541:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739072.613728:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665760128 00000100:00100000:1.0:1787739072.613733:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665760128 00000100:00100000:1.0:1787739072.613758:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665760000 00000100:00100000:1.0:1787739072.613761:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665760000 00000100:00100000:1.0:1787739072.613780:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1380 00000100:00100000:1.0:1787739072.613786:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba2c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+16:3653:x1874579665760128:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739072.613794:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739072.613806:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841400/0x7ee4453e4a1ae560 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1470:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae559 expref: 16 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.613817:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1470:0x0].0x0 (ffffa07305eae000) 00010000:00010000:1.0:1787739072.613823:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739072.613853:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba2c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+15:3653:x1874579665760128:12345-0@lo:103:ldlm_bl.0 Request processed in 68us (374us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.613859:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1380 00000100:00100000:1.0:1787739072.613893:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1381 00000100:00100000:1.0:1787739072.613899:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d85e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+14:3653:x1874579665760000:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739072.613907:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739072.613918:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae57c lrc: 0/0,0 mode: --/EX res: [0x240000401:0x73:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae575 expref: 14 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.613929:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x73:0x0].0x0 (ffffa0730608b400) 00010000:00010000:1.0:1787739072.613934:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739072.613968:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d85e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+13:3653:x1874579665760000:12345-0@lo:103:ldlm_bl.0 Request processed in 69us (426us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.613974:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1381 00000100:00100000:1.0:1787739072.614003:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356af480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665760000:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739072.614011:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665760128:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739072.614478:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739072.614489:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739072.623632:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739072.623782:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652533120 00000100:00100000:3.0:1787739072.623792:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652533120 00000100:00100000:3.0:1787739072.623851:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23668 00000100:00100000:3.0:1787739072.623862:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+58:17726:x1874579652533120:12345-192.168.203.15@tcp:101:rm.0 00010000:00010000:3.0:1787739072.623884:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739072.623904:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ae583 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x14cd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095478 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739072.624083:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14cd:0x0]: returning size 4096 00010000:00010000:3.0:1787739072.624180:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ae583 lrc: 1/0,0 mode: --/CR res: [0x200000402:0x14cd:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095478 expref: 60 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.624197:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14cd:0x0].0x0 (ffffa07403eeb800) 00010000:00010000:3.0:1787739072.624200:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739072.624205:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ae583 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x14cd:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095478 expref: 60 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.624220:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aac00, rc 0) 00000100:00100000:3.0:1787739072.625025:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+59:17726:x1874579652533120:12345-192.168.203.15@tcp:101:rm.0 Request processed in 1160us (1398us total) trans 4294983467 rc 301/301 00000100:00100000:3.0:1787739072.625034:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23668 00000100:00100000:3.0:1787739072.636382:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739072.636522:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652533376 00000100:00100000:1.0:1787739072.636532:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652533376 00000100:00100000:1.0:1787739072.636580:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5062 00000100:00100000:1.0:1787739072.636590:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d861c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+59:17726:x1874579652533376:12345-192.168.203.15@tcp:35:rm.0 00000100:00100000:1.0:1787739072.636812:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d861c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:17726:x1874579652533376:12345-192.168.203.15@tcp:35:rm.0 Request processed in 223us (437us total) trans 4294983468 rc 0/0 00000100:00100000:1.0:1787739072.636821:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5062 00000100:00100000:0.0:1787739072.649961:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739072.650033:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652533504 00000100:00100000:1.0:1787739072.650115:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652533504 00000100:00100000:1.0:1787739072.650233:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1382 00000100:00100000:1.0:1787739072.650243:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304245e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+41:2407:x1874579652533504:12345-192.168.203.15@tcp:103:rm.0 00010000:00010000:1.0:1787739072.650260:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739072.650284:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ca00/0x7ee4453e4a1ae17e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14cd:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095352 expref: 41 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.650305:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14cd:0x0].0x0 (ffffa0730713a900) 00010000:00010000:1.0:1787739072.650316:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739072.650374:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304245e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+40:2407:x1874579652533504:12345-192.168.203.15@tcp:103:rm.0 Request processed in 132us (431us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.650382:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1382 00000100:00100000:2.0:1787739072.660637:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739072.660716:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652533760 00000100:00100000:1.0:1787739072.660726:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652533760 00000100:00100000:1.0:1787739072.660786:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23669 00000100:00100000:1.0:1787739072.660797:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:17726:x1874579652533760:12345-192.168.203.15@tcp:36:rm.0 00010000:00010000:1.0:1787739072.660858:0:6524:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739072.660884:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d400/0x7ee4453e4a1ae19a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14cd:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095360 expref: 60 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.660907:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14cd:0x0].0x0 (ffffa07403eeb800) 00010000:00010000:1.0:1787739072.660917:0:6524:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739072.660981:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841400/0x7ee4453e4a1ae591 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.660990:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841400/0x7ee4453e4a1ae591 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.660999:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841400/0x7ee4453e4a1ae591 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.661019:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665760384:0@lo:101:mdt00_001.0 00000100:00100000:1.0:1787739072.661118:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739072.661152:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739072.661170:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665760384 00000100:00100000:3.0:1787739072.661176:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665760384 00000100:00100000:3.0:1787739072.661221:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3544 00000100:00100000:3.0:1787739072.661229:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+13:6524:x1874579665760384:12345-0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787739072.661241:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739072.661255:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae598 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae591 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.661278:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae598 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae591 expref: 14 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.661291:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae598 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae591 expref: 14 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.661310:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b900) 00010000:00010000:3.0:1787739072.661313:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739072.661316:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab600, rc 0) 00000100:00100000:3.0:1787739072.661613:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+14:6524:x1874579665760384:12345-0@lo:101:mdt00_001.0 Request processed in 384us (493us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739072.661620:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3544 00000100:00100000:1.0:1787739072.662682:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665760384:0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739072.662707:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841400/0x7ee4453e4a1ae591 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae598 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.662721:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841400/0x7ee4453e4a1ae591 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae598 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739072.662744:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739072.662783:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841a00/0x7ee4453e4a1ae59f lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.662791:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841a00/0x7ee4453e4a1ae59f lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.662799:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841a00/0x7ee4453e4a1ae59f lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.662815:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665760512:0@lo:101:mdt00_001.0 00000100:00100000:1.0:1787739072.662860:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739072.662892:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739072.666095:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665760512 00000100:00100000:3.0:1787739072.666112:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665760512 00000100:00100000:3.0:1787739072.666178:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3545 00000100:00100000:3.0:1787739072.666188:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+14:6524:x1874579665760512:12345-0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787739072.666208:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739072.666231:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae5a6 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae59f expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.666262:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae5a6 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae59f expref: 15 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.666277:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae5a6 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae59f expref: 15 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.666285:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x32e1d295 (ffffa0730608b300) 00010000:00010000:3.0:1787739072.666288:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739072.666291:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739072.666401:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+15:6524:x1874579665760512:12345-0@lo:101:mdt00_001.0 Request processed in 214us (3540us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739072.666409:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3545 00000100:00100000:1.0:1787739072.666684:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665760512:0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739072.666706:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841a00/0x7ee4453e4a1ae59f lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae5a6 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.666718:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841a00/0x7ee4453e4a1ae59f lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae5a6 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739072.666741:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:1.0:1787739072.666792:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665760640:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739072.666832:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739072.666870:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288540 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.666930:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665760640 00000100:00100000:0.0:1787739072.666936:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665760640 00000100:00100000:0.0:1787739072.666986:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3546 00000100:00100000:0.0:1787739072.666994:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+15:6524:x1874579665760640:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739072.667197:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ade00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+15:6524:x1874579665760640:12345-0@lo:1000:mdt00_001.0 Request processed in 204us (364us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.667205:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3546 00000100:00100000:1.0:1787739072.667269:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665760640:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739072.667307:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665760768:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739072.667332:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739072.667354:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288540 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.667400:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665760768 00000100:00100000:0.0:1787739072.667436:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665760768 00000100:00100000:0.0:1787739072.667464:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3547 00000100:00100000:0.0:1787739072.667471:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+15:6524:x1874579665760768:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739072.667545:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+15:6524:x1874579665760768:12345-0@lo:1000:mdt00_001.0 Request processed in 74us (212us total) trans 0 rc 1/1 00000100:00100000:0.0:1787739072.667551:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3547 00000100:00100000:1.0:1787739072.667732:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665760768:0@lo:1000:mdt00_001.0 00010000:00010000:1.0:1787739072.667783:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841c00/0x7ee4453e4a1ae5ad lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.667792:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841c00/0x7ee4453e4a1ae5ad lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.667802:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841c00/0x7ee4453e4a1ae5ad lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.667814:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665760896:0@lo:101:mdt00_001.0 00000100:00100000:1.0:1787739072.667838:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739072.667869:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.667882:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665760896 00000100:00100000:1.0:1787739072.667886:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665760896 00000100:00100000:1.0:1787739072.667907:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3548 00000100:00100000:1.0:1787739072.667912:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356aed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+15:6524:x1874579665760896:12345-0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739072.667921:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739072.667935:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840200/0x7ee4453e4a1ae5b4 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae5ad expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.667968:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840200/0x7ee4453e4a1ae5b4 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae5ad expref: 16 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.667980:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840200/0x7ee4453e4a1ae5b4 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae5ad expref: 16 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.667989:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14cd:0x0].0x0 (ffffa07305eaed00) 00010000:00010000:1.0:1787739072.667992:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739072.667995:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306840200, rc 0) 00000100:00100000:1.0:1787739072.668026:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356aed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+16:6524:x1874579665760896:12345-0@lo:101:mdt00_001.0 Request processed in 113us (186us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.668032:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3548 00000100:00100000:1.0:1787739072.668198:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665760896:0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739072.668210:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841c00/0x7ee4453e4a1ae5ad lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae5b4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.668219:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841c00/0x7ee4453e4a1ae5ad lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae5b4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.668240:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1ae5bb lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.668248:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1ae5bb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.668255:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1ae5bb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.668272:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae5c2 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xa3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.668283:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ae1a8 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095367 expref: 59 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.668297:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae5c2 lrc: 4/0,1 mode: --/EX res: [0x200000400:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.668316:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ae1a8 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095367 expref: 59 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.668329:0:6524:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ae1a8 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095367 expref: 59 pid: 6525 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.668340:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665761024:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739072.668393:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.678267:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665761024:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739072.678311:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae5c2 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.683636:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739072.683926:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652533888 00000100:00100000:0.0:1787739072.683936:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652533888 00010000:00010000:0.0:1787739072.683987:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ae1a8 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095367 expref: 60 pid: 6525 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.684033:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1580 00000100:00100000:0.0:1787739072.684045:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:2409:x1874579652533888:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739072.684069:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739072.684076:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ae1a8 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095367 expref: 60 pid: 6525 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.684089:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ae1a8 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095367 expref: 60 pid: 6525 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.684112:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c200/0x7ee4453e4a1ae1a8 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095367 expref: 60 pid: 6525 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.684129:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xa3:0x0].0x0 (ffffa0730713a600) 00010000:00010000:0.0:1787739072.684134:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae5c2 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.684148:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae5c2 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.684156:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae5c2 lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.684187:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739072.684243:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+59:2409:x1874579652533888:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 201us (615us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.684251:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1580 00010000:00010000:1.0:1787739072.684437:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae5c2 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.684447:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae5c2 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.684505:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840400/0x7ee4453e4a1ae5c9 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.684513:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840400/0x7ee4453e4a1ae5c9 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.684523:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840400/0x7ee4453e4a1ae5c9 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.684544:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665761152:0@lo:101:mdt00_001.0 00000100:00100000:1.0:1787739072.684605:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739072.684634:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.686682:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665761152 00000100:00100000:0.0:1787739072.686697:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665761152 00000100:00100000:0.0:1787739072.686771:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3549 00000100:00100000:0.0:1787739072.686782:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+16:6524:x1874579665761152:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739072.686800:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739072.686820:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae5d0 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae5c9 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.686850:0:6528:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae1b6 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09536e expref: 39 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.686861:0:6528:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae5d0 lrc: 4/0,0 mode: --/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae5c9 expref: 17 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.686892:0:6528:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae1b6 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09536e expref: 39 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.686905:0:6528:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae1b6 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09536e expref: 39 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.686921:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042452c0 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001_UUID:6528:1874579665761280:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739072.687018:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46c0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.694854:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042452c0 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001_UUID:6528:1874579665761280:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739072.694893:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae5d0 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae5c9 expref: 17 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.694906:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xa3:0x0].0x0 (ffffa0730713a300) 00010000:00010000:0.0:1787739072.694909:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739072.694913:0:6528:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae5d0 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae5c9 expref: 17 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.694922:0:6528:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae5d0 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae5c9 expref: 17 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.694929:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306088a00, rc 0) 00000100:00100000:0.0:1787739072.695066:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+17:6524:x1874579665761152:12345-0@lo:101:mdt00_001.0 Request processed in 8286us (10460us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.695075:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3549 00000100:00100000:1.0:1787739072.697835:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665761152:0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739072.697865:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840400/0x7ee4453e4a1ae5c9 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae5d0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.697878:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840400/0x7ee4453e4a1ae5c9 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae5d0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.698898:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739072.698997:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652534016 00000100:00100000:1.0:1787739072.699005:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652534016 00010000:00010000:1.0:1787739072.699042:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae1b6 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09536e expref: 40 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.699158:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1581 00000100:00100000:1.0:1787739072.699168:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+40:2409:x1874579652534016:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739072.699183:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739072.699190:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae1b6 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09536e expref: 40 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.699202:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae1b6 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09536e expref: 40 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.699225:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1ce00/0x7ee4453e4a1ae1b6 lrc: 0/0,0 mode: --/PR res: [0x240000401:0xa3:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09536e expref: 40 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.699246:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xa3:0x0].0x0 (ffffa0730713a300) 00010000:00010000:1.0:1787739072.699252:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae5d0 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae5c9 expref: 16 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.699263:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae5d0 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae5c9 expref: 16 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.699272:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae5d0 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae5c9 expref: 16 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.699305:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae5d0 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae5c9 expref: 16 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.699319:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ad2c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665761408:0@lo:105: 00000100:00100000:1.0:1787739072.699742:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739072.699783:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.700166:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665761408 00000100:00100000:0.0:1787739072.700172:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665761408 00000100:00100000:0.0:1787739072.700220:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1206 00000100:00100000:0.0:1787739072.700229:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ae940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665761408:12345-0@lo:105: 00010000:00010000:0.0:1787739072.700241:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840400/0x7ee4453e4a1ae5c9 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae5d0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.700256:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840400/0x7ee4453e4a1ae5c9 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae5d0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.700265:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840400/0x7ee4453e4a1ae5c9 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae5d0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.700273:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840400/0x7ee4453e4a1ae5c9 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae5d0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.700280:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840400/0x7ee4453e4a1ae5c9 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae5d0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.700288:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840400/0x7ee4453e4a1ae5c9 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ae5d0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.700495:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07306840400) 00000100:00100000:0.0:1787739072.700559:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ae940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665761408:12345-0@lo:105: Request processed in 330us (816us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.700566:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1206 00010000:00010000:1.0:1787739072.701049:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840400/0x7ee4453e4a1ae5c9 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae5d0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.701133:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840400/0x7ee4453e4a1ae5c9 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae5d0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.701324:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ad2c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665761408:0@lo:105: 00010000:00010000:3.0:1787739072.701340:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739072.701396:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+39:2409:x1874579652534016:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 2228us (2501us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739072.701406:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1581 00000100:00100000:1.0:1787739072.702148:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665761536:0@lo:403:mdt00_001.0 00000100:00100000:1.0:1787739072.702203:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739072.703118:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288540 going to sleep for 11 seconds 00000100:00100000:3.0:1787739072.703204:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665761536 00000100:00100000:3.0:1787739072.703212:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665761536 00000100:00100000:3.0:1787739072.703267:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3550 00000100:00100000:3.0:1787739072.703277:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+17:6524:x1874579665761536:12345-0@lo:403:mdt00_001.0 00000100:00100000:3.0:1787739072.705446:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+17:6524:x1874579665761536:12345-0@lo:403:mdt00_001.0 Request processed in 2133us (3205us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739072.705459:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3550 00000100:00100000:1.0:1787739072.706265:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665761536:0@lo:403:mdt00_001.0 00000040:00080000:1.0:1787739072.707794:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].394, 952 off596752 00000040:00080000:1.0:1787739072.707983:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].393, 1032 off646256 00000004:00080000:1.0:1787739072.707989:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306288540 with version 747 00000004:00080000:0.0:1787739072.708176:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 747 rpc_version 747 00000020:00100000:0.0:1787739072.708214:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739072.708224:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073042452c0 x1874579665761792/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739072.708246:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042452c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665761792:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739072.708391:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739072.708449:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.708649:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665761792 00000100:00100000:1.0:1787739072.708658:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665761792 00000100:00100000:1.0:1787739072.708707:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3551 00000100:00100000:1.0:1787739072.708715:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042470c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+17:7740:x1874579665761792:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739072.709993:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042470c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+18:7740:x1874579665761792:12345-0@lo:1000:osp_up1-0.0 Request processed in 1274us (1599us total) trans 4294968455 rc 0/0 00000100:00100000:1.0:1787739072.710006:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3551 00000100:00100000:3.0:1787739072.710240:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042452c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665761792:0@lo:1000:osp_up1-0.0 00000004:00200000:1.0:1787739072.710362:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14cd:0x0]: returning size 4096 00010000:00010000:1.0:1787739072.710386:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae5c2 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.710395:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae5c2 lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.710403:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae5c2 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.710409:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840c00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739072.710529:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae5c2 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.710540:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae5c2 lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.710556:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xa3:0x0].0x0 (ffffa0730713a600) 00010000:00010000:1.0:1787739072.710559:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739072.710564:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae5c2 lrc: 2/0,0 mode: --/EX res: [0x200000400:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.710571:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae5c2 lrc: 0/0,0 mode: --/EX res: [0x200000400:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.710799:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840400/0x7ee4453e4a1ae5c9 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae5d0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.710808:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840400/0x7ee4453e4a1ae5c9 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae5d0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.710816:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840400/0x7ee4453e4a1ae5c9 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae5d0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.710839:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739072.710848:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xa3:0x0] 00010000:00010000:1.0:1787739072.710858:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1ae5bb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.710865:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1ae5bb lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.710870:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1ae5bb lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.710875:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840e00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739072.710878:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1ae5bb lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.710884:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1ae5bb lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.710897:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14cd:0x0].0x0 (ffffa07305eae600) 00010000:00010000:1.0:1787739072.710899:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739072.710902:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1ae5bb lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.710908:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1ae5bb lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.710918:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841c00/0x7ee4453e4a1ae5ad lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae5b4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.710925:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841c00/0x7ee4453e4a1ae5ad lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae5b4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.710932:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841c00/0x7ee4453e4a1ae5ad lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae5b4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.711017:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:1.0:1787739072.711029:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739072.711035:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841400/0x7ee4453e4a1ae591 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae598 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.711043:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841400/0x7ee4453e4a1ae591 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae598 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.711051:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841400/0x7ee4453e4a1ae591 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae598 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.711153:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1787739072.711160:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840400/0x7ee4453e4a1ae5c9 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae5d0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.711167:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739072.711172:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840400/0x7ee4453e4a1ae5c9 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae5d0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739072.711179:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840400/0x7ee4453e4a1ae5c9 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae5d0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.711210:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840400/0x7ee4453e4a1ae5c9 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae5d0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.711217:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739072.711248:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840400/0x7ee4453e4a1ae5c9 lrc: 1/0,0 mode: --/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae5d0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.711255:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840400/0x7ee4453e4a1ae5c9 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae5d0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.711269:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787739072.711272:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841c00/0x7ee4453e4a1ae5ad lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae5b4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.711278:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306841c00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739072.711281:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841c00/0x7ee4453e4a1ae5ad lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae5b4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739072.711288:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841c00/0x7ee4453e4a1ae5ad lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae5b4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.711315:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841c00/0x7ee4453e4a1ae5ad lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae5b4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.711321:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739072.711336:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841c00/0x7ee4453e4a1ae5ad lrc: 1/0,0 mode: --/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae5b4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.711344:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841c00/0x7ee4453e4a1ae5ad lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae5b4 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.711355:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739072.711358:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841400/0x7ee4453e4a1ae591 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae598 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.711363:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306841400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739072.711366:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841400/0x7ee4453e4a1ae591 lrc: 5/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae598 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739072.711372:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841400/0x7ee4453e4a1ae591 lrc: 5/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae598 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.711392:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841400/0x7ee4453e4a1ae591 lrc: 4/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae598 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.711398:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739072.711403:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841400/0x7ee4453e4a1ae591 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae598 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.711410:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739072.711437:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739072.711439:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739072.711447:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787739072.711461:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d86580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665761920:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739072.711524:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739072.711589:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d870c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665762176:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739072.711613:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739072.711641:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787739072.711648:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d87c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665762048:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739072.711665:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665761920 00000100:00100000:2.0:1787739072.711673:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665761920 00000100:00100000:3.0:1787739072.711675:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.711732:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665762176 00000100:00100000:2.0:1787739072.711736:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665762176 00010000:00010000:0.0:1787739072.711751:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739072.711765:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665762048 00000100:00100000:2.0:1787739072.711769:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665762048 00000100:00100000:0.0:1787739072.711789:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1383 00000100:00100000:0.0:1787739072.711841:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d852c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+20:3650:x1874579665761920:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739072.711855:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739072.711873:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae5d0 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xa3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae5c9 expref: 20 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.711886:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xa3:0x0].0x0 (ffffa0730713a300) 00010000:00010000:0.0:1787739072.711894:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739072.711929:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d852c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+19:3650:x1874579665761920:12345-0@lo:103:ldlm_bl.0 Request processed in 132us (404us total) trans 0 rc 0/0 00010000:00010000:1.0:1787739072.712314:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:1.0:1787739072.712322:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841400/0x7ee4453e4a1ae591 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae598 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739072.712339:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739072.712347:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841a00/0x7ee4453e4a1ae59f lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae5a6 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.712355:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841a00/0x7ee4453e4a1ae59f lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae5a6 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.712439:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+59:17726:x1874579652533760:12345-192.168.203.15@tcp:36:rm.0 Request processed in 51642us (51812us total) trans 4294983469 rc 0/0 00000100:00100000:1.0:1787739072.712447:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23669 00000100:00100000:1.0:1787739072.713639:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d86580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665761920:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739072.718214:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1384 00000100:00100000:2.0:1787739072.718241:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+19:3653:x1874579665762176:12345-0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739072.718244:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1385 00000100:00100000:3.0:1787739072.718260:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba1450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+19:3653:x1874579665762048:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739072.718264:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739072.718272:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739072.718298:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae598 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae591 expref: 19 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.718309:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840200/0x7ee4453e4a1ae5b4 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14cd:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae5ad expref: 19 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.718321:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b900) 00010000:00010000:3.0:1787739072.718337:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14cd:0x0].0x0 (ffffa07305eaed00) 00010000:00010000:2.0:1787739072.718337:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739072.718350:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739072.718499:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba1450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+17:3653:x1874579665762048:12345-0@lo:103:ldlm_bl.0 Request processed in 243us (6823us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739072.718508:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1385 00000100:00100000:0.0:1787739072.718868:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1383 00010000:00010000:3.0:1787739072.719654:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739072.726386:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+15:3653:x1874579665762176:12345-0@lo:103:ldlm_bl.0 Request processed in 8148us (14761us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.726400:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1384 00000100:00100000:1.0:1787739072.726767:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d87c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665762048:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739072.726798:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d870c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665762176:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739072.742945:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739072.743358:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652534272 00000100:00100000:0.0:1787739072.743368:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652534272 00000100:00100000:0.0:1787739072.743465:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23670 00000100:00100000:0.0:1787739072.743485:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d87840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+59:17726:x1874579652534272:12345-192.168.203.15@tcp:101:rm.0 00010000:00010000:0.0:1787739072.743508:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739072.743529:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae5d7 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x14c0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09547f expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739072.743673:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14c0:0x0]: returning size 4096 00010000:00010000:0.0:1787739072.744041:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae5d7 lrc: 1/0,0 mode: --/CR res: [0x200000402:0x14c0:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09547f expref: 61 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.744067:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14c0:0x0].0x0 (ffffa0730713aa00) 00010000:00010000:0.0:1787739072.744070:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739072.744076:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae5d7 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x14c0:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09547f expref: 61 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.744094:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306088a00, rc 0) 00000100:00100000:0.0:1787739072.744157:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d87840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:17726:x1874579652534272:12345-192.168.203.15@tcp:101:rm.0 Request processed in 675us (1224us total) trans 4294983470 rc 301/301 00000100:00100000:0.0:1787739072.744165:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23670 00000100:00100000:2.0:1787739072.770671:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739072.771196:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652534528 00000100:00100000:0.0:1787739072.771206:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652534528 00000100:00100000:0.0:1787739072.771261:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5063 00000100:00100000:0.0:1787739072.771271:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:17726:x1874579652534528:12345-192.168.203.15@tcp:35:rm.0 00000100:00100000:0.0:1787739072.771508:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:17726:x1874579652534528:12345-192.168.203.15@tcp:35:rm.0 Request processed in 239us (848us total) trans 4294983471 rc 0/0 00000100:00100000:0.0:1787739072.771516:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5063 00000100:00100000:3.0:1787739072.776150:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739072.776318:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652534656 00000100:00100000:0.0:1787739072.776334:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652534656 00000100:00100000:0.0:1787739072.776406:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1386 00000100:00100000:0.0:1787739072.776450:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d87480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+39:2407:x1874579652534656:12345-192.168.203.15@tcp:103:rm.0 00010000:00010000:0.0:1787739072.776465:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739072.776489:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ae0e4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14c0:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095313 expref: 39 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.776508:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14c0:0x0].0x0 (ffffa0730608bf00) 00010000:00010000:0.0:1787739072.776518:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739072.776567:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d87480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+38:2407:x1874579652534656:12345-192.168.203.15@tcp:103:rm.0 Request processed in 152us (429us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.776575:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1386 00000100:00100000:3.0:1787739072.787476:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.788223:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652534912 00000100:00100000:2.0:1787739072.788234:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652534912 00000100:00100000:2.0:1787739072.788299:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23671 00000100:00100000:2.0:1787739072.788310:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d85680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:17726:x1874579652534912:12345-192.168.203.15@tcp:36:rm.0 00010000:00010000:2.0:1787739072.788367:0:7944:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739072.788395:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c800/0x7ee4453e4a1ae100 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14c0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095321 expref: 61 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.788439:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14c0:0x0].0x0 (ffffa0730713aa00) 00010000:00010000:2.0:1787739072.788451:0:7944:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739072.788507:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae5e5 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.788516:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae5e5 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.788526:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae5e5 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.788548:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665762304:0@lo:101:mdt00_003.0 00000100:00100000:2.0:1787739072.788600:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.788653:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.788820:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665762304 00000100:00100000:0.0:1787739072.788825:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665762304 00000100:00100000:0.0:1787739072.788866:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3552 00000100:00100000:0.0:1787739072.788874:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+15:7944:x1874579665762304:12345-0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739072.788886:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739072.788901:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae5ec lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae5e5 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.788925:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae5ec lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae5e5 expref: 16 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.788938:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae5ec lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae5e5 expref: 16 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.788945:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305ca2500) 00010000:00010000:0.0:1787739072.788960:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739072.788963:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306088a00, rc 0) 00000100:00100000:0.0:1787739072.789012:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+16:7944:x1874579665762304:12345-0@lo:101:mdt00_003.0 Request processed in 139us (411us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.789019:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3552 00000100:00100000:2.0:1787739072.789486:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665762304:0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739072.789503:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae5e5 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae5ec expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.789513:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae5e5 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae5ec expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739072.789533:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:2.0:1787739072.789560:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0600/0x7ee4453e4a1ae5f3 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x44749f58 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.789566:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0600/0x7ee4453e4a1ae5f3 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x44749f58 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.789573:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0600/0x7ee4453e4a1ae5f3 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x44749f58 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.789586:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665762432:0@lo:101:mdt00_003.0 00000100:00100000:2.0:1787739072.789614:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.789654:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.789686:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665762432 00000100:00100000:1.0:1787739072.789693:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665762432 00000100:00100000:1.0:1787739072.789739:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3553 00000100:00100000:1.0:1787739072.789747:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a312c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+16:7944:x1874579665762432:12345-0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739072.789757:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739072.789770:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841400/0x7ee4453e4a1ae5fa lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x44749f58 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae5f3 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.789787:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841400/0x7ee4453e4a1ae5fa lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x44749f58 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae5f3 expref: 17 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.789806:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841400/0x7ee4453e4a1ae5fa lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x44749f58 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae5f3 expref: 17 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.789813:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x44749f58 (ffffa0730713a600) 00010000:00010000:1.0:1787739072.789816:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739072.789820:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841400, rc 0) 00000100:00100000:1.0:1787739072.789864:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a312c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+17:7944:x1874579665762432:12345-0@lo:101:mdt00_003.0 Request processed in 118us (248us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.789870:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3553 00000100:00100000:2.0:1787739072.792053:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665762432:0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739072.792083:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0600/0x7ee4453e4a1ae5f3 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x44749f58 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae5fa expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.792097:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0600/0x7ee4453e4a1ae5f3 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x44749f58 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae5fa expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739072.792118:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:2.0:1787739072.792169:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665762560:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739072.792220:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.792258:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410404240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.792440:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665762560 00000100:00100000:0.0:1787739072.792447:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665762560 00000100:00100000:0.0:1787739072.792504:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3554 00000100:00100000:0.0:1787739072.792514:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+17:7944:x1874579665762560:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1787739072.792593:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+17:7944:x1874579665762560:12345-0@lo:1000:mdt00_003.0 Request processed in 80us (372us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.792601:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3554 00000100:00100000:3.0:1787739072.792687:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665762560:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739072.792731:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d86940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665762688:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1787739072.792761:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739072.792802:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739072.792929:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665762688 00000100:00100000:2.0:1787739072.792934:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665762688 00000100:00100000:2.0:1787739072.792979:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3555 00000100:00100000:2.0:1787739072.792985:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+17:7944:x1874579665762688:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1787739072.793071:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+17:7944:x1874579665762688:12345-0@lo:1000:mdt00_003.0 Request processed in 86us (309us total) trans 0 rc 1/1 00000100:00100000:2.0:1787739072.793080:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3555 00000100:00100000:3.0:1787739072.793147:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d86940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665762688:0@lo:1000:mdt00_003.0 00010000:00010000:3.0:1787739072.793200:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae601 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.793208:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae601 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.793217:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae601 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.793228:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d86940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665762816:0@lo:101:mdt00_003.0 00000100:00100000:3.0:1787739072.793250:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739072.793276:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.793706:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665762816 00000100:00100000:0.0:1787739072.793712:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665762816 00000100:00100000:0.0:1787739072.793746:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3556 00000100:00100000:0.0:1787739072.793753:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d843c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+17:7944:x1874579665762816:12345-0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739072.793763:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739072.793778:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae608 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae601 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.793801:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae608 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae601 expref: 18 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.793814:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae608 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae601 expref: 18 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.793822:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14c0:0x0].0x0 (ffffa07305ca2800) 00010000:00010000:0.0:1787739072.793825:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739072.793828:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306088e00, rc 0) 00000100:00100000:0.0:1787739072.793867:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d843c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+18:7944:x1874579665762816:12345-0@lo:101:mdt00_003.0 Request processed in 114us (615us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.793872:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3556 00000100:00100000:0.0:1787739072.793904:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d86940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665762816:0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739072.793917:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae601 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae608 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.793927:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae601 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae608 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.793964:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1ae60f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.793976:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1ae60f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.793984:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1ae60f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.794000:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae616 lrc: 3/0,1 mode: --/EX res: [0x200000400:0x9d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.794012:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ae10e lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095328 expref: 60 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.794020:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae616 lrc: 4/0,1 mode: --/EX res: [0x200000400:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.794044:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ae10e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095328 expref: 60 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.794057:0:7944:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ae10e lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095328 expref: 60 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.794068:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304245e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665762944:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739072.794130:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a46c0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.800116:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739072.800222:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304245e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665762944:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739072.800249:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae616 lrc: 3/0,1 mode: --/EX res: [0x200000400:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.800323:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652535040 00000100:00100000:2.0:1787739072.800330:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652535040 00010000:00010000:2.0:1787739072.800365:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ae10e lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095328 expref: 61 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.800403:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1582 00000100:00100000:2.0:1787739072.800442:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304247c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:2408:x1874579652535040:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739072.800458:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739072.800465:0:6510:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ae10e lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095328 expref: 61 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.800477:0:6510:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ae10e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095328 expref: 61 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.800500:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1d800/0x7ee4453e4a1ae10e lrc: 0/0,0 mode: --/PR res: [0x200000400:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095328 expref: 61 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.800516:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x9d:0x0].0x0 (ffffa0730713a100) 00010000:00010000:2.0:1787739072.800521:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae616 lrc: 3/0,1 mode: --/EX res: [0x200000400:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.800530:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae616 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.800537:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae616 lrc: 4/0,1 mode: EX/EX res: [0x200000400:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.800570:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739072.800626:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304247c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:2408:x1874579652535040:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 214us (520us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.800797:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1582 00010000:00010000:3.0:1787739072.800858:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae616 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.800866:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae616 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.800913:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ae61d lrc: 3/0,1 mode: --/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.800920:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ae61d lrc: 3/0,1 mode: --/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.800929:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ae61d lrc: 3/0,1 mode: --/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.801009:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d86940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665763072:0@lo:101:mdt00_003.0 00000100:00100000:3.0:1787739072.801063:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739072.801112:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.801118:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665763072 00000100:00100000:1.0:1787739072.801124:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665763072 00000100:00100000:1.0:1787739072.801168:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3557 00000100:00100000:1.0:1787739072.801176:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d870c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+18:7944:x1874579665763072:12345-0@lo:101:mdt00_003.0 00010000:00010000:1.0:1787739072.801190:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739072.801203:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae624 lrc: 2/0,0 mode: --/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae61d expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.801225:0:6529:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ae11c lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09532f expref: 37 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.801234:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae624 lrc: 4/0,0 mode: --/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae61d expref: 19 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.801257:0:6529:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ae11c lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09532f expref: 37 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.801275:0:6529:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ae11c lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09532f expref: 37 pid: 6524 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.801285:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665763200:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739072.801347:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.808245:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665763200:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739072.808284:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae624 lrc: 3/0,0 mode: --/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae61d expref: 19 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.808298:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x9d:0x0].0x0 (ffffa07305ca2b00) 00010000:00010000:1.0:1787739072.808301:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739072.808305:0:6529:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae624 lrc: 3/0,0 mode: --/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae61d expref: 19 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.808317:0:6529:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae624 lrc: 3/0,0 mode: --/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae61d expref: 19 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.808336:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306840c00, rc 0) 00000100:00100000:1.0:1787739072.808399:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d870c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+19:7944:x1874579665763072:12345-0@lo:101:mdt00_003.0 Request processed in 7223us (7345us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.808409:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3557 00000100:00100000:3.0:1787739072.808630:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d86940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665763072:0@lo:101:mdt00_003.0 00010000:00010000:3.0:1787739072.808649:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ae61d lrc: 4/0,1 mode: --/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae624 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.808660:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ae61d lrc: 4/0,1 mode: --/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae624 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.810823:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.811043:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652535168 00000100:00100000:2.0:1787739072.811061:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652535168 00010000:00010000:2.0:1787739072.811097:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ae11c lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09532f expref: 38 pid: 6524 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.811135:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1583 00000100:00100000:2.0:1787739072.811144:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ae580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+38:2409:x1874579652535168:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739072.811159:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739072.811165:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ae11c lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09532f expref: 38 pid: 6524 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.811178:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ae11c lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09532f expref: 38 pid: 6524 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.811200:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609a600/0x7ee4453e4a1ae11c lrc: 0/0,0 mode: --/PR res: [0x240000401:0x9d:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09532f expref: 38 pid: 6524 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.811214:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x9d:0x0].0x0 (ffffa07305ca2b00) 00010000:00010000:2.0:1787739072.811221:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae624 lrc: 2/0,0 mode: --/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae61d expref: 18 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.811232:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae624 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae61d expref: 18 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.811242:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae624 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae61d expref: 18 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.811271:0:10340:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae624 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae61d expref: 18 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.811285:0:10340:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665763328:0@lo:105: 00000100:00100000:2.0:1787739072.811338:0:10340:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.811373:0:10340:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062340c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.811495:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665763328 00000100:00100000:1.0:1787739072.811500:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665763328 00000100:00100000:1.0:1787739072.811529:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1207 00000100:00100000:1.0:1787739072.811535:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a321c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665763328:12345-0@lo:105: 00010000:00010000:1.0:1787739072.811546:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ae61d lrc: 5/0,1 mode: --/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae624 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.811557:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ae61d lrc: 5/0,1 mode: --/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae624 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.811565:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ae61d lrc: 5/0,1 mode: --/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae624 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.811572:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ae61d lrc: 5/0,1 mode: EX/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae624 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.811580:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ae61d lrc: 6/0,1 mode: EX/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae624 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.811588:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ae61d lrc: 6/0,1 mode: EX/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ae624 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.811614:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069aac00) 00000100:00100000:1.0:1787739072.811649:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a321c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665763328:12345-0@lo:105: Request processed in 115us (311us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.811657:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1207 00010000:00010000:3.0:1787739072.811821:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ae61d lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae624 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.811832:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ae61d lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae624 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.812462:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d86940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665763456:0@lo:403:mdt00_003.0 00000100:00100000:3.0:1787739072.812514:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739072.812541:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739072.812778:0:10340:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665763328:0@lo:105: 00010000:00010000:2.0:1787739072.812792:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739072.812837:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ae580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+37:2409:x1874579652535168:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 1693us (2019us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.812845:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1583 00000100:00100000:2.0:1787739072.812871:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665763456 00000100:00100000:2.0:1787739072.812874:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665763456 00000100:00100000:2.0:1787739072.812900:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3558 00000100:00100000:2.0:1787739072.812906:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d87c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+19:7944:x1874579665763456:12345-0@lo:403:mdt00_003.0 00000100:00100000:2.0:1787739072.816528:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d87c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+19:7944:x1874579665763456:12345-0@lo:403:mdt00_003.0 Request processed in 3613us (4004us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.816542:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3558 00000100:00100000:0.0:1787739072.817967:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d86940 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665763456:0@lo:403:mdt00_003.0 00000040:00080000:0.0:1787739072.819353:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].395, 952 off597704 00000040:00080000:0.0:1787739072.819869:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].394, 1032 off647288 00000004:00080000:0.0:1787739072.819876:0:7944:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0740a46c0c0 with version 748 00000004:00080000:3.0:1787739072.819981:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 748 rpc_version 748 00000020:00100000:3.0:1787739072.820015:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739072.820026:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304d85e00 x1874579665763712/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739072.820050:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d85e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665763712:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739072.820115:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739072.820200:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e780 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.820250:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665763712 00000100:00100000:1.0:1787739072.820257:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665763712 00000100:00100000:1.0:1787739072.820346:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3559 00000100:00100000:1.0:1787739072.820355:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+19:7740:x1874579665763712:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739072.821090:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+20:7740:x1874579665763712:12345-0@lo:1000:osp_up1-0.0 Request processed in 734us (972us total) trans 4294968456 rc 0/0 00000100:00100000:1.0:1787739072.821104:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3559 00000100:00100000:0.0:1787739072.821388:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d85e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665763712:0@lo:1000:osp_up1-0.0 00000004:00200000:3.0:1787739072.821651:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14c0:0x0]: returning size 4096 00010000:00010000:2.0:1787739072.821805:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae616 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.821814:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae616 lrc: 2/0,0 mode: EX/EX res: [0x200000400:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.821825:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae616 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.821831:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306089600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739072.821834:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae616 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.821841:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae616 lrc: 4/0,0 mode: EX/EX res: [0x200000400:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.821855:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x9d:0x0].0x0 (ffffa0730713a100) 00010000:00010000:2.0:1787739072.821858:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739072.821862:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae616 lrc: 2/0,0 mode: --/EX res: [0x200000400:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.821868:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae616 lrc: 0/0,0 mode: --/EX res: [0x200000400:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.821889:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ae61d lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae624 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.821898:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ae61d lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae624 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.821905:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ae61d lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae624 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.821961:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739072.821969:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x9d:0x0] 00010000:00010000:2.0:1787739072.821977:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1ae60f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.821985:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1ae60f lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.821991:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1ae60f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.821996:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306089e00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739072.821999:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1ae60f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.822005:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1ae60f lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.822013:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14c0:0x0].0x0 (ffffa07305ca2d00) 00010000:00010000:2.0:1787739072.822015:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739072.822018:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1ae60f lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.822024:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1ae60f lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.822035:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae601 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae608 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.822042:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae601 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae608 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.822049:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae601 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae608 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.822083:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787739072.822089:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ae61d lrc: 2/0,0 mode: EX/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae624 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.822097:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aac00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739072.822101:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ae61d lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae624 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739072.822109:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ae61d lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae624 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.822138:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ae61d lrc: 2/0,0 mode: --/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae624 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.822145:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739072.822175:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ae61d lrc: 1/0,0 mode: --/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae624 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.822181:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aac00/0x7ee4453e4a1ae61d lrc: 0/0,0 mode: --/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae624 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.822195:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739072.822198:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae601 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae608 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.822203:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739072.822206:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae601 lrc: 5/0,0 mode: EX/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae608 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739072.822217:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae601 lrc: 5/0,0 mode: EX/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae608 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.822233:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae601 lrc: 4/0,0 mode: --/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae608 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.822239:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739072.822245:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae601 lrc: 3/0,0 mode: --/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae608 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.822252:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739072.822254:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739072.822256:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739072.822296:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304245e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665763840:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739072.822342:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739072.822377:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787739072.822383:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665763968:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739072.822403:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739072.822452:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665763840 00000100:00100000:1.0:1787739072.822457:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665763840 00000100:00100000:1.0:1787739072.822497:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665763968 00000100:00100000:1.0:1787739072.822500:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665763968 00000100:00100000:1.0:1787739072.822518:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1387 00000100:00100000:1.0:1787739072.822525:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+21:3650:x1874579665763840:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739072.822536:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739072.822552:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae624 lrc: 0/0,0 mode: --/EX res: [0x240000401:0x9d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae61d expref: 21 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.822564:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x9d:0x0].0x0 (ffffa07305ca2b00) 00010000:00010000:1.0:1787739072.822571:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739072.822609:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+20:3650:x1874579665763840:12345-0@lo:103:ldlm_bl.0 Request processed in 84us (266us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.822615:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1387 00000100:00100000:1.0:1787739072.822629:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1388 00000100:00100000:1.0:1787739072.822635:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+19:3650:x1874579665763968:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739072.822641:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739072.822651:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae608 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae601 expref: 19 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.822666:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14c0:0x0].0x0 (ffffa07305ca2800) 00010000:00010000:1.0:1787739072.822672:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739072.822689:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+18:3650:x1874579665763968:12345-0@lo:103:ldlm_bl.0 Request processed in 55us (286us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.822695:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1388 00000100:00100000:1.0:1787739072.822728:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f770c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665763968:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739072.822736:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304245e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665763840:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739072.823051:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:2.0:1787739072.823057:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae601 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14c0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae608 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739072.823080:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:2.0:1787739072.823085:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae5e5 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae5ec expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.823093:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae5e5 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae5ec expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.823100:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae5e5 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae5ec expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.823165:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739072.823171:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae5e5 lrc: 4/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae5ec expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.823178:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1e00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739072.823182:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae5e5 lrc: 5/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae5ec expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739072.823190:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae5e5 lrc: 5/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae5ec expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.823213:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae5e5 lrc: 4/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae5ec expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.823219:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1787739072.823260:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665764096:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739072.823289:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787739072.823335:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae5e5 lrc: 3/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae5ec expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.823344:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739072.823346:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739072.823348:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739072.823390:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665764096 00000100:00100000:1.0:1787739072.823394:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665764096 00000100:00100000:1.0:1787739072.823446:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1389 00000100:00100000:1.0:1787739072.823453:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+18:3652:x1874579665764096:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739072.823462:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739072.823483:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae5ec lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae5e5 expref: 18 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.823494:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305ca2500) 00010000:00010000:1.0:1787739072.823501:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739072.823684:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:2.0:1787739072.823689:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae5e5 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae5ec expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739072.823705:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:2.0:1787739072.823710:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0600/0x7ee4453e4a1ae5f3 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x44749f58 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae5fa expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.823726:0:7944:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0600/0x7ee4453e4a1ae5f3 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x44749f58 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae5fa expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.823788:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d85680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:17726:x1874579652534912:12345-192.168.203.15@tcp:36:rm.0 Request processed in 35479us (36320us total) trans 4294983472 rc 0/0 00000100:00100000:2.0:1787739072.823798:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23671 00010000:00010000:3.0:1787739072.823906:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739072.824001:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+17:3652:x1874579665764096:12345-0@lo:103:ldlm_bl.0 Request processed in 547us (709us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.824008:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1389 00000100:00100000:0.0:1787739072.824064:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665764096:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739072.829710:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.829869:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652535424 00000100:00100000:2.0:1787739072.829879:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652535424 00000100:00100000:2.0:1787739072.829972:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23672 00000100:00100000:2.0:1787739072.829983:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:17726:x1874579652535424:12345-192.168.203.15@tcp:101:rm.0 00010000:00010000:2.0:1787739072.830007:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739072.830027:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae62b lrc: 2/0,0 mode: --/CR res: [0x200000402:0x14e6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095486 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739072.830176:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14e6:0x0]: returning size 4096 00010000:00010000:2.0:1787739072.830272:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae62b lrc: 1/0,0 mode: --/CR res: [0x200000402:0x14e6:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095486 expref: 62 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.830286:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14e6:0x0].0x0 (ffffa07403eeb100) 00010000:00010000:2.0:1787739072.830289:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739072.830294:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae62b lrc: 0/0,0 mode: --/CR res: [0x200000402:0x14e6:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095486 expref: 62 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.830307:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa0e00, rc 0) 00000100:00100000:2.0:1787739072.830383:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:17726:x1874579652535424:12345-192.168.203.15@tcp:101:rm.0 Request processed in 401us (681us total) trans 4294983473 rc 301/301 00000100:00100000:2.0:1787739072.830390:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23672 00000100:00100000:3.0:1787739072.835558:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739072.836090:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652535680 00000100:00100000:0.0:1787739072.836101:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652535680 00000100:00100000:0.0:1787739072.836187:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5064 00000100:00100000:0.0:1787739072.836197:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:17726:x1874579652535680:12345-192.168.203.15@tcp:35:rm.0 00000100:00100000:0.0:1787739072.836390:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+62:17726:x1874579652535680:12345-192.168.203.15@tcp:35:rm.0 Request processed in 195us (841us total) trans 4294983474 rc 0/0 00000100:00100000:0.0:1787739072.836397:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5064 00010000:00010000:3.0:1787739072.842728:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739072.843578:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.844063:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652535936 00000100:00100000:2.0:1787739072.844072:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652535936 00000100:00100000:2.0:1787739072.844132:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23673 00000100:00100000:2.0:1787739072.844142:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d86580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+62:17726:x1874579652535936:12345-192.168.203.15@tcp:36:rm.0 00010000:00010000:2.0:1787739072.844200:0:13793:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1787739072.844227:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1ae3d8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14e6:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09544e expref: 62 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.844249:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633d200/0x7ee4453e4a1ae234 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14e6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09539f expref: 61 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.844260:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14e6:0x0].0x0 (ffffa07403eeb100) 00010000:00010000:2.0:1787739072.844269:0:13793:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739072.844292:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1ae639 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.844304:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1ae639 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.844313:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1ae639 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.844336:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1ae640 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.844344:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1ae640 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.844350:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1ae640 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.844431:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1c00/0x7ee4453e4a1ae647 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14e6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.844439:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1c00/0x7ee4453e4a1ae647 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14e6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.844446:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1c00/0x7ee4453e4a1ae647 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14e6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.844463:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae64e lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.844675:0:13793:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae242 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953a6 expref: 60 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.844682:0:13793:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae64e lrc: 4/0,1 mode: --/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.844725:0:13793:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae242 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953a6 expref: 60 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.844741:0:13793:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae242 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953a6 expref: 60 pid: 6525 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.844757:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665764224:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739072.844835:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89600 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.848852:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.848944:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:13793:1874579665764224:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739072.848982:0:13793:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae64e lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.849053:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652536064 00000100:00100000:0.0:1787739072.849061:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652536064 00010000:00010000:0.0:1787739072.849099:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae242 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953a6 expref: 61 pid: 6525 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.849141:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1584 00000100:00100000:0.0:1787739072.849150:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba0850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:2408:x1874579652536064:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739072.849163:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739072.849170:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae242 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953a6 expref: 61 pid: 6525 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.849180:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae242 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953a6 expref: 61 pid: 6525 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.849203:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae242 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953a6 expref: 61 pid: 6525 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.849219:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb0:0x0].0x0 (ffffa0730608b200) 00010000:00010000:0.0:1787739072.849223:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae64e lrc: 3/0,1 mode: --/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.849232:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae64e lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.849239:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae64e lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.849267:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739072.849314:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba0850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:2408:x1874579652536064:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 165us (471us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.849322:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1584 00010000:00010000:1.0:1787739072.849676:0:13793:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae64e lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.849685:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae64e lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.849735:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840c00/0x7ee4453e4a1ae655 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.849742:0:13793:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840c00/0x7ee4453e4a1ae655 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.849752:0:13793:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840c00/0x7ee4453e4a1ae655 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.849771:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f3840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665764352:0@lo:101:mdt00_004.0 00000100:00100000:1.0:1787739072.849812:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739072.849844:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739072.849894:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665764352 00000100:00100000:2.0:1787739072.849900:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665764352 00000100:00100000:2.0:1787739072.849933:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3560 00000100:00100000:2.0:1787739072.849940:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+17:13793:x1874579665764352:12345-0@lo:101:mdt00_004.0 00010000:00010000:2.0:1787739072.849969:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739072.849985:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1ae65c lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae655 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.850004:0:6528:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae250 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953ad expref: 36 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.850013:0:6528:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1ae65c lrc: 4/0,0 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae655 expref: 18 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.850036:0:6528:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae250 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953ad expref: 36 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.850047:0:6528:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae250 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953ad expref: 36 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.850057:0:6528:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001_UUID:6528:1874579665764480:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739072.850106:0:6528:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07302e89600 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.861441:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739072.861522:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652536192 00000100:00100000:0.0:1787739072.861532:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652536192 00010000:00010000:0.0:1787739072.861571:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae250 lrc: 5/0,0 mode: PR/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953ad expref: 37 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.861617:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1585 00000100:00100000:0.0:1787739072.861626:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304247c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+37:2408:x1874579652536192:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739072.861641:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739072.861648:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae250 lrc: 5/0,0 mode: PR/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953ad expref: 37 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.861660:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae250 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953ad expref: 37 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.861683:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb0:0x0].0x0 (ffffa0730713a500) 00010000:00010000:0.0:1787739072.861690:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1ae65c lrc: 3/0,0 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae655 expref: 18 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.861702:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1ae65c lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae655 expref: 18 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.861711:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1ae65c lrc: 4/0,0 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x50080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae655 expref: 18 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.861746:0:9525:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1ae65c lrc: 4/0,0 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae655 expref: 18 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.861761:0:9525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74b40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665764608:0@lo:105: 00000100:00100000:0.0:1787739072.861809:0:9525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739072.861857:0:9525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062486c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.862468:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665764608 00000100:00100000:1.0:1787739072.862475:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665764608 00000100:00100000:1.0:1787739072.862514:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1208 00000100:00100000:1.0:1787739072.862522:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f743c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665764608:12345-0@lo:105: 00010000:00010000:1.0:1787739072.862534:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840c00/0x7ee4453e4a1ae655 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae65c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.862546:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840c00/0x7ee4453e4a1ae655 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae65c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.862556:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840c00/0x7ee4453e4a1ae655 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae65c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.862564:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840c00/0x7ee4453e4a1ae655 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae65c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.862577:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840c00/0x7ee4453e4a1ae655 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae65c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.862585:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840c00/0x7ee4453e4a1ae655 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ae65c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.862610:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07306840c00) 00000100:00100000:1.0:1787739072.862657:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f743c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:9525:x1874579665764608:12345-0@lo:105: Request processed in 135us (847us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.862665:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1208 00000100:00100000:1.0:1787739072.862701:0:9525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74b40 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001_UUID:9525:1874579665764608:0@lo:105: 00010000:00010000:1.0:1787739072.862718:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739072.862758:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304247c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+37:2408:x1874579652536192:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 1132us (1347us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.862766:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1585 00010000:00010000:2.0:1787739072.868745:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d600/0x7ee4453e4a1ae250 lrc: 0/0,0 mode: --/PR res: [0x240000401:0xb0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953ad expref: 36 pid: 13793 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.868774:0:6528:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001_UUID:6528:1874579665764480:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739072.868809:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1ae65c lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae655 expref: 18 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.868819:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb0:0x0].0x0 (ffffa0730713a500) 00010000:00010000:2.0:1787739072.868822:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739072.868825:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa1e00, rc 0) 00000100:00100000:2.0:1787739072.868900:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+18:13793:x1874579665764352:12345-0@lo:101:mdt00_004.0 Request processed in 18960us (19087us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.868908:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3560 00000100:00100000:1.0:1787739072.869028:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f3840 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665764352:0@lo:101:mdt00_004.0 00010000:00010000:1.0:1787739072.869047:0:13793:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840c00/0x7ee4453e4a1ae655 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae65c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.869794:0:13793:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304245e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665764736:0@lo:403:mdt00_004.0 00000100:00100000:1.0:1787739072.869849:0:13793:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739072.869914:0:13793:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288c00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.871255:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665764736 00000100:00100000:0.0:1787739072.871266:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665764736 00000100:00100000:0.0:1787739072.871374:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3561 00000100:00100000:0.0:1787739072.871385:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+18:13793:x1874579665764736:12345-0@lo:403:mdt00_004.0 00000100:00100000:1.0:1787739072.876581:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+18:13793:x1874579665764736:12345-0@lo:403:mdt00_004.0 Request processed in 5188us (6721us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.876606:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3561 00000100:00100000:2.0:1787739072.876801:0:13793:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304245e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:13793:1874579665764736:0@lo:403:mdt00_004.0 00000040:00080000:2.0:1787739072.878145:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].396, 952 off598656 00000040:00080000:2.0:1787739072.878319:0:13793:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].395, 1032 off648320 00000004:00080000:2.0:1787739072.878325:0:13793:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07302e89600 with version 749 00000004:00080000:0.0:1787739072.879086:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 749 rpc_version 749 00000020:00100000:0.0:1787739072.879128:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739072.879137:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303f74b40 x1874579665764992/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1992/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739072.879160:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f74b40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665764992:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739072.879221:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739072.880327:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306248300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739072.880373:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665764992 00000100:00100000:2.0:1787739072.880382:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665764992 00000100:00100000:2.0:1787739072.880453:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3562 00000100:00100000:2.0:1787739072.880472:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f770c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+18:7740:x1874579665764992:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739072.880921:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f770c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+19:7740:x1874579665764992:12345-0@lo:1000:osp_up1-0.0 Request processed in 459us (1698us total) trans 4294968457 rc 0/0 00000100:00100000:2.0:1787739072.880929:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3562 00000100:00100000:0.0:1787739072.881198:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f74b40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665764992:0@lo:1000:osp_up1-0.0 00000004:00200000:2.0:1787739072.881250:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x14e6:0x0]: returning size 4096 00010000:00010000:2.0:1787739072.881270:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae64e lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.881279:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae64e lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.881286:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae64e lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.881292:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739072.881296:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae64e lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.881938:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae64e lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.881966:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xb0:0x0].0x0 (ffffa0730608b200) 00010000:00010000:2.0:1787739072.881969:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739072.881974:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae64e lrc: 2/0,0 mode: --/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.881980:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae64e lrc: 0/0,0 mode: --/EX res: [0x200000400:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.882003:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840c00/0x7ee4453e4a1ae655 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae65c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.882012:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840c00/0x7ee4453e4a1ae655 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae65c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.882019:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840c00/0x7ee4453e4a1ae655 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae65c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.882036:0:13793:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739072.882044:0:13793:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xb0:0x0] 00010000:00010000:2.0:1787739072.882053:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1c00/0x7ee4453e4a1ae647 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14e6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.882183:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739072.882190:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840c00/0x7ee4453e4a1ae655 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae65c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.882212:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840c00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739072.882217:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840c00/0x7ee4453e4a1ae655 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae65c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739072.882231:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840c00/0x7ee4453e4a1ae655 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae65c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.882272:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840c00/0x7ee4453e4a1ae655 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae65c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.882280:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1787739072.882401:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665765120:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739072.882569:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1c00/0x7ee4453e4a1ae647 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x14e6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.882575:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1c00/0x7ee4453e4a1ae647 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14e6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.882585:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1c00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739072.882588:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1c00/0x7ee4453e4a1ae647 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14e6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.882595:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1c00/0x7ee4453e4a1ae647 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x14e6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.882604:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14e6:0x0].0x0 (ffffa074419d5200) 00010000:00010000:2.0:1787739072.882607:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739072.882610:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1c00/0x7ee4453e4a1ae647 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14e6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.882615:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1c00/0x7ee4453e4a1ae647 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14e6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.882872:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840c00/0x7ee4453e4a1ae655 lrc: 1/0,0 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae65c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.882881:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840c00/0x7ee4453e4a1ae655 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae65c expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.882898:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739072.882900:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739072.882901:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739072.882962:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739072.883146:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665765120 00000100:00100000:0.0:1787739072.883153:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665765120 00000100:00100000:0.0:1787739072.883206:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1390 00000100:00100000:0.0:1787739072.883214:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304263c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+19:3653:x1874579665765120:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739072.883226:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739072.883243:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1ae65c lrc: 0/0,0 mode: --/EX res: [0x240000401:0xb0:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae655 expref: 19 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.883254:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xb0:0x0].0x0 (ffffa0730713a500) 00010000:00010000:0.0:1787739072.883262:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739072.883606:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1ae639 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.883616:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1ae639 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.883622:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1ae639 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.883628:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1ae639 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.883633:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739072.883636:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1ae639 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.883646:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1ae639 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.883655:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5600) 00010000:00010000:2.0:1787739072.883658:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739072.883661:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1ae639 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.883667:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1ae639 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739072.883682:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e027b924 reply state 00000000dcfe4768 transno 4294983475 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1ae640 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.883695:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1ae640 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.883701:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x3fb0b00e (ffffa074419d5300) 00010000:00010000:2.0:1787739072.883703:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739072.883706:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1787739072.883843:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665765120:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739072.884053:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d86580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:17726:x1874579652535936:12345-192.168.203.15@tcp:36:rm.0 Request processed in 39911us (40478us total) trans 4294983475 rc 0/0 00000100:00100000:2.0:1787739072.884071:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23673 00000100:00100000:0.0:1787739072.884137:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304263c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+18:3653:x1874579665765120:12345-0@lo:103:ldlm_bl.0 Request processed in 86us (338us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.884145:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1390 00000100:00100000:1.0:1787739072.898974:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.899136:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652536448 00000100:00100000:2.0:1787739072.899145:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652536448 00000100:00100000:2.0:1787739072.899207:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23674 00000100:00100000:2.0:1787739072.899216:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304261e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:17726:x1874579652536448:12345-192.168.203.15@tcp:101:rm.0 00010000:00010000:2.0:1787739072.899240:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739072.899258:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1ae663 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1518:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09548d expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739072.899448:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1518:0x0]: returning size 4096 00010000:00010000:2.0:1787739072.899553:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1ae663 lrc: 1/0,0 mode: --/CR res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09548d expref: 63 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.899568:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1518:0x0].0x0 (ffffa07305eaeb00) 00010000:00010000:2.0:1787739072.899572:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739072.899577:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1ae663 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09548d expref: 63 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.899590:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa0400, rc 0) 00000100:00100000:2.0:1787739072.899640:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304261e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+62:17726:x1874579652536448:12345-192.168.203.15@tcp:101:rm.0 Request processed in 425us (676us total) trans 4294983476 rc 301/301 00000100:00100000:2.0:1787739072.899647:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23674 00000100:00100000:3.0:1787739072.918653:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739072.918885:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652536704 00000100:00100000:2.0:1787739072.918896:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652536704 00000100:00100000:2.0:1787739072.918956:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5065 00000100:00100000:2.0:1787739072.918966:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d87480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+62:17726:x1874579652536704:12345-192.168.203.15@tcp:35:rm.0 00000100:00100000:2.0:1787739072.919197:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d87480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+63:17726:x1874579652536704:12345-192.168.203.15@tcp:35:rm.0 Request processed in 232us (568us total) trans 4294983477 rc 0/0 00000100:00100000:2.0:1787739072.919204:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5065 00000100:00100000:1.0:1787739072.925368:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739072.925451:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739072.925588:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652536832 00000100:00100000:0.0:1787739072.925609:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652536832 00000100:00100000:0.0:1787739072.925685:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1391 00000100:00100000:0.0:1787739072.925695:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+36:2409:x1874579652536832:12345-192.168.203.15@tcp:103:rm.0 00010000:00010000:0.0:1787739072.925714:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739072.925749:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd600/0x7ee4453e4a1ae26c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953b4 expref: 36 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.925770:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1518:0x0].0x0 (ffffa0730713a000) 00010000:00010000:0.0:1787739072.925781:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739072.925840:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:2409:x1874579652536832:12345-192.168.203.15@tcp:103:rm.0 Request processed in 147us (425us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.925848:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1391 00000100:00100000:2.0:1787739072.928475:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652537088 00000100:00100000:2.0:1787739072.928491:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652537088 00000100:00100000:2.0:1787739072.928559:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23675 00000100:00100000:2.0:1787739072.928568:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+63:17726:x1874579652537088:12345-192.168.203.15@tcp:36:rm.0 00010000:00010000:2.0:1787739072.928648:0:6524:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739072.928942:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c600/0x7ee4453e4a1ae288 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1518:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953c2 expref: 63 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.928973:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1518:0x0].0x0 (ffffa07305eaeb00) 00010000:00010000:2.0:1787739072.928983:0:6524:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739072.929029:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae671 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.929037:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae671 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.929047:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae671 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.929067:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665765248:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1787739072.929143:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.929250:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074078cb480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.930991:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665765248 00000100:00100000:1.0:1787739072.931005:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665765248 00000100:00100000:1.0:1787739072.931066:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3563 00000100:00100000:1.0:1787739072.931075:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+18:6524:x1874579665765248:12345-0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739072.931096:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739072.931270:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1ae678 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae671 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.931304:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1ae678 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae671 expref: 19 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.931319:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1ae678 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae671 expref: 19 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.931327:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730713ad00) 00010000:00010000:1.0:1787739072.931330:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739072.931334:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306840800, rc 0) 00000100:00100000:1.0:1787739072.931474:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+19:6524:x1874579665765248:12345-0@lo:101:mdt00_001.0 Request processed in 400us (2330us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.931482:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3563 00000100:00100000:2.0:1787739072.931970:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665765248:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739072.931990:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae671 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae678 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.932001:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae671 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae678 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739072.932023:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:2.0:1787739072.932056:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1c00/0x7ee4453e4a1ae67f lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.932063:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1c00/0x7ee4453e4a1ae67f lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.932075:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1c00/0x7ee4453e4a1ae67f lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.932092:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665765376:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1787739072.932141:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.932183:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074078cb480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.932233:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665765376 00000100:00100000:0.0:1787739072.932239:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665765376 00000100:00100000:0.0:1787739072.932278:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3564 00000100:00100000:0.0:1787739072.932286:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+19:6524:x1874579665765376:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739072.932297:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739072.932310:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088200/0x7ee4453e4a1ae686 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae67f expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.932327:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088200/0x7ee4453e4a1ae686 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae67f expref: 20 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.932338:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088200/0x7ee4453e4a1ae686 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae67f expref: 20 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.932346:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xda636067 (ffffa07305ca2f00) 00010000:00010000:0.0:1787739072.932348:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739072.932351:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306088200, rc 0) 00000100:00100000:0.0:1787739072.932388:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+20:6524:x1874579665765376:12345-0@lo:101:mdt00_001.0 Request processed in 103us (247us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.932394:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3564 00000100:00100000:2.0:1787739072.932589:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665765376:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739072.932609:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1c00/0x7ee4453e4a1ae67f lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae686 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.932618:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1c00/0x7ee4453e4a1ae67f lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae686 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739072.932631:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:2.0:1787739072.932659:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665765504:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739072.932694:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739072.932884:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665765504 00000100:00100000:1.0:1787739072.932889:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665765504 00000100:00100000:1.0:1787739072.932921:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3565 00000100:00100000:1.0:1787739072.932929:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a330c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+20:6524:x1874579665765504:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739072.933000:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a330c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+20:6524:x1874579665765504:12345-0@lo:1000:mdt00_001.0 Request processed in 71us (304us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.933006:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3565 00000100:00100000:2.0:1787739072.933139:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740352a540 going to sleep for 11 seconds 00000100:00100000:2.0:1787739072.933155:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665765504:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739072.933193:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665765632:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739072.933216:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.933238:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740352a300 going to sleep for 11 seconds 00000100:00100000:3.0:1787739072.933283:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665765632 00000100:00100000:3.0:1787739072.933288:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665765632 00000100:00100000:3.0:1787739072.933320:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3566 00000100:00100000:3.0:1787739072.933327:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+20:6524:x1874579665765632:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739072.933440:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+20:6524:x1874579665765632:12345-0@lo:1000:mdt00_001.0 Request processed in 85us (194us total) trans 0 rc 1/1 00000100:00100000:3.0:1787739072.933446:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3566 00000100:00100000:2.0:1787739072.934098:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665765632:0@lo:1000:mdt00_001.0 00010000:00010000:2.0:1787739072.934164:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae68d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.934173:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae68d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.934184:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae68d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.934197:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665765760:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1787739072.934227:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.934258:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740352a540 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.934301:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665765760 00000100:00100000:0.0:1787739072.934306:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665765760 00000100:00100000:0.0:1787739072.934344:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3567 00000100:00100000:0.0:1787739072.934351:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+20:6524:x1874579665765760:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739072.934361:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739072.934376:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088000/0x7ee4453e4a1ae694 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae68d expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.934396:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088000/0x7ee4453e4a1ae694 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae68d expref: 21 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.934409:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088000/0x7ee4453e4a1ae694 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae68d expref: 21 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.934455:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1518:0x0].0x0 (ffffa07305ca2100) 00010000:00010000:0.0:1787739072.934458:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739072.934461:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306088000, rc 0) 00000100:00100000:0.0:1787739072.934496:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+21:6524:x1874579665765760:12345-0@lo:101:mdt00_001.0 Request processed in 146us (269us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.934502:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3567 00000100:00100000:2.0:1787739072.934974:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665765760:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739072.934988:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae68d lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae694 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.934997:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae68d lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae694 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.935023:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae69b lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.935032:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae69b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.935040:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae69b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.935057:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae6a2 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.935068:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae296 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953c9 expref: 62 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.935076:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae6a2 lrc: 4/0,1 mode: --/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.935099:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae296 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953c9 expref: 62 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.935112:0:6524:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae296 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953c9 expref: 62 pid: 7944 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.935123:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665765888:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739072.935185:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740352a540 going to sleep for 11 seconds 00000100:00100000:2.0:1787739072.940588:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665765888:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739072.940627:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae6a2 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.941481:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739072.941603:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652537216 00000100:00100000:0.0:1787739072.941611:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652537216 00010000:00010000:0.0:1787739072.941657:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae296 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953c9 expref: 63 pid: 7944 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.941700:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1586 00000100:00100000:0.0:1787739072.941709:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042630c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+63:2409:x1874579652537216:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739072.941732:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739072.941738:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae296 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953c9 expref: 63 pid: 7944 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.941750:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae296 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953c9 expref: 63 pid: 7944 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.941772:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1de00/0x7ee4453e4a1ae296 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953c9 expref: 63 pid: 7944 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.941788:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc6:0x0].0x0 (ffffa07305eaea00) 00010000:00010000:0.0:1787739072.941793:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae6a2 lrc: 3/0,1 mode: --/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.941803:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae6a2 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.941812:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae6a2 lrc: 4/0,1 mode: EX/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.941834:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739072.941970:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae6a2 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.941978:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae6a2 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.942019:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae6a9 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.942027:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae6a9 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.942036:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae6a9 lrc: 3/0,1 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739072.942055:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665766016:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1787739072.942099:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739072.942123:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740352a540 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.942270:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042630c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+62:2409:x1874579652537216:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 560us (791us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.942279:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1586 00000100:00100000:1.0:1787739072.943026:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665766016 00000100:00100000:1.0:1787739072.943033:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665766016 00000100:00100000:1.0:1787739072.943073:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3568 00000100:00100000:1.0:1787739072.943081:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304245e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+21:6524:x1874579665766016:12345-0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739072.943096:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739072.943111:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840600/0x7ee4453e4a1ae6b0 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae6a9 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.943134:0:6529:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc800/0x7ee4453e4a1ae2a4 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953d0 expref: 34 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.943144:0:6529:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840600/0x7ee4453e4a1ae6b0 lrc: 4/0,0 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6a9 expref: 22 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.943168:0:6529:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc800/0x7ee4453e4a1ae2a4 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953d0 expref: 34 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.943182:0:6529:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc800/0x7ee4453e4a1ae2a4 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953d0 expref: 34 pid: 6523 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.943193:0:6529:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304260f00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665766144:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739072.943244:0:6529:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739072.951084:0:6529:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304260f00 pname:cluuid:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001_UUID:6529:1874579665766144:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739072.951123:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840600/0x7ee4453e4a1ae6b0 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6a9 expref: 22 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.951136:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc6:0x0].0x0 (ffffa07305eaee00) 00010000:00010000:2.0:1787739072.951138:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739072.951142:0:6529:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840600/0x7ee4453e4a1ae6b0 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6a9 expref: 22 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.951151:0:6529:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840600/0x7ee4453e4a1ae6b0 lrc: 3/0,0 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6a9 expref: 22 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739072.951158:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306840600, rc 0) 00000100:00100000:2.0:1787739072.951231:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304245e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+22:6524:x1874579665766016:12345-0@lo:101:mdt00_001.0 Request processed in 8150us (9130us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739072.951239:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3568 00000100:00100000:1.0:1787739072.951357:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665766016:0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739072.951374:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae6a9 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae6b0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.951385:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae6a9 lrc: 4/0,1 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae6b0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739072.952219:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739072.952289:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652537344 00000100:00100000:0.0:1787739072.952296:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652537344 00010000:00010000:0.0:1787739072.952339:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc800/0x7ee4453e4a1ae2a4 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953d0 expref: 35 pid: 6523 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.952370:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1587 00000100:00100000:0.0:1787739072.952379:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042621c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:2409:x1874579652537344:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739072.952396:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739072.952402:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc800/0x7ee4453e4a1ae2a4 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953d0 expref: 35 pid: 6523 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.952446:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc800/0x7ee4453e4a1ae2a4 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953d0 expref: 35 pid: 6523 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.952468:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc800/0x7ee4453e4a1ae2a4 lrc: 0/0,0 mode: --/PR res: [0x240000401:0xc6:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953d0 expref: 35 pid: 6523 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.952484:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc6:0x0].0x0 (ffffa07305eaee00) 00010000:00010000:0.0:1787739072.952490:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840600/0x7ee4453e4a1ae6b0 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6a9 expref: 21 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.952501:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840600/0x7ee4453e4a1ae6b0 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6a9 expref: 21 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.952509:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840600/0x7ee4453e4a1ae6b0 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6a9 expref: 21 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.952539:0:10340:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840600/0x7ee4453e4a1ae6b0 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6a9 expref: 21 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.952552:0:10340:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665766272:0@lo:105: 00000100:00100000:0.0:1787739072.952595:0:10340:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739072.952627:0:10340:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306248300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.952840:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665766272 00000100:00100000:1.0:1787739072.952845:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665766272 00000100:00100000:1.0:1787739072.952882:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1209 00000100:00100000:1.0:1787739072.952889:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665766272:12345-0@lo:105: 00010000:00010000:1.0:1787739072.952900:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae6a9 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae6b0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.952910:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae6a9 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae6b0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.952918:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae6a9 lrc: 5/0,1 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae6b0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.952926:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae6a9 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae6b0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.952933:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae6a9 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae6b0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.952955:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae6a9 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ae6b0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739072.952982:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07306aa1600) 00000100:00100000:1.0:1787739072.953022:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665766272:12345-0@lo:105: Request processed in 133us (426us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.953028:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1209 00000100:00100000:1.0:1787739072.953063:0:10340:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665766272:0@lo:105: 00010000:00010000:1.0:1787739072.953077:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739072.953115:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042621c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+34:2409:x1874579652537344:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 736us (900us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.953122:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1587 00010000:00010000:3.0:1787739072.953705:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae6a9 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae6b0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.953724:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae6a9 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae6b0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739072.954410:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665766400:0@lo:403:mdt00_001.0 00000100:00100000:3.0:1787739072.956146:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739072.956241:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ef00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739072.956300:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665766400 00000100:00100000:0.0:1787739072.956309:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665766400 00000100:00100000:0.0:1787739072.956449:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3569 00000100:00100000:0.0:1787739072.956459:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d86940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+22:6524:x1874579665766400:12345-0@lo:403:mdt00_001.0 00000100:00100000:1.0:1787739072.961535:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d86940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+22:6524:x1874579665766400:12345-0@lo:403:mdt00_001.0 Request processed in 5069us (5381us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739072.961550:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3569 00000100:00100000:3.0:1787739072.961857:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665766400:0@lo:403:mdt00_001.0 00000040:00080000:0.0:1787739072.967023:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].397, 952 off599608 00000040:00080000:0.0:1787739072.967148:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].396, 1032 off649352 00000004:00080000:0.0:1787739072.967155:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306248540 with version 750 00000004:00080000:1.0:1787739072.967348:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 750 rpc_version 750 00000020:00100000:1.0:1787739072.967385:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739072.967396:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073042612c0 x1874579665766656/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739072.967442:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042612c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665766656:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739072.967510:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739072.967543:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739072.967567:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665766656 00000100:00100000:1.0:1787739072.967573:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665766656 00000100:00100000:1.0:1787739072.967629:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3570 00000100:00100000:1.0:1787739072.967648:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+22:7740:x1874579665766656:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739072.967934:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+23:7740:x1874579665766656:12345-0@lo:1000:osp_up1-0.0 Request processed in 287us (423us total) trans 4294968458 rc 0/0 00000100:00100000:1.0:1787739072.967942:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3570 00000100:00100000:1.0:1787739072.968006:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042612c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665766656:0@lo:1000:osp_up1-0.0 00000004:00200000:0.0:1787739072.968531:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1518:0x0]: returning size 4096 00010000:00010000:0.0:1787739072.968555:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae6a2 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.968565:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae6a2 lrc: 2/0,0 mode: EX/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.968572:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae6a2 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.968579:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739072.968584:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae6a2 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.968602:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae6a2 lrc: 4/0,0 mode: EX/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.968618:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xc6:0x0].0x0 (ffffa07305eaea00) 00010000:00010000:0.0:1787739072.968621:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739072.968626:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae6a2 lrc: 2/0,0 mode: --/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.968641:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1ae6a2 lrc: 0/0,0 mode: --/EX res: [0x200000400:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.968821:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae6a9 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae6b0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.968831:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae6a9 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae6b0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.968837:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae6a9 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae6b0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.968860:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739072.968869:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xc6:0x0] 00010000:00010000:0.0:1787739072.968878:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae69b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.968884:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae69b lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.968890:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae69b lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.968896:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0e00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739072.968899:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae69b lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.968905:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae69b lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.968914:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1518:0x0].0x0 (ffffa074419d5e00) 00010000:00010000:0.0:1787739072.968916:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739072.968919:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae69b lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.968925:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae69b lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.968938:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae68d lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae694 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.968954:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae68d lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae694 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.968962:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae68d lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae694 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.968975:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:0.0:1787739072.968985:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739072.968989:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae671 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae678 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.968996:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae671 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae678 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.969002:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae671 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae678 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.969020:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:0.0:1787739072.969024:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:0.0:1787739072.969028:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1c00/0x7ee4453e4a1ae67f lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae686 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.969035:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1c00/0x7ee4453e4a1ae67f lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae686 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.969223:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+62:17726:x1874579652537088:12345-192.168.203.15@tcp:36:rm.0 Request processed in 40653us (43861us total) trans 4294983478 rc 0/0 00000100:00100000:0.0:1787739072.969231:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23675 00010000:00010000:3.0:1787739072.969313:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1787739072.969321:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae6a9 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae6b0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.969328:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739072.969332:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae6a9 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae6b0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739072.969340:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae6a9 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae6b0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.969372:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae6a9 lrc: 2/0,0 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae6b0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.969379:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739072.969406:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae6a9 lrc: 1/0,0 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae6b0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.969433:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae6a9 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae6b0 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.969447:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1787739072.969450:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae68d lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae694 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.969457:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739072.969461:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae68d lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae694 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739072.969474:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae68d lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae694 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.969490:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae68d lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae694 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.969497:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739072.969503:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae68d lrc: 1/0,0 mode: --/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae694 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.969509:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae68d lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae694 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.969519:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787739072.969522:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae671 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae678 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.969527:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:3.0:1787739072.969531:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae671 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae678 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739072.969538:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae671 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae678 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.969553:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae671 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae678 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.969560:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739072.969576:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae671 lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae678 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.969582:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae671 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae678 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739072.969913:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665766784:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739072.969975:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739072.970008:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787739072.970015:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665766912:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739072.970042:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739072.970085:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665766784 00000100:00100000:0.0:1787739072.970091:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665766784 00000100:00100000:0.0:1787739072.970130:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665766912 00000100:00100000:0.0:1787739072.970133:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665766912 00000100:00100000:0.0:1787739072.970150:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1392 00000100:00100000:0.0:1787739072.970156:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306392580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+24:3650:x1874579665766784:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739072.970168:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739072.970186:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840600/0x7ee4453e4a1ae6b0 lrc: 0/0,0 mode: --/EX res: [0x240000401:0xc6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6a9 expref: 24 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.970199:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xc6:0x0].0x0 (ffffa07305eaee00) 00010000:00010000:0.0:1787739072.970207:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739072.970237:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306392580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+23:3650:x1874579665766784:12345-0@lo:103:ldlm_bl.0 Request processed in 80us (262us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.970245:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1392 00000100:00100000:0.0:1787739072.970259:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1393 00000100:00100000:0.0:1787739072.970266:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba2850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+22:3650:x1874579665766912:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739072.970275:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739072.970286:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088000/0x7ee4453e4a1ae694 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1518:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae68d expref: 22 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739072.970295:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1518:0x0].0x0 (ffffa07305ca2100) 00010000:00010000:0.0:1787739072.970300:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739072.970321:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba2850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+21:3650:x1874579665766912:12345-0@lo:103:ldlm_bl.0 Request processed in 56us (279us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739072.970327:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1393 00000100:00100000:0.0:1787739072.970377:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665766912:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739072.970386:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665766784:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739072.970458:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739072.971033:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665767040:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739072.971072:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787739072.971147:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739072.971150:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787739072.971152:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739072.971163:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665767040 00000100:00100000:3.0:1787739072.971168:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665767040 00000100:00100000:3.0:1787739072.971246:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1394 00000100:00100000:3.0:1787739072.971253:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304247c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+21:3652:x1874579665767040:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739072.971263:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739072.971278:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1ae678 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae671 expref: 21 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739072.971290:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730713ad00) 00010000:00010000:3.0:1787739072.971298:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739072.971341:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304247c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+20:3652:x1874579665767040:12345-0@lo:103:ldlm_bl.0 Request processed in 87us (268us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739072.971348:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1394 00010000:00010000:1.0:1787739072.971452:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739072.975306:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432de00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665767040:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739073.003050:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739073.005237:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652537600 00000100:00100000:2.0:1787739073.005252:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652537600 00000100:00100000:2.0:1787739073.005318:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23676 00000100:00100000:2.0:1787739073.005329:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304261a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+34:17726:x1874579652537600:12345-192.168.203.15@tcp:101:rm.0 00010000:00010000:2.0:1787739073.005360:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739073.005383:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0800/0x7ee4453e4a1ae6b7 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x17c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095494 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739073.005550:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17c:0x0]: returning size 4096 00010000:00010000:2.0:1787739073.005995:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0800/0x7ee4453e4a1ae6b7 lrc: 1/0,0 mode: --/CR res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095494 expref: 36 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.006010:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17c:0x0].0x0 (ffffa07305eae500) 00010000:00010000:2.0:1787739073.006013:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739073.006018:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0800/0x7ee4453e4a1ae6b7 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095494 expref: 36 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.006032:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa0800, rc 0) 00000100:00100000:2.0:1787739073.006137:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304261a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:17726:x1874579652537600:12345-192.168.203.15@tcp:101:rm.0 Request processed in 810us (3097us total) trans 4294968459 rc 301/301 00000100:00100000:2.0:1787739073.006146:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23676 00000100:00100000:1.0:1787739073.022404:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739073.022752:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652537856 00000100:00100000:0.0:1787739073.022761:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652537856 00000100:00100000:0.0:1787739073.022857:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5066 00000100:00100000:0.0:1787739073.022868:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074332abc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:17726:x1874579652537856:12345-192.168.203.15@tcp:35:rm.0 00000100:00100000:0.0:1787739073.023120:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074332abc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+36:17726:x1874579652537856:12345-192.168.203.15@tcp:35:rm.0 Request processed in 253us (722us total) trans 4294968460 rc 0/0 00000100:00100000:0.0:1787739073.023128:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5066 00000100:00100000:2.0:1787739073.030997:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739073.031093:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652537984 00000100:00100000:0.0:1787739073.031102:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652537984 00000100:00100000:0.0:1787739073.031165:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1395 00000100:00100000:0.0:1787739073.031175:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+62:2408:x1874579652537984:12345-192.168.203.15@tcp:103:rm.0 00010000:00010000:0.0:1787739073.031192:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739073.031222:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07400031400/0x7ee4453e4a1ae2c0 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953d7 expref: 62 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.031249:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17c:0x0].0x0 (ffffa07403eebf00) 00010000:00010000:0.0:1787739073.031259:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739073.031305:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:2408:x1874579652537984:12345-192.168.203.15@tcp:103:rm.0 Request processed in 132us (318us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739073.031313:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1395 00000100:00100000:1.0:1787739073.035490:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739073.035627:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652538240 00000100:00100000:0.0:1787739073.035647:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652538240 00000100:00100000:0.0:1787739073.035707:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23677 00000100:00100000:0.0:1787739073.035717:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304263480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+36:17726:x1874579652538240:12345-192.168.203.15@tcp:36:rm.0 00010000:00010000:0.0:1787739073.035775:0:6523:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739073.035804:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1d000/0x7ee4453e4a1ae2dc lrc: 0/0,0 mode: --/PR res: [0x240000402:0x17c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953e5 expref: 36 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.035823:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17c:0x0].0x0 (ffffa07305eae500) 00010000:00010000:0.0:1787739073.035832:0:6523:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739073.035878:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1ae6c5 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.035886:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1ae6c5 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.035896:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1ae6c5 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739073.035915:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665767168:0@lo:101:mdt00_000.0 00000100:00100000:0.0:1787739073.035975:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739073.036040:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062489c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739073.036154:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665767168 00000100:00100000:1.0:1787739073.036159:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665767168 00000100:00100000:1.0:1787739073.036187:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3571 00000100:00100000:1.0:1787739073.036193:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306391680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+17:6523:x1874579665767168:12345-0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739073.036205:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739073.036220:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1ae6cc lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae6c5 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.036242:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1ae6cc lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6c5 expref: 18 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.036254:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1ae6cc lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6c5 expref: 18 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.036262:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730713ab00) 00010000:00010000:1.0:1787739073.036265:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739073.036267:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306840e00, rc 0) 00000100:00100000:1.0:1787739073.036312:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306391680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+18:6523:x1874579665767168:12345-0@lo:101:mdt00_000.0 Request processed in 119us (337us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739073.036321:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3571 00000100:00100000:0.0:1787739073.036835:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665767168:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739073.036852:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1ae6c5 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae6cc expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.036862:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1ae6c5 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae6cc expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739073.036881:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739073.036907:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089400/0x7ee4453e4a1ae6d3 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.036913:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089400/0x7ee4453e4a1ae6d3 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.036920:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089400/0x7ee4453e4a1ae6d3 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739073.036933:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665767296:0@lo:101:mdt00_000.0 00000100:00100000:0.0:1787739073.036978:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739073.037012:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062489c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739073.037100:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665767296 00000100:00100000:2.0:1787739073.037105:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665767296 00000100:00100000:2.0:1787739073.037143:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3572 00000100:00100000:2.0:1787739073.037150:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306393480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+18:6523:x1874579665767296:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739073.037160:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739073.037174:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0800/0x7ee4453e4a1ae6da lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae6d3 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.037190:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0800/0x7ee4453e4a1ae6da lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6d3 expref: 19 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.037202:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0800/0x7ee4453e4a1ae6da lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6d3 expref: 19 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.037208:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd1c5f96a (ffffa074419d5600) 00010000:00010000:2.0:1787739073.037211:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739073.037213:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa0800, rc 0) 00000100:00100000:2.0:1787739073.037255:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306393480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+19:6523:x1874579665767296:12345-0@lo:101:mdt00_000.0 Request processed in 105us (276us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739073.037261:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3572 00000100:00100000:0.0:1787739073.038893:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665767296:0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739073.038916:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089400/0x7ee4453e4a1ae6d3 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae6da expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.038930:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089400/0x7ee4453e4a1ae6d3 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae6da expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739073.038961:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:0.0:1787739073.039010:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665767424:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1787739073.039061:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739073.039102:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062486c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739073.039208:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665767424 00000100:00100000:2.0:1787739073.039214:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665767424 00000100:00100000:2.0:1787739073.039262:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3573 00000100:00100000:2.0:1787739073.039271:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073063903c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+19:6523:x1874579665767424:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739073.039358:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073063903c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+19:6523:x1874579665767424:12345-0@lo:1000:mdt00_000.0 Request processed in 88us (296us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739073.039364:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3573 00000100:00100000:3.0:1787739073.039519:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665767424:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739073.039561:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665767552:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739073.039590:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739073.039627:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739073.039685:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665767552 00000100:00100000:1.0:1787739073.039690:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665767552 00000100:00100000:1.0:1787739073.039724:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3574 00000100:00100000:1.0:1787739073.039731:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+19:6523:x1874579665767552:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739073.039819:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+19:6523:x1874579665767552:12345-0@lo:1000:mdt00_000.0 Request processed in 88us (228us total) trans 0 rc 1/1 00000100:00100000:1.0:1787739073.039826:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3574 00000100:00100000:3.0:1787739073.039889:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665767552:0@lo:1000:mdt00_000.0 00010000:00010000:3.0:1787739073.039938:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ae6e1 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.039957:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ae6e1 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.039967:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ae6e1 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739073.039978:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665767680:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739073.040002:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739073.040027:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ef00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739073.040068:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665767680 00000100:00100000:1.0:1787739073.040072:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665767680 00000100:00100000:1.0:1787739073.040090:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3575 00000100:00100000:1.0:1787739073.040096:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+19:6523:x1874579665767680:12345-0@lo:101:mdt00_000.0 00010000:00010000:1.0:1787739073.040104:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739073.040120:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840200/0x7ee4453e4a1ae6e8 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae6e1 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.040141:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840200/0x7ee4453e4a1ae6e8 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6e1 expref: 20 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.040153:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840200/0x7ee4453e4a1ae6e8 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6e1 expref: 20 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.040162:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17c:0x0].0x0 (ffffa0730713a700) 00010000:00010000:1.0:1787739073.040165:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739073.040168:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306840200, rc 0) 00000100:00100000:1.0:1787739073.040203:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+20:6523:x1874579665767680:12345-0@lo:101:mdt00_000.0 Request processed in 107us (200us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739073.040213:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3575 00000100:00100000:3.0:1787739073.040272:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730432e940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665767680:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739073.040283:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ae6e1 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae6e8 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.040291:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ae6e1 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae6e8 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.040315:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae6ef lrc: 3/0,1 mode: --/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.040322:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae6ef lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.040339:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae6ef lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.040357:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae6f6 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.040369:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae2ea lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953ec expref: 35 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.040379:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae6f6 lrc: 4/0,1 mode: --/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.040399:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae2ea lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953ec expref: 35 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.040443:0:6523:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae2ea lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953ec expref: 35 pid: 6523 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739073.040456:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f6940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665767808:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739073.040511:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ef00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739073.045261:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f6940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001_UUID:6523:1874579665767808:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739073.045292:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae6f6 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739073.052093:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739073.052188:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652538368 00000100:00100000:0.0:1787739073.052197:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652538368 00010000:00010000:0.0:1787739073.052237:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae2ea lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953ec expref: 36 pid: 6523 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739073.052277:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1588 00000100:00100000:0.0:1787739073.052286:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304263840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+36:2408:x1874579652538368:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739073.052301:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739073.052307:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae2ea lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953ec expref: 36 pid: 6523 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.052320:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae2ea lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953ec expref: 36 pid: 6523 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.052342:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae2ea lrc: 0/0,0 mode: --/PR res: [0x240000400:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953ec expref: 36 pid: 6523 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.052358:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc5:0x0].0x0 (ffffa0730608bc00) 00010000:00010000:0.0:1787739073.052363:0:6508:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae6f6 lrc: 3/0,1 mode: --/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.052371:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae6f6 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.052379:0:6508:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae6f6 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.052448:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739073.052494:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304263840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:2408:x1874579652538368:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 209us (411us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739073.052502:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1588 00010000:00010000:3.0:1787739073.052866:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae6f6 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.052876:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae6f6 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.052922:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae6fd lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.052930:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae6fd lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.052940:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae6fd lrc: 3/0,1 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739073.052973:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f6940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665767936:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739073.053018:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739073.053067:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ef00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739073.053348:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665767936 00000100:00100000:0.0:1787739073.053354:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665767936 00000100:00100000:0.0:1787739073.053396:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3576 00000100:00100000:0.0:1787739073.053403:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+20:6523:x1874579665767936:12345-0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739073.053451:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739073.053467:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1ae704 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae6fd expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.053491:0:12410:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc5800/0x7ee4453e4a1ae2f8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953f3 expref: 60 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.053500:0:12410:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1ae704 lrc: 4/0,0 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6fd expref: 21 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.053524:0:12410:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc5800/0x7ee4453e4a1ae2f8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953f3 expref: 60 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.053536:0:12410:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc5800/0x7ee4453e4a1ae2f8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953f3 expref: 60 pid: 6525 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739073.053546:0:12410:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:12410:1874579665768064:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739073.053820:0:12410:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062489c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739073.060062:0:12410:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:12410:1874579665768064:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739073.060098:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1ae704 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6fd expref: 21 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.060109:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc5:0x0].0x0 (ffffa07305eae100) 00010000:00010000:0.0:1787739073.060112:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739073.060115:0:12410:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1ae704 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6fd expref: 21 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.060126:0:12410:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1ae704 lrc: 3/0,0 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6fd expref: 21 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.060133:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306089c00, rc 0) 00000100:00100000:0.0:1787739073.060195:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f6580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+21:6523:x1874579665767936:12345-0@lo:101:mdt00_000.0 Request processed in 6791us (7175us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739073.060204:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3576 00000100:00100000:3.0:1787739073.060622:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f6940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665767936:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739073.060641:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae6fd lrc: 4/0,1 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae704 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.060653:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae6fd lrc: 4/0,1 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae704 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739073.070495:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739073.070894:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652538496 00000100:00100000:2.0:1787739073.070903:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652538496 00010000:00010000:2.0:1787739073.070943:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc5800/0x7ee4453e4a1ae2f8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953f3 expref: 61 pid: 6525 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739073.070992:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1589 00000100:00100000:2.0:1787739073.071001:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304261680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:2409:x1874579652538496:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739073.071017:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739073.071023:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc5800/0x7ee4453e4a1ae2f8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953f3 expref: 61 pid: 6525 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.071038:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc5800/0x7ee4453e4a1ae2f8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953f3 expref: 61 pid: 6525 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.071060:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07404cc5800/0x7ee4453e4a1ae2f8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc5:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0953f3 expref: 61 pid: 6525 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.071078:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc5:0x0].0x0 (ffffa07305eae100) 00010000:00010000:2.0:1787739073.071084:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1ae704 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6fd expref: 20 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.071094:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1ae704 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6fd expref: 20 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.071103:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1ae704 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6fd expref: 20 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.071137:0:10340:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1ae704 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6fd expref: 20 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739073.071151:0:10340:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306007c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:10340:1874579665768192:0@lo:105: 00000100:00100000:2.0:1787739073.071203:0:10340:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739073.071245:0:10340:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740352a6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739073.071345:0:6507:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787739073.071357:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665768192 00000100:00100000:1.0:1787739073.071360:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665768192 00000100:00100000:1.0:1787739073.071395:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1210 00000100:00100000:1.0:1787739073.071403:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743f7d8050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:10340:x1874579665768192:12345-0@lo:105: 00010000:00010000:1.0:1787739073.071436:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae6fd lrc: 5/0,1 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae704 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.071447:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae6fd lrc: 5/0,1 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae704 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.071455:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae6fd lrc: 5/0,1 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae704 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.071463:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae6fd lrc: 5/0,1 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae704 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.071469:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae6fd lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae704 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.071477:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae6fd lrc: 6/0,1 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ae704 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.071503:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa073069aac00) 00000100:00100000:1.0:1787739073.071550:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743f7d8050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:10340:x1874579665768192:12345-0@lo:105: Request processed in 149us (346us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739073.071557:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1210 00000100:00100000:0.0:1787739073.071663:0:10340:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306007c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000_UUID:10340:1874579665768192:0@lo:105: 00010000:00010000:0.0:1787739073.071680:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739073.071727:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304261680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:2409:x1874579652538496:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 726us (1240us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739073.071734:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1589 00010000:00010000:3.0:1787739073.071801:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae6fd lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae704 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.071811:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae6fd lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae704 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739073.072506:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f6940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665768320:0@lo:403:mdt00_000.0 00000100:00100000:3.0:1787739073.072551:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739073.072759:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ef00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739073.076650:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665768320 00000100:00100000:0.0:1787739073.076665:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665768320 00000100:00100000:1.0:1787739073.078030:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3577 00000100:00100000:1.0:1787739073.078042:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+21:6523:x1874579665768320:12345-0@lo:403:mdt00_000.0 00000100:00100000:1.0:1787739073.079819:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+21:6523:x1874579665768320:12345-0@lo:403:mdt00_000.0 Request processed in 1772us (7260us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739073.079833:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3577 00000100:00100000:3.0:1787739073.084080:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f6940 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-mdtlov_UUID:6523:1874579665768320:0@lo:403:mdt00_000.0 00000040:00080000:3.0:1787739073.085069:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].378, 952 off567560 00000040:00080000:3.0:1787739073.085241:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].378, 1032 off619376 00000004:00080000:3.0:1787739073.085249:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa0730610ef00 with version 717 00000004:00080000:2.0:1787739073.085360:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 717 rpc_version 717 00000020:00100000:2.0:1787739073.085396:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739073.085405:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306007c00 x1874579665768576/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2608/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739073.085456:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306007c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665768576:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739073.085521:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739073.085548:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740352ae40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739073.085704:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665768576 00000100:00100000:0.0:1787739073.085712:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665768576 00000100:00100000:0.0:1787739073.085768:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3578 00000100:00100000:0.0:1787739073.085778:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+21:7467:x1874579665768576:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739073.086191:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+22:7467:x1874579665768576:12345-0@lo:1000:osp_up0-1.0 Request processed in 414us (668us total) trans 4294983479 rc 0/0 00000100:00100000:0.0:1787739073.086200:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3578 00000100:00100000:2.0:1787739073.087011:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306007c00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665768576:0@lo:1000:osp_up0-1.0 00000004:00200000:3.0:1787739073.087649:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x17c:0x0]: returning size 4096 00010000:00010000:3.0:1787739073.087674:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae6f6 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.087689:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae6f6 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.087696:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae6f6 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.087702:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739073.087707:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae6f6 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.087714:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae6f6 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.087728:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xc5:0x0].0x0 (ffffa0730608bc00) 00010000:00010000:3.0:1787739073.087731:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739073.087736:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae6f6 lrc: 2/0,0 mode: --/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.087741:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae6f6 lrc: 0/0,0 mode: --/EX res: [0x240000400:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.087760:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae6fd lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae704 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.087768:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae6fd lrc: 4/0,1 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae704 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.087776:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae6fd lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae704 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.087797:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739073.087805:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0xc5:0x0] 00010000:00010000:3.0:1787739073.087814:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae6ef lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.087820:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae6ef lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.087825:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae6ef lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.087831:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aaa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739073.087833:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae6ef lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.087839:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae6ef lrc: 4/0,0 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.087847:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17c:0x0].0x0 (ffffa0730608b400) 00010000:00010000:3.0:1787739073.087850:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739073.087852:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae6ef lrc: 2/0,0 mode: --/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.087858:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae6ef lrc: 0/0,0 mode: --/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.087871:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ae6e1 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae6e8 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.087878:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ae6e1 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae6e8 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.087886:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ae6e1 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae6e8 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.087963:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:3.0:1787739073.087975:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739073.087980:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1ae6c5 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae6cc expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.087987:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1ae6c5 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae6cc expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.087993:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1ae6c5 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae6cc expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.088118:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1787739073.088125:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae6fd lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae704 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.088132:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aac00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739073.088136:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae6fd lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae704 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739073.088144:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae6fd lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae704 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.088176:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae6fd lrc: 2/0,0 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae704 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.088182:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739073.088214:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae6fd lrc: 1/0,0 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae704 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.088222:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069aac00/0x7ee4453e4a1ae6fd lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae704 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.088235:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1787739073.088239:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ae6e1 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae6e8 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.088246:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739073.088249:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ae6e1 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae6e8 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739073.088255:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ae6e1 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae6e8 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.088271:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ae6e1 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae6e8 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.088276:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739073.088289:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ae6e1 lrc: 1/0,0 mode: --/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae6e8 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.088295:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab400/0x7ee4453e4a1ae6e1 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae6e8 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.088304:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739073.088308:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1ae6c5 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae6cc expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.088314:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306088000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739073.088317:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1ae6c5 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae6cc expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739073.088323:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1ae6c5 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae6cc expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.088340:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1ae6c5 lrc: 4/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae6cc expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.088345:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739073.088349:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1ae6c5 lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae6cc expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.088355:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739073.088356:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739073.088358:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739073.088366:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787739073.088381:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665768704:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739073.088451:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739073.088493:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665768832:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739073.088517:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739073.088535:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787739073.088541:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665768960:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739073.088564:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739073.088596:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665768704 00000100:00100000:2.0:1787739073.088603:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665768704 00000100:00100000:2.0:1787739073.089325:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665768832 00000100:00100000:2.0:1787739073.089331:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665768832 00000100:00100000:2.0:1787739073.089367:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665768960 00000100:00100000:2.0:1787739073.089370:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665768960 00000100:00100000:2.0:1787739073.089385:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1396 00000100:00100000:2.0:1787739073.089393:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+24:3652:x1874579665768704:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739073.089407:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739073.089450:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1ae704 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xc5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6fd expref: 24 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.089465:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc5:0x0].0x0 (ffffa07305eae100) 00010000:00010000:2.0:1787739073.089473:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739073.089518:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+23:3652:x1874579665768704:12345-0@lo:103:ldlm_bl.0 Request processed in 124us (1065us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739073.089525:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1396 00000100:00100000:2.0:1787739073.089541:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1397 00000100:00100000:2.0:1787739073.089547:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+22:3650:x1874579665768832:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739073.089554:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739073.089566:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840200/0x7ee4453e4a1ae6e8 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x17c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6e1 expref: 22 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.089577:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x17c:0x0].0x0 (ffffa0730713a700) 00010000:00010000:2.0:1787739073.089584:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739073.089610:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+21:3650:x1874579665768832:12345-0@lo:103:ldlm_bl.0 Request processed in 64us (1092us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739073.089616:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1397 00000100:00100000:2.0:1787739073.089629:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1398 00000100:00100000:2.0:1787739073.089651:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba0050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+20:3650:x1874579665768960:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739073.089659:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739073.089670:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1ae6cc lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6c5 expref: 20 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.089679:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730713ab00) 00010000:00010000:2.0:1787739073.089686:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739073.089709:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba0050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+19:3650:x1874579665768960:12345-0@lo:103:ldlm_bl.0 Request processed in 75us (1144us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739073.089715:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1398 00000100:00100000:2.0:1787739073.089780:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665768960:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739073.089791:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665768832:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739073.089822:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665768704:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739073.090065:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:3.0:1787739073.090073:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1ae6c5 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae6cc expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739073.090094:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739073.090133:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089400/0x7ee4453e4a1ae6d3 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae6da expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.090142:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089400/0x7ee4453e4a1ae6d3 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae6da expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739073.090214:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304263480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:17726:x1874579652538240:12345-192.168.203.15@tcp:36:rm.0 Request processed in 54497us (54731us total) trans 4294968461 rc 0/0 00000100:00100000:3.0:1787739073.090225:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23677 00010000:00010000:2.0:1787739073.090586:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739073.090821:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739073.104687:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739073.104781:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652538624 00000100:00100000:0.0:1787739073.104789:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652538624 00000100:00100000:0.0:1787739073.104852:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23678 00000100:00100000:0.0:1787739073.104862:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304260b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:17726:x1874579652538624:12345-192.168.203.15@tcp:101:rm.0 00010000:00010000:0.0:1787739073.104886:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739073.104906:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088800/0x7ee4453e4a1ae70b lrc: 2/0,0 mode: --/CR res: [0x240000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09549b expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.105037:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089000/0x7ee4453e4a1ae712 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.105053:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089000/0x7ee4453e4a1ae712 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.105066:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089000/0x7ee4453e4a1ae712 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.105077:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089000/0x7ee4453e4a1ae712 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.105084:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089000/0x7ee4453e4a1ae712 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.105090:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089000/0x7ee4453e4a1ae712 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.105100:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306089000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739073.105105:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089000/0x7ee4453e4a1ae712 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.105112:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089000/0x7ee4453e4a1ae712 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.105123:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x3e:0x0].0x0 (ffffa0730713af00) 00010000:00010000:0.0:1787739073.105126:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739073.105131:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089000/0x7ee4453e4a1ae712 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.105137:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089000/0x7ee4453e4a1ae712 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.105169:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088800/0x7ee4453e4a1ae70b lrc: 1/0,0 mode: --/CR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09549b expref: 36 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.105179:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x3e:0x0].0x0 (ffffa0730713af00) 00010000:00010000:0.0:1787739073.105181:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739073.105184:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088800/0x7ee4453e4a1ae70b lrc: 0/0,0 mode: --/CR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09549b expref: 36 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.105194:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306088800, rc 0) 00000100:00100000:0.0:1787739073.105255:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304260b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:17726:x1874579652538624:12345-192.168.203.15@tcp:101:rm.0 Request processed in 395us (578us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739073.105263:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23678 00000100:00100000:3.0:1787739073.117479:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739073.117583:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652538752 00000100:00100000:2.0:1787739073.117603:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652538752 00000100:00100000:2.0:1787739073.117674:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23679 00000100:00100000:2.0:1787739073.117685:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:17726:x1874579652538752:12345-192.168.203.15@tcp:101:rm.0 00010000:00010000:2.0:1787739073.117705:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739073.117737:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae719 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0954a2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.117855:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1ae720 lrc: 3/1,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.117870:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1ae720 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.117882:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1ae720 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.117892:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1ae720 lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.117899:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1ae720 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.117907:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1ae720 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.117913:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739073.117917:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1ae720 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.117925:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1ae720 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.117936:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x3e:0x0].0x0 (ffffa0730713af00) 00010000:00010000:2.0:1787739073.117939:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739073.117943:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1ae720 lrc: 2/0,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.117958:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1ae720 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.117991:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae719 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954a2 expref: 36 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.118001:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x3e:0x0].0x0 (ffffa0730713af00) 00010000:00010000:2.0:1787739073.118003:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739073.118007:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae719 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954a2 expref: 36 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.118015:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa1a00, rc 0) 00000100:00100000:2.0:1787739073.118076:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:17726:x1874579652538752:12345-192.168.203.15@tcp:101:rm.0 Request processed in 393us (605us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739073.118088:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23679 00000100:00100000:1.0:1787739073.122264:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739073.122346:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652539008 00000100:00100000:3.0:1787739073.122356:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652539008 00000100:00100000:3.0:1787739073.122434:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23680 00000100:00100000:3.0:1787739073.122445:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304263840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:17726:x1874579652539008:12345-192.168.203.15@tcp:101:rm.0 00010000:00010000:3.0:1787739073.122466:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739073.122486:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae727 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1406:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0954a9 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739073.122643:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1406:0x0]: returning size 4096 00010000:00010000:3.0:1787739073.122855:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae727 lrc: 1/0,0 mode: --/CR res: [0x200000402:0x1406:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954a9 expref: 62 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.122871:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1406:0x0].0x0 (ffffa07403eeb000) 00010000:00010000:3.0:1787739073.122874:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739073.122879:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae727 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1406:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954a9 expref: 62 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.122893:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aaa00, rc 0) 00000100:00100000:3.0:1787739073.123001:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304263840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:17726:x1874579652539008:12345-192.168.203.15@tcp:101:rm.0 Request processed in 558us (744us total) trans 4294983480 rc 301/301 00000100:00100000:3.0:1787739073.123009:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23680 00000100:00100000:1.0:1787739073.136301:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739073.136440:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652539136 00000100:00100000:2.0:1787739073.136450:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652539136 00000100:00100000:1.0:1787739073.136533:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23681 00000100:00100000:1.0:1787739073.136544:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304261680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:17726:x1874579652539136:12345-192.168.203.15@tcp:101:rm.0 00010000:00010000:1.0:1787739073.136563:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739073.136584:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841e00/0x7ee4453e4a1ae735 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0954b0 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.136729:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1ae73c lrc: 3/1,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.136743:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1ae73c lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.136754:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1ae73c lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.136764:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1ae73c lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.136774:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1ae73c lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.136781:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1ae73c lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.136787:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739073.136791:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1ae73c lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.136798:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1ae73c lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.136809:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x3e:0x0].0x0 (ffffa0730713af00) 00010000:00010000:1.0:1787739073.136812:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1787739073.136816:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1ae73c lrc: 2/0,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.136823:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1ae73c lrc: 0/0,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.136852:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841e00/0x7ee4453e4a1ae735 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954b0 expref: 36 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.136861:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x3e:0x0].0x0 (ffffa0730713af00) 00010000:00010000:1.0:1787739073.136864:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739073.136867:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841e00/0x7ee4453e4a1ae735 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954b0 expref: 36 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.136876:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841e00, rc 0) 00000100:00100000:1.0:1787739073.136935:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304261680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:17726:x1874579652539136:12345-192.168.203.15@tcp:101:rm.0 Request processed in 393us (642us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739073.136942:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23681 00000100:00100000:1.0:1787739073.152036:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739073.152216:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652539264 00000100:00100000:3.0:1787739073.152225:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652539264 00000100:00100000:3.0:1787739073.152285:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23682 00000100:00100000:3.0:1787739073.152296:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304263c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:17726:x1874579652539264:12345-192.168.203.15@tcp:101:rm.0 00010000:00010000:3.0:1787739073.152316:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739073.152337:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae743 lrc: 2/0,0 mode: --/CR res: [0x240000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0954b7 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.152480:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae74a lrc: 3/1,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.152494:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae74a lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.152505:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae74a lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.152515:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae74a lrc: 3/1,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.152521:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae74a lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.152605:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae74a lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.152613:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aaa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739073.152618:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae74a lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.152625:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae74a lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.152638:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x3e:0x0].0x0 (ffffa0730713af00) 00010000:00010000:3.0:1787739073.152641:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739073.152645:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae74a lrc: 2/0,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.152652:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aaa00/0x7ee4453e4a1ae74a lrc: 0/0,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.152685:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae743 lrc: 1/0,0 mode: --/CR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954b7 expref: 36 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.152696:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x3e:0x0].0x0 (ffffa0730713af00) 00010000:00010000:3.0:1787739073.152699:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739073.152702:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae743 lrc: 0/0,0 mode: --/CR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954b7 expref: 36 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.152712:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abe00, rc 0) 00000100:00100000:3.0:1787739073.152764:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304263c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:17726:x1874579652539264:12345-192.168.203.15@tcp:101:rm.0 Request processed in 471us (739us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739073.152772:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23682 00000100:00100000:3.0:1787739073.164237:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739073.164438:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652539392 00000100:00100000:0.0:1787739073.164448:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652539392 00000100:00100000:0.0:1787739073.164510:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5067 00000100:00100000:0.0:1787739073.164520:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f43c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:17726:x1874579652539392:12345-192.168.203.15@tcp:37:rm.0 00000100:00100000:0.0:1787739073.165011:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f43c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:17726:x1874579652539392:12345-192.168.203.15@tcp:37:rm.0 Request processed in 491us (783us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739073.165020:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5067 00000100:00100000:1.0:1787739073.193187:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739073.193331:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652539648 00000100:00100000:0.0:1787739073.193341:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652539648 00000100:00100000:0.0:1787739073.193400:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5068 00000100:00100000:0.0:1787739073.193443:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:17726:x1874579652539648:12345-192.168.203.15@tcp:35:rm.0 00000100:00100000:0.0:1787739073.193652:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+62:17726:x1874579652539648:12345-192.168.203.15@tcp:35:rm.0 Request processed in 243us (485us total) trans 4294983481 rc 0/0 00000100:00100000:0.0:1787739073.193660:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5068 00000100:00100000:1.0:1787739073.198811:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739073.198854:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739073.199643:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652540032 00000100:00100000:3.0:1787739073.199653:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652540032 00000100:00100000:3.0:1787739073.199710:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23683 00000100:00100000:3.0:1787739073.199725:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042621c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+62:17726:x1874579652540032:12345-192.168.203.15@tcp:36:rm.0 00010000:00010000:3.0:1787739073.199776:0:6523:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739073.199801:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633cc00/0x7ee4453e4a1adf55 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1406:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09526b expref: 62 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.199822:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1406:0x0].0x0 (ffffa07403eeb000) 00010000:00010000:3.0:1787739073.199831:0:6523:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739073.199879:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae751 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.199888:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae751 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.199897:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae751 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739073.199918:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665769088:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739073.199986:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739073.200062:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739073.200210:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665769088 00000100:00100000:0.0:1787739073.200215:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665769088 00000100:00100000:0.0:1787739073.200257:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3579 00000100:00100000:0.0:1787739073.200264:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+20:6523:x1874579665769088:12345-0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739073.200277:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739073.200291:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088800/0x7ee4453e4a1ae758 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae751 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.200313:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088800/0x7ee4453e4a1ae758 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae751 expref: 21 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.200337:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088800/0x7ee4453e4a1ae758 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae751 expref: 21 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.200349:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305ca2100) 00010000:00010000:0.0:1787739073.200352:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739073.200354:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306088800, rc 0) 00000100:00100000:0.0:1787739073.200399:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+21:6523:x1874579665769088:12345-0@lo:101:mdt00_000.0 Request processed in 136us (413us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739073.200406:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3579 00000100:00100000:2.0:1787739073.200734:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652539776 00000100:00100000:2.0:1787739073.200739:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652539776 00000100:00100000:2.0:1787739073.200783:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1399 00000100:00100000:2.0:1787739073.200791:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042630c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:2408:x1874579652539776:12345-192.168.203.15@tcp:103:rm.0 00010000:00010000:2.0:1787739073.200800:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739073.200812:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd400/0x7ee4453e4a1adf1d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1406:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09524f expref: 35 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.200825:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1406:0x0].0x0 (ffffa0730713a400) 00010000:00010000:2.0:1787739073.200835:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739073.201127:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042630c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+34:2408:x1874579652539776:12345-192.168.203.15@tcp:103:rm.0 Request processed in 93us (2081us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739073.201135:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1399 00000100:00100000:3.0:1787739073.201271:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665769088:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739073.201288:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae751 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae758 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.201299:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae751 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae758 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739073.201317:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739073.201344:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1ae75f lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x23e4280e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.201351:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1ae75f lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.201358:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1ae75f lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739073.201371:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665769216:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739073.201402:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739073.201461:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739073.202160:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665769216 00000100:00100000:2.0:1787739073.202167:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665769216 00000100:00100000:2.0:1787739073.202216:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3580 00000100:00100000:2.0:1787739073.202225:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+21:6523:x1874579665769216:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739073.202237:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739073.202252:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae766 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x23e4280e bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae75f expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.202270:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae766 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae75f expref: 22 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.202286:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae766 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae75f expref: 22 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.202293:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x23e4280e (ffffa074419d5d00) 00010000:00010000:2.0:1787739073.202295:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739073.202298:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa1a00, rc 0) 00000100:00100000:2.0:1787739073.202349:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+22:6523:x1874579665769216:12345-0@lo:101:mdt00_000.0 Request processed in 126us (945us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739073.202355:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3580 00000100:00100000:3.0:1787739073.203513:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665769216:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739073.203535:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1ae75f lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae766 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.203547:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1ae75f lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae766 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739073.203567:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:3.0:1787739073.203614:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665769344:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739073.203659:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739073.203752:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739073.204254:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665769344 00000100:00100000:1.0:1787739073.204261:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665769344 00000100:00100000:1.0:1787739073.204345:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3581 00000100:00100000:1.0:1787739073.204354:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+22:6523:x1874579665769344:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1787739073.204456:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+22:6523:x1874579665769344:12345-0@lo:1000:mdt00_000.0 Request processed in 103us (797us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739073.204463:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3581 00000100:00100000:3.0:1787739073.204543:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665769344:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739073.204586:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665769472:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1787739073.204613:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739073.204929:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739073.205004:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665769472 00000100:00100000:2.0:1787739073.205008:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665769472 00000100:00100000:2.0:1787739073.205032:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3582 00000100:00100000:2.0:1787739073.205037:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+22:6523:x1874579665769472:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1787739073.205395:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+22:6523:x1874579665769472:12345-0@lo:1000:mdt00_000.0 Request processed in 356us (780us total) trans 0 rc 1/1 00000100:00100000:2.0:1787739073.205402:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3582 00000100:00100000:3.0:1787739073.205526:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665769472:0@lo:1000:mdt00_000.0 00010000:00010000:3.0:1787739073.205581:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1ae76d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.205590:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1ae76d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.205599:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1ae76d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739073.205610:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665769600:0@lo:101:mdt00_000.0 00000100:00100000:3.0:1787739073.205635:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739073.205665:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739073.205701:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665769600 00000100:00100000:2.0:1787739073.205705:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665769600 00000100:00100000:2.0:1787739073.205744:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3583 00000100:00100000:2.0:1787739073.205751:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+22:6523:x1874579665769600:12345-0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739073.205760:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739073.205777:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1ae774 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae76d expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.205799:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1ae774 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae76d expref: 23 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.205819:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1ae774 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae76d expref: 23 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.205826:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1406:0x0].0x0 (ffffa074419d5700) 00010000:00010000:2.0:1787739073.205829:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739073.205832:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa1200, rc 0) 00000100:00100000:2.0:1787739073.205868:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+23:6523:x1874579665769600:12345-0@lo:101:mdt00_000.0 Request processed in 118us (232us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739073.205874:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3583 00000100:00100000:3.0:1787739073.208602:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665769600:0@lo:101:mdt00_000.0 00010000:00010000:3.0:1787739073.208634:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1ae76d lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae774 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.208650:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1ae76d lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae774 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.208692:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae77b lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.208702:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae77b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.208726:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae77b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.208750:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae782 lrc: 3/0,1 mode: --/EX res: [0x200000400:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.208762:0:6523:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adf39 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09525d expref: 61 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.208772:0:6523:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae782 lrc: 4/0,1 mode: --/EX res: [0x200000400:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.208806:0:6523:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adf39 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09525d expref: 61 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.208828:0:6523:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adf39 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09525d expref: 61 pid: 13793 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739073.208842:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665769728:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739073.208919:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739073.213123:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739073.213357:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652540160 00000100:00100000:2.0:1787739073.213366:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652540160 00010000:00010000:2.0:1787739073.213406:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adf39 lrc: 5/0,0 mode: PR/PR res: [0x200000400:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09525d expref: 62 pid: 13793 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739073.213486:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1590 00000100:00100000:2.0:1787739073.213496:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304260f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+62:2409:x1874579652540160:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739073.213511:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739073.213518:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adf39 lrc: 5/0,0 mode: PR/PR res: [0x200000400:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09525d expref: 62 pid: 13793 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.213532:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adf39 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09525d expref: 62 pid: 13793 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.213554:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x3e:0x0].0x0 (ffffa07403eebd00) 00010000:00010000:2.0:1787739073.213559:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae782 lrc: 3/0,1 mode: --/EX res: [0x200000400:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.213574:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae782 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.213582:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae782 lrc: 4/0,1 mode: EX/EX res: [0x200000400:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.213611:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739073.213666:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304260f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+62:2409:x1874579652540160:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 172us (553us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739073.213675:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1590 00010000:00010000:3.0:1787739073.215899:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c000/0x7ee4453e4a1adf39 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09525d expref: 61 pid: 13793 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739073.215922:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4f00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6523:1874579665769728:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739073.215956:0:6523:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae782 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.216058:0:6523:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae782 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.216066:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae782 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.216176:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae789 lrc: 3/0,1 mode: --/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.216185:0:6523:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae789 lrc: 3/0,1 mode: --/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.216196:0:6523:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae789 lrc: 3/0,1 mode: --/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739073.216217:0:6523:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665769856:0@lo:101:mdt00_000.0 00000100:00100000:2.0:1787739073.216278:0:6523:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739073.216346:0:6523:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740352a840 going to sleep for 11 seconds 00000100:00100000:0.0:1787739073.216966:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665769856 00000100:00100000:0.0:1787739073.216973:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665769856 00000100:00100000:0.0:1787739073.217073:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3584 00000100:00100000:0.0:1787739073.217092:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+23:6523:x1874579665769856:12345-0@lo:101:mdt00_000.0 00010000:00010000:0.0:1787739073.217107:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739073.217124:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae790 lrc: 2/0,0 mode: --/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae789 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.217151:0:12410:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adf86 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095287 expref: 33 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.217162:0:12410:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae790 lrc: 4/0,0 mode: --/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae789 expref: 24 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.217187:0:12410:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adf86 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095287 expref: 33 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.217201:0:12410:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adf86 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095287 expref: 33 pid: 7944 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739073.217217:0:12410:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001_UUID:12410:1874579665769984:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739073.217286:0:12410:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306248c00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739073.222624:0:12410:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001_UUID:12410:1874579665769984:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739073.222662:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae790 lrc: 3/0,0 mode: --/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae789 expref: 24 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.222676:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x3e:0x0].0x0 (ffffa0730713af00) 00010000:00010000:0.0:1787739073.222679:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1787739073.222683:0:12410:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae790 lrc: 3/0,0 mode: --/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae789 expref: 24 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.222693:0:12410:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae790 lrc: 3/0,0 mode: --/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae789 expref: 24 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.222701:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306088600, rc 0) 00000100:00100000:0.0:1787739073.222821:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+24:6523:x1874579665769856:12345-0@lo:101:mdt00_000.0 Request processed in 5731us (6542us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739073.222830:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3584 00000100:00100000:3.0:1787739073.224188:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739073.224678:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652540288 00000100:00100000:1.0:1787739073.224687:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652540288 00010000:00010000:1.0:1787739073.224733:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adf86 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095287 expref: 34 pid: 7944 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739073.224773:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1591 00000100:00100000:1.0:1787739073.224782:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+34:2409:x1874579652540288:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739073.224795:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739073.224801:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adf86 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095287 expref: 34 pid: 7944 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.224812:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adf86 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095287 expref: 34 pid: 7944 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.224833:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fda00/0x7ee4453e4a1adf86 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095287 expref: 34 pid: 7944 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.224849:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x3e:0x0].0x0 (ffffa0730713af00) 00010000:00010000:1.0:1787739073.224855:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae790 lrc: 2/0,0 mode: --/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae789 expref: 23 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.224865:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae790 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae789 expref: 23 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.224874:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae790 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae789 expref: 23 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.224907:0:10340:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae790 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae789 expref: 23 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739073.224920:0:10340:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f743c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665770112:0@lo:105: 00000100:00100000:1.0:1787739073.224977:0:10340:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739073.225016:0:10340:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739073.225737:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665770112 00000100:00100000:0.0:1787739073.225744:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665770112 00000100:00100000:0.0:1787739073.225783:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1211 00000100:00100000:0.0:1787739073.225789:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c41d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665770112:12345-0@lo:105: 00010000:00010000:0.0:1787739073.225800:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae789 lrc: 4/0,1 mode: --/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae790 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.225814:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae789 lrc: 4/0,1 mode: --/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae790 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.225821:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae789 lrc: 4/0,1 mode: --/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae790 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.225829:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae789 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae790 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.225836:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae789 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae790 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.225844:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae789 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ae790 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.225857:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07306aa0400) 00000100:00100000:0.0:1787739073.225912:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c41d2c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665770112:12345-0@lo:105: Request processed in 122us (933us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739073.225918:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1211 00000100:00100000:3.0:1787739073.226009:0:10340:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f743c0 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665770112:0@lo:105: 00010000:00010000:3.0:1787739073.226027:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739073.226084:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+33:2409:x1874579652540288:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 1294us (1892us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739073.226092:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1591 00000100:00100000:2.0:1787739073.227756:0:6523:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:6523:1874579665769856:0@lo:101:mdt00_000.0 00010000:00010000:2.0:1787739073.227779:0:6523:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae789 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae790 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1787739073.228269:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].398, 888 off600496 00000040:00080000:2.0:1787739073.228588:0:6523:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].397, 968 off650320 00000004:00080000:2.0:1787739073.228700:0:6523:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0740352a840 with version 751 00000004:00080000:1.0:1787739073.228860:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 751 rpc_version 751 00000020:00100000:1.0:1787739073.228894:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739073.228902:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074356ada40 x1874579665770368/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2208/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739073.228925:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ada40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665770368:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739073.229005:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739073.229085:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288300 going to sleep for 11 seconds 00000100:00100000:0.0:1787739073.229160:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665770368 00000100:00100000:0.0:1787739073.229166:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665770368 00000100:00100000:0.0:1787739073.229247:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3585 00000100:00100000:0.0:1787739073.229257:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410b08050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+24:7740:x1874579665770368:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739073.229669:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410b08050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+25:7740:x1874579665770368:12345-0@lo:1000:osp_up1-0.0 Request processed in 412us (662us total) trans 4294968462 rc 0/0 00000100:00100000:0.0:1787739073.229677:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3585 00000100:00100000:2.0:1787739073.230912:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ada40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665770368:0@lo:1000:osp_up1-0.0 00000004:00200000:3.0:1787739073.231010:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1406:0x0]: returning size 4096 00010000:00010000:3.0:1787739073.231034:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae782 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.231043:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae782 lrc: 2/0,0 mode: EX/EX res: [0x200000400:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.231051:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae782 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.231059:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739073.231063:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae782 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.231071:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae782 lrc: 4/0,0 mode: EX/EX res: [0x200000400:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.231133:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x3e:0x0].0x0 (ffffa07403eebd00) 00010000:00010000:3.0:1787739073.231136:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739073.231141:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae782 lrc: 2/0,0 mode: --/EX res: [0x200000400:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.231147:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae782 lrc: 0/0,0 mode: --/EX res: [0x200000400:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.231167:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae789 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae790 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.231175:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae789 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae790 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.231182:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae789 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae790 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.231235:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739073.231245:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x3e:0x0] 00010000:00010000:3.0:1787739073.231254:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae77b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.231261:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae77b lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.231267:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae77b lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.231271:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739073.231274:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae77b lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.231279:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae77b lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.231287:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1406:0x0].0x0 (ffffa0730608ba00) 00010000:00010000:3.0:1787739073.231290:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739073.231293:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae77b lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.231299:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae77b lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.231311:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1ae76d lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae774 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.231317:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1ae76d lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae774 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.231323:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1ae76d lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae774 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.231353:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1787739073.231360:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae789 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae790 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.231366:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739073.231370:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae789 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae790 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739073.231378:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae789 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae790 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.231408:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae789 lrc: 2/0,0 mode: --/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae790 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.231549:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1787739073.231602:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665770496:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739073.231656:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739073.231725:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665770496 00000100:00100000:0.0:1787739073.231731:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665770496 00000100:00100000:0.0:1787739073.231776:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1400 00000100:00100000:0.0:1787739073.231784:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+25:3652:x1874579665770496:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739073.231797:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739073.231813:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088600/0x7ee4453e4a1ae790 lrc: 0/0,0 mode: --/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae789 expref: 25 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.231825:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x3e:0x0].0x0 (ffffa0730713af00) 00010000:00010000:0.0:1787739073.231833:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739073.231880:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae789 lrc: 1/0,0 mode: --/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae790 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.231888:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0400/0x7ee4453e4a1ae789 lrc: 0/0,0 mode: --/EX res: [0x240000401:0x3e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae790 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.231902:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739073.231916:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1ae76d lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae774 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.231921:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa400 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739073.231925:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1ae76d lrc: 5/0,0 mode: EX/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae774 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739073.231931:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1ae76d lrc: 5/0,0 mode: EX/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae774 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.231958:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1ae76d lrc: 4/0,0 mode: --/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae774 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.231964:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739073.232485:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00000004:00080000:3.0:1787739073.232502:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739073.232508:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae751 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae758 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.232516:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae751 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae758 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.232523:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae751 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae758 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.232537:0:6523:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739073.232540:0:6523:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739073.232545:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1ae75f lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae766 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.232552:0:6523:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1ae75f lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae766 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739073.232624:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042621c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:17726:x1874579652540032:12345-192.168.203.15@tcp:36:rm.0 Request processed in 32898us (33767us total) trans 4294983482 rc 0/0 00000100:00100000:3.0:1787739073.232632:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23683 00000100:00100000:0.0:1787739073.232854:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+24:3652:x1874579665770496:12345-0@lo:103:ldlm_bl.0 Request processed in 1069us (1197us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739073.232861:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1400 00000100:00100000:2.0:1787739073.232914:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665770496:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739073.232932:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1787739073.232939:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356afc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665770624:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739073.232978:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739073.233015:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665770624 00000100:00100000:2.0:1787739073.233020:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665770624 00010000:00010000:1.0:1787739073.233712:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1ae76d lrc: 1/0,0 mode: --/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae774 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.233800:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aa400/0x7ee4453e4a1ae76d lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae774 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.233820:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739073.233824:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae751 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae758 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.233831:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739073.233836:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae751 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae758 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739073.233844:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae751 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae758 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.233873:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae751 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae758 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.233880:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739073.233897:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae751 lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae758 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.233909:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1ae751 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae758 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.233921:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739073.233922:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739073.233924:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739073.235194:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1401 00000100:00100000:2.0:1787739073.235213:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+24:3652:x1874579665770624:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739073.235227:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739073.235248:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1ae774 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1406:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae76d expref: 24 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.235262:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1406:0x0].0x0 (ffffa074419d5700) 00010000:00010000:2.0:1787739073.235272:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739073.235321:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+23:3652:x1874579665770624:12345-0@lo:103:ldlm_bl.0 Request processed in 116us (2342us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739073.235328:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1401 00010000:00010000:1.0:1787739073.235630:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739073.235636:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739073.238529:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665770752:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739073.238737:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739073.238925:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356afc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665770624:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739073.238992:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739073.239307:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665770752 00000100:00100000:1.0:1787739073.239315:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665770752 00000100:00100000:1.0:1787739073.239381:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1402 00000100:00100000:1.0:1787739073.239391:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be43c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+23:3650:x1874579665770752:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739073.239407:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739073.239461:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088800/0x7ee4453e4a1ae758 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae751 expref: 23 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.239481:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa07305ca2100) 00010000:00010000:1.0:1787739073.239490:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739073.239539:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be43c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+22:3650:x1874579665770752:12345-0@lo:103:ldlm_bl.0 Request processed in 150us (803us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739073.239547:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1402 00000100:00100000:2.0:1787739073.239696:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652540544 00000100:00100000:2.0:1787739073.239701:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652540544 00000100:00100000:2.0:1787739073.239747:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23684 00000100:00100000:2.0:1787739073.239754:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be5e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+33:17726:x1874579652540544:12345-192.168.203.15@tcp:101:rm.0 00010000:00010000:2.0:1787739073.239769:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739073.239783:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae797 lrc: 2/0,0 mode: --/CR res: [0x240000402:0xe0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0954be expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739073.239914:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xe0:0x0]: returning size 4096 00010000:00010000:2.0:1787739073.240842:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae797 lrc: 1/0,0 mode: --/CR res: [0x240000402:0xe0:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954be expref: 35 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.240857:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe0:0x0].0x0 (ffffa0730608b700) 00010000:00010000:2.0:1787739073.240861:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739073.240866:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1000/0x7ee4453e4a1ae797 lrc: 0/0,0 mode: --/CR res: [0x240000402:0xe0:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954be expref: 35 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.240878:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa1000, rc 0) 00000100:00100000:2.0:1787739073.240936:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be5e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+34:17726:x1874579652540544:12345-192.168.203.15@tcp:101:rm.0 Request processed in 1182us (1944us total) trans 4294968463 rc 301/301 00000100:00100000:2.0:1787739073.240957:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23684 00000100:00100000:1.0:1787739073.241693:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665770752:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739073.257811:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739073.257904:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652540800 00000100:00100000:0.0:1787739073.257913:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652540800 00000100:00100000:0.0:1787739073.258019:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5069 00000100:00100000:0.0:1787739073.258029:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ae580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+34:17726:x1874579652540800:12345-192.168.203.15@tcp:35:rm.0 00000100:00100000:0.0:1787739073.258267:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ae580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:17726:x1874579652540800:12345-192.168.203.15@tcp:35:rm.0 Request processed in 240us (465us total) trans 4294968464 rc 0/0 00000100:00100000:0.0:1787739073.258275:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5069 00000100:00100000:0.0:1787739073.262566:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739073.262608:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739073.262754:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652540928 00000100:00100000:3.0:1787739073.262764:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652540928 00000100:00100000:3.0:1787739073.262833:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1403 00000100:00100000:3.0:1787739073.262844:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba1c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:2408:x1874579652540928:12345-192.168.203.15@tcp:103:rm.0 00010000:00010000:3.0:1787739073.262860:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1787739073.262873:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652541184 00000100:00100000:2.0:1787739073.262878:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652541184 00010000:00010000:3.0:1787739073.262886:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07441a1c000/0x7ee4453e4a1ae090 lrc: 0/0,0 mode: --/PR res: [0x240000402:0xe0:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952f0 expref: 61 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.262904:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe0:0x0].0x0 (ffffa0730713ae00) 00010000:00010000:3.0:1787739073.262915:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739073.262962:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23685 00000100:00100000:2.0:1787739073.262970:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306391680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:17726:x1874579652541184:12345-192.168.203.15@tcp:36:rm.0 00010000:00010000:2.0:1787739073.263019:0:6525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739073.263034:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1ae0ac lrc: 0/0,0 mode: --/PR res: [0x240000402:0xe0:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952fe expref: 35 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.263052:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe0:0x0].0x0 (ffffa0730608b700) 00010000:00010000:2.0:1787739073.263059:0:6525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739073.263121:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1000/0x7ee4453e4a1ae7a5 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.263130:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1000/0x7ee4453e4a1ae7a5 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.263139:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1000/0x7ee4453e4a1ae7a5 lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739073.263161:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252b0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665770880:0@lo:101:mdt00_002.0 00000100:00100000:2.0:1787739073.263202:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739073.263209:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba1c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:2408:x1874579652540928:12345-192.168.203.15@tcp:103:rm.0 Request processed in 367us (652us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739073.263216:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1403 00000100:00100000:2.0:1787739073.263231:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074078cb240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739073.263313:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665770880 00000100:00100000:0.0:1787739073.263317:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665770880 00000100:00100000:0.0:1787739073.263380:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3586 00000100:00100000:0.0:1787739073.263388:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+19:6525:x1874579665770880:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787739073.263400:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739073.263441:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae7ac lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae7a5 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.263465:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae7ac lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae7a5 expref: 20 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.263477:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae7ac lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae7a5 expref: 20 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.263485:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305ca2500) 00010000:00010000:0.0:1787739073.263488:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739073.263490:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306088600, rc 0) 00000100:00100000:0.0:1787739073.263527:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+20:6525:x1874579665770880:12345-0@lo:101:mdt00_002.0 Request processed in 140us (323us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739073.263543:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3586 00000100:00100000:3.0:1787739073.263822:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252b0c0 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665770880:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739073.263837:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1000/0x7ee4453e4a1ae7a5 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae7ac expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.263847:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1000/0x7ee4453e4a1ae7a5 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae7ac expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739073.263865:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:3.0:1787739073.263886:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ae7b3 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x98a090de bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.263892:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ae7b3 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x98a090de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.263898:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ae7b3 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x98a090de bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739073.263910:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4f00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665771008:0@lo:101:mdt00_002.0 00000100:00100000:3.0:1787739073.263934:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739073.264506:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665771008 00000100:00100000:0.0:1787739073.264513:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665771008 00000100:00100000:3.0:1787739073.264520:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e000 going to sleep for 11 seconds 00000100:00100000:0.0:1787739073.264640:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3587 00000100:00100000:0.0:1787739073.264648:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be43c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+20:6525:x1874579665771008:12345-0@lo:101:mdt00_002.0 00010000:00010000:0.0:1787739073.264659:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739073.264673:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088000/0x7ee4453e4a1ae7ba lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x98a090de bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae7b3 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.264692:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088000/0x7ee4453e4a1ae7ba lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x98a090de bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae7b3 expref: 21 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.264704:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088000/0x7ee4453e4a1ae7ba lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x98a090de bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae7b3 expref: 21 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.264730:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x98a090de (ffffa07305ca2800) 00010000:00010000:0.0:1787739073.264734:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739073.264737:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306088000, rc 0) 00000100:00100000:0.0:1787739073.264779:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be43c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+21:6525:x1874579665771008:12345-0@lo:101:mdt00_002.0 Request processed in 132us (843us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739073.264786:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3587 00000100:00100000:3.0:1787739073.264807:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4f00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665771008:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739073.264819:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ae7b3 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x98a090de bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae7ba expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.264827:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ae7b3 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x98a090de bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae7ba expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739073.264842:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00000100:00100000:3.0:1787739073.264874:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4f00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665771136:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1787739073.264903:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739073.264917:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739073.265032:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665771136 00000100:00100000:1.0:1787739073.265037:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665771136 00000100:00100000:1.0:1787739073.265069:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3588 00000100:00100000:1.0:1787739073.265165:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+21:6525:x1874579665771136:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1787739073.265267:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+21:6525:x1874579665771136:12345-0@lo:1000:mdt00_002.0 Request processed in 191us (362us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739073.265275:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3588 00000100:00100000:3.0:1787739073.265598:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4f00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665771136:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739073.265691:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665771264:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1787739073.265800:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739073.265844:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074078cb480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739073.265868:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665771264 00000100:00100000:0.0:1787739073.265871:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665771264 00000100:00100000:0.0:1787739073.265896:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3589 00000100:00100000:0.0:1787739073.265902:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+21:6525:x1874579665771264:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1787739073.266036:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+21:6525:x1874579665771264:12345-0@lo:1000:mdt00_002.0 Request processed in 132us (234us total) trans 0 rc 1/1 00000100:00100000:0.0:1787739073.266043:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3589 00000100:00100000:2.0:1787739073.266143:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665771264:0@lo:1000:mdt00_002.0 00010000:00010000:2.0:1787739073.266186:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1200/0x7ee4453e4a1ae7c1 lrc: 3/0,1 mode: --/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.266192:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1200/0x7ee4453e4a1ae7c1 lrc: 3/0,1 mode: --/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.266198:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1200/0x7ee4453e4a1ae7c1 lrc: 3/0,1 mode: --/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739073.266205:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665771392:0@lo:101:mdt00_002.0 00000100:00100000:2.0:1787739073.266223:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739073.266240:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074078cbf00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739073.266260:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665771392 00000100:00100000:1.0:1787739073.266265:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665771392 00000100:00100000:1.0:1787739073.266295:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3590 00000100:00100000:1.0:1787739073.266301:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+21:6525:x1874579665771392:12345-0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787739073.266310:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739073.266324:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1ae7c8 lrc: 2/0,0 mode: --/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae7c1 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.266350:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1ae7c8 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae7c1 expref: 22 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.266360:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1ae7c8 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae7c1 expref: 22 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.266365:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe0:0x0].0x0 (ffffa0730713af00) 00010000:00010000:1.0:1787739073.266368:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739073.266370:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841000, rc 0) 00000100:00100000:1.0:1787739073.266397:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+22:6525:x1874579665771392:12345-0@lo:101:mdt00_002.0 Request processed in 96us (172us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739073.266403:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3590 00000100:00100000:2.0:1787739073.266437:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665771392:0@lo:101:mdt00_002.0 00010000:00010000:2.0:1787739073.266449:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1200/0x7ee4453e4a1ae7c1 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae7c8 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.266459:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1200/0x7ee4453e4a1ae7c1 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae7c8 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.266481:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae7cf lrc: 3/0,1 mode: --/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.266490:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae7cf lrc: 3/0,1 mode: EX/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.266496:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae7cf lrc: 3/0,1 mode: EX/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.266509:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae7d6 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x76:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.266518:0:6525:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ae0ba lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x76:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095305 expref: 34 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.266523:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae7d6 lrc: 4/0,1 mode: --/EX res: [0x240000400:0x76:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.266664:0:6525:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ae0ba lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x76:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095305 expref: 34 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.266684:0:6525:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ae0ba lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x76:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095305 expref: 34 pid: 6523 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739073.266696:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665771520:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739073.266882:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074078cbf00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739073.270320:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001_UUID:6525:1874579665771520:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739073.270362:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae7d6 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x76:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739073.271294:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739073.271369:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652541312 00000100:00100000:2.0:1787739073.271377:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652541312 00010000:00010000:2.0:1787739073.271441:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ae0ba lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x76:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095305 expref: 35 pid: 6523 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739073.271481:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1592 00000100:00100000:2.0:1787739073.271491:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356afc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+35:2409:x1874579652541312:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739073.271505:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739073.271512:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ae0ba lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x76:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095305 expref: 35 pid: 6523 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.271524:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ae0ba lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x76:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095305 expref: 35 pid: 6523 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.271556:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07441a1cc00/0x7ee4453e4a1ae0ba lrc: 0/0,0 mode: --/PR res: [0x240000400:0x76:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095305 expref: 35 pid: 6523 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.271577:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x76:0x0].0x0 (ffffa0730713a200) 00010000:00010000:2.0:1787739073.271582:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae7d6 lrc: 3/0,1 mode: --/EX res: [0x240000400:0x76:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.271592:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae7d6 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x76:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.271599:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae7d6 lrc: 4/0,1 mode: EX/EX res: [0x240000400:0x76:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.271622:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739073.271668:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356afc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+34:2409:x1874579652541312:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 179us (377us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739073.271676:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1592 00010000:00010000:0.0:1787739073.271843:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae7d6 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.271851:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae7d6 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.271894:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089600/0x7ee4453e4a1ae7dd lrc: 3/0,1 mode: --/EX res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.271901:0:6525:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089600/0x7ee4453e4a1ae7dd lrc: 3/0,1 mode: --/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.271910:0:6525:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089600/0x7ee4453e4a1ae7dd lrc: 3/0,1 mode: --/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739073.271929:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306393480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665771648:0@lo:101:mdt00_002.0 00000100:00100000:0.0:1787739073.271982:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739073.272016:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306248840 going to sleep for 11 seconds 00000100:00100000:1.0:1787739073.272208:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665771648 00000100:00100000:1.0:1787739073.272213:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665771648 00000100:00100000:1.0:1787739073.272246:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3591 00000100:00100000:1.0:1787739073.272252:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073063903c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+22:6525:x1874579665771648:12345-0@lo:101:mdt00_002.0 00010000:00010000:1.0:1787739073.272265:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739073.272279:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1ae7e4 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x76:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae7dd expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.272301:0:12410:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ae0c8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09530c expref: 59 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.272310:0:12410:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1ae7e4 lrc: 4/0,0 mode: --/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae7dd expref: 23 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.272332:0:12410:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ae0c8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09530c expref: 59 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.272354:0:12410:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ae0c8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09530c expref: 59 pid: 6524 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739073.272365:0:12410:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae940 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:12410:1874579665771776:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739073.272402:0:12410:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288c00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739073.277302:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739073.277751:0:12410:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae940 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:12410:1874579665771776:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739073.277785:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1ae7e4 lrc: 3/0,0 mode: --/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae7dd expref: 23 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.277798:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffffa07305ca2000) 00010000:00010000:1.0:1787739073.277801:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739073.277805:0:12410:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1ae7e4 lrc: 3/0,0 mode: --/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae7dd expref: 23 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.277819:0:12410:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1ae7e4 lrc: 3/0,0 mode: --/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae7dd expref: 23 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.277826:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841800, rc 0) 00000100:00100000:1.0:1787739073.277881:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073063903c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+23:6525:x1874579665771648:12345-0@lo:101:mdt00_002.0 Request processed in 5628us (5897us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739073.277889:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3591 00000100:00100000:3.0:1787739073.278047:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306393480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665771648:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1787739073.278065:0:6525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089600/0x7ee4453e4a1ae7dd lrc: 4/0,1 mode: --/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae7e4 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.278301:0:6525:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089600/0x7ee4453e4a1ae7dd lrc: 4/0,1 mode: --/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae7e4 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739073.278768:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652541440 00000100:00100000:0.0:1787739073.278776:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652541440 00010000:00010000:0.0:1787739073.278810:0:6510:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ae0c8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09530c expref: 60 pid: 6524 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739073.278835:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1593 00000100:00100000:0.0:1787739073.278844:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be61c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:2408:x1874579652541440:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739073.278857:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739073.278864:0:6510:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ae0c8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09530c expref: 60 pid: 6524 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.278875:0:6510:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ae0c8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x76:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09530c expref: 60 pid: 6524 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.279030:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730609ba00/0x7ee4453e4a1ae0c8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x76:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09530c expref: 60 pid: 6524 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.279044:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffffa07305ca2000) 00010000:00010000:0.0:1787739073.279050:0:6510:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1ae7e4 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae7dd expref: 22 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.279060:0:6510:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1ae7e4 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae7dd expref: 22 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.279068:0:6510:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1ae7e4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae7dd expref: 22 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.279206:0:6510:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1ae7e4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae7dd expref: 22 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739073.279220:0:6510:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6510:1874579665771904:0@lo:105: 00000100:00100000:0.0:1787739073.279274:0:6510:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739073.279318:0:6510:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306248840 going to sleep for 11 seconds 00000100:00100000:2.0:1787739073.279396:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665771904 00000100:00100000:2.0:1787739073.279401:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665771904 00000100:00100000:2.0:1787739073.279468:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1212 00000100:00100000:2.0:1787739073.279476:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074080f1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6510:x1874579665771904:12345-0@lo:105: 00010000:00010000:2.0:1787739073.279485:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089600/0x7ee4453e4a1ae7dd lrc: 5/0,1 mode: --/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae7e4 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.279495:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089600/0x7ee4453e4a1ae7dd lrc: 5/0,1 mode: --/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae7e4 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.279503:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089600/0x7ee4453e4a1ae7dd lrc: 5/0,1 mode: --/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae7e4 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.279511:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089600/0x7ee4453e4a1ae7dd lrc: 5/0,1 mode: EX/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae7e4 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.279518:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089600/0x7ee4453e4a1ae7dd lrc: 6/0,1 mode: EX/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae7e4 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.279526:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089600/0x7ee4453e4a1ae7dd lrc: 6/0,1 mode: EX/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ae7e4 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.279541:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07306089600) 00000100:00100000:2.0:1787739073.279597:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074080f1680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:6510:x1874579665771904:12345-0@lo:105: Request processed in 114us (315us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739073.279604:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1212 00010000:00010000:3.0:1787739073.279729:0:6525:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089600/0x7ee4453e4a1ae7dd lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae7e4 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.279739:0:6525:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089600/0x7ee4453e4a1ae7dd lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae7e4 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739073.281046:0:6510:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306392580 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:6510:1874579665771904:0@lo:105: 00010000:00010000:0.0:1787739073.281064:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739073.281217:0:6525:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4f00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665772032:0@lo:403:mdt00_002.0 00000100:00100000:3.0:1787739073.281272:0:6525:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739073.281305:0:6525:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787739073.281393:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be61c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+59:2408:x1874579652541440:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 2548us (4098us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739073.281402:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1593 00000100:00100000:1.0:1787739073.281960:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665772032 00000100:00100000:1.0:1787739073.281968:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665772032 00000100:00100000:1.0:1787739073.282057:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3592 00000100:00100000:1.0:1787739073.282068:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306392580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+23:6525:x1874579665772032:12345-0@lo:403:mdt00_002.0 00000100:00100000:1.0:1787739073.285608:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306392580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+23:6525:x1874579665772032:12345-0@lo:403:mdt00_002.0 Request processed in 3534us (4327us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739073.285622:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3592 00000100:00100000:3.0:1787739073.294273:0:6525:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4f00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6525:1874579665772032:0@lo:403:mdt00_002.0 00000040:00080000:2.0:1787739073.295439:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].379, 952 off568512 00000040:00080000:2.0:1787739073.295620:0:6525:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].379, 1032 off620408 00000004:00080000:2.0:1787739073.295626:0:6525:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa074078cbf00 with version 718 00000004:00080000:0.0:1787739073.295973:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 718 rpc_version 718 00000020:00100000:0.0:1787739073.296006:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739073.296015:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074080f3840 x1874579665772288/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739073.296038:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074080f3840 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665772288:0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739073.296104:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739073.296140:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306248840 going to sleep for 11 seconds 00000100:00100000:1.0:1787739073.296207:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665772288 00000100:00100000:1.0:1787739073.296214:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665772288 00000100:00100000:1.0:1787739073.296263:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3593 00000100:00100000:1.0:1787739073.296273:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042630c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+23:7467:x1874579665772288:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739073.296608:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042630c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+24:7467:x1874579665772288:12345-0@lo:1000:osp_up0-1.0 Request processed in 336us (502us total) trans 4294983483 rc 0/0 00000100:00100000:1.0:1787739073.296616:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3593 00000100:00100000:0.0:1787739073.296806:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074080f3840 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665772288:0@lo:1000:osp_up0-1.0 00000004:00200000:2.0:1787739073.299319:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0xe0:0x0]: returning size 4096 00010000:00010000:2.0:1787739073.299352:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae7d6 lrc: 3/0,1 mode: EX/EX res: [0x240000400:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.299363:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae7d6 lrc: 2/0,0 mode: EX/EX res: [0x240000400:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.299372:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae7d6 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.299379:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0e00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739073.299383:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae7d6 lrc: 3/0,0 mode: EX/EX res: [0x240000400:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.299392:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae7d6 lrc: 4/0,0 mode: EX/EX res: [0x240000400:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.299408:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0x76:0x0].0x0 (ffffa0730713a200) 00010000:00010000:2.0:1787739073.299426:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739073.299432:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae7d6 lrc: 2/0,0 mode: --/EX res: [0x240000400:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.299439:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae7d6 lrc: 0/0,0 mode: --/EX res: [0x240000400:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.299462:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089600/0x7ee4453e4a1ae7dd lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae7e4 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.299470:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089600/0x7ee4453e4a1ae7dd lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae7e4 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.299477:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089600/0x7ee4453e4a1ae7dd lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae7e4 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.299494:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739073.299504:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000401:0x76:0x0] 00010000:00010000:2.0:1787739073.299513:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae7cf lrc: 3/0,1 mode: EX/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.299518:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae7cf lrc: 2/0,0 mode: EX/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.299524:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae7cf lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.299530:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739073.299533:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae7cf lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.299539:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae7cf lrc: 4/0,0 mode: EX/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.299547:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe0:0x0].0x0 (ffffa074419d5a00) 00010000:00010000:2.0:1787739073.299550:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739073.299553:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae7cf lrc: 2/0,0 mode: --/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.299559:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae7cf lrc: 0/0,0 mode: --/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.299565:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739073.299572:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1200/0x7ee4453e4a1ae7c1 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae7c8 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.299575:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089600/0x7ee4453e4a1ae7dd lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae7e4 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.299588:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1200/0x7ee4453e4a1ae7c1 lrc: 4/0,1 mode: EX/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae7c8 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.299593:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306089600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739073.299600:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1200/0x7ee4453e4a1ae7c1 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae7c8 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.299602:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089600/0x7ee4453e4a1ae7dd lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae7e4 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739073.299616:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089600/0x7ee4453e4a1ae7dd lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae7e4 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.299628:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739073.299640:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:2.0:1787739073.299644:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1000/0x7ee4453e4a1ae7a5 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae7ac expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.299651:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089600/0x7ee4453e4a1ae7dd lrc: 2/0,0 mode: --/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae7e4 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.299653:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1000/0x7ee4453e4a1ae7a5 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae7ac expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.299670:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739073.299672:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1000/0x7ee4453e4a1ae7a5 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae7ac expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.299687:0:6525:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:2.0:1787739073.299691:0:6525:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0x5:0x0] 00010000:00010000:0.0:1787739073.299702:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089600/0x7ee4453e4a1ae7dd lrc: 1/0,0 mode: --/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae7e4 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.299711:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089600/0x7ee4453e4a1ae7dd lrc: 0/0,0 mode: --/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae7e4 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.299725:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787739073.299729:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1200/0x7ee4453e4a1ae7c1 lrc: 2/0,0 mode: EX/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae7c8 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.299735:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739073.299738:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1200/0x7ee4453e4a1ae7c1 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae7c8 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739073.299745:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1200/0x7ee4453e4a1ae7c1 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae7c8 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.299761:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1200/0x7ee4453e4a1ae7c1 lrc: 2/0,0 mode: --/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae7c8 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.299766:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739073.299783:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1200/0x7ee4453e4a1ae7c1 lrc: 1/0,0 mode: --/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae7c8 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.299791:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1200/0x7ee4453e4a1ae7c1 lrc: 0/0,0 mode: --/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae7c8 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.299806:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739073.299807:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739073.299811:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739073.299815:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1000/0x7ee4453e4a1ae7a5 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae7ac expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.299815:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739073.299825:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1000 already unused, calling callback (ffffffffc18e6ee0) 00000100:00100000:0.0:1787739073.299826:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00010000:00010000:1.0:1787739073.299831:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1000/0x7ee4453e4a1ae7a5 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae7ac expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739073.299839:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1000/0x7ee4453e4a1ae7a5 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae7ac expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739073.299847:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304262940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665772544:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739073.299862:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1000/0x7ee4453e4a1ae7a5 lrc: 2/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae7ac expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.299870:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739073.299879:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1000/0x7ee4453e4a1ae7a5 lrc: 1/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae7ac expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.299885:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306aa1000/0x7ee4453e4a1ae7a5 lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae7ac expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.299897:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739073.299898:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739073.299900:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739073.299907:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787739073.299917:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304260f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665772416:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739073.299927:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739073.299963:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739073.299986:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665772544 00000100:00100000:0.0:1787739073.299993:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665772544 00000100:00100000:0.0:1787739073.300048:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665772416 00000100:00100000:0.0:1787739073.300052:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665772416 00000100:00100000:0.0:1787739073.300076:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1404 00000100:00100000:0.0:1787739073.300084:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+25:3653:x1874579665772544:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739073.300097:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739073.300112:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1ae7c8 lrc: 0/0,0 mode: --/EX res: [0x240000402:0xe0:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae7c1 expref: 25 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.300125:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0xe0:0x0].0x0 (ffffa0730713af00) 00010000:00010000:0.0:1787739073.300132:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739073.300162:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+24:3653:x1874579665772544:12345-0@lo:103:ldlm_bl.0 Request processed in 79us (236us total) trans 0 rc 0/0 00010000:00010000:2.0:1787739073.300222:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ae7b3 lrc: 4/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x98a090de bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae7ba expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.300231:0:6525:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ae7b3 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x98a090de bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae7ba expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739073.300275:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1404 00000100:00100000:2.0:1787739073.301363:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306391680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+34:17726:x1874579652541184:12345-192.168.203.15@tcp:36:rm.0 Request processed in 38387us (38747us total) trans 4294968465 rc 0/0 00000100:00100000:2.0:1787739073.301373:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23685 00010000:00010000:2.0:1787739073.301721:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739073.303595:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1405 00000100:00100000:3.0:1787739073.303621:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356af480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+23:3650:x1874579665772416:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739073.303645:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739073.303678:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1ae7e4 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x76:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae7dd expref: 23 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.306042:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x76:0x0].0x0 (ffffa07305ca2000) 00010000:00010000:3.0:1787739073.306071:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739073.306326:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356af480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+22:3650:x1874579665772416:12345-0@lo:103:ldlm_bl.0 Request processed in 2716us (6359us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739073.306338:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1405 00000100:00100000:3.0:1787739073.306375:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1787739073.306392:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665772672:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739073.313084:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304260f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665772416:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739073.314011:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739073.314186:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665772672 00000100:00100000:2.0:1787739073.314202:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665772672 00000100:00100000:2.0:1787739073.314259:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1406 00000100:00100000:2.0:1787739073.314268:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306393480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+22:3652:x1874579665772672:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739073.314283:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739073.314558:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae7ac lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae7a5 expref: 22 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.314578:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa07305ca2500) 00010000:00010000:2.0:1787739073.314589:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739073.315515:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665772672:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739073.315566:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306393480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+21:3652:x1874579665772672:12345-0@lo:103:ldlm_bl.0 Request processed in 1296us (1554us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739073.315576:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1406 00000100:00100000:1.0:1787739073.324074:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304262940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665772544:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739073.330047:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739073.331522:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652541696 00000100:00100000:3.0:1787739073.331537:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652541696 00000100:00100000:3.0:1787739073.331606:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23686 00000100:00100000:3.0:1787739073.331617:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+59:17726:x1874579652541696:12345-192.168.203.15@tcp:101:rm.0 00010000:00010000:3.0:1787739073.331640:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739073.331664:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae7eb lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1467:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0954c5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739073.331818:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1467:0x0]: returning size 4096 00010000:00010000:3.0:1787739073.331911:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae7eb lrc: 1/0,0 mode: --/CR res: [0x200000402:0x1467:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954c5 expref: 61 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.331927:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1467:0x0].0x0 (ffffa07403eeb300) 00010000:00010000:3.0:1787739073.331930:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739073.331936:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae7eb lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1467:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954c5 expref: 61 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.331960:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab400, rc 0) 00000100:00100000:3.0:1787739073.332030:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356acf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:17726:x1874579652541696:12345-192.168.203.15@tcp:101:rm.0 Request processed in 415us (1990us total) trans 4294983484 rc 301/301 00000100:00100000:3.0:1787739073.332038:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23686 00000100:00100000:0.0:1787739073.352023:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739073.352391:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652541952 00000100:00100000:2.0:1787739073.352402:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652541952 00000100:00100000:2.0:1787739073.352492:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5070 00000100:00100000:2.0:1787739073.352502:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:17726:x1874579652541952:12345-192.168.203.15@tcp:35:rm.0 00000040:00080000:2.0:1787739073.352762:0:6526:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].399, 344 off600840 00000040:00080000:2.0:1787739073.352807:0:6526:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].398, 424 off650744 00000004:00080000:2.0:1787739073.352812:0:6526:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07410b4d840 with version 752 00000004:00080000:1.0:1787739073.353628:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 752 rpc_version 752 00000020:00100000:1.0:1787739073.353664:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739073.353672:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074356ade00 x1874579665772928/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1304/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739073.353694:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ade00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665772928:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739073.353753:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739073.353806:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739073.353873:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665772928 00000100:00100000:2.0:1787739073.353878:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665772928 00000100:00100000:2.0:1787739073.353932:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3594 00000100:00100000:2.0:1787739073.353940:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+22:7740:x1874579665772928:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739073.354125:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+23:7740:x1874579665772928:12345-0@lo:1000:osp_up1-0.0 Request processed in 185us (371us total) trans 4294968466 rc 0/0 00000100:00100000:2.0:1787739073.354132:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3594 00000100:00100000:1.0:1787739073.354369:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ade00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665772928:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739073.354617:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:17726:x1874579652541952:12345-192.168.203.15@tcp:35:rm.0 Request processed in 2114us (2601us total) trans 4294983485 rc 0/0 00000100:00100000:2.0:1787739073.354625:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5070 00000100:00100000:0.0:1787739073.377108:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739073.377201:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739073.377488:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652542336 00000100:00100000:2.0:1787739073.377498:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652542336 00000100:00100000:2.0:1787739073.377546:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23687 00000100:00100000:2.0:1787739073.377555:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:17726:x1874579652542336:12345-192.168.203.15@tcp:36:rm.0 00010000:00010000:2.0:1787739073.377625:0:6524:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739073.377651:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c400/0x7ee4453e4a1adfbe lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1467:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09529c expref: 61 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.377671:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1467:0x0].0x0 (ffffa07403eeb300) 00010000:00010000:2.0:1787739073.377683:0:6524:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739073.377736:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae7f9 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.377744:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae7f9 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.377753:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae7f9 lrc: 3/0,1 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739073.377772:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730252bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665773056:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1787739073.377820:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739073.377877:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739073.378986:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665773056 00000100:00100000:3.0:1787739073.378996:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665773056 00000100:00100000:3.0:1787739073.379044:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3595 00000100:00100000:1.0:1787739073.379047:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652542080 00000100:00100000:3.0:1787739073.379053:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+23:6524:x1874579665773056:12345-0@lo:101:mdt00_001.0 00000100:00100000:1.0:1787739073.379054:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652542080 00010000:00010000:3.0:1787739073.379076:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739073.379095:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae800 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae7f9 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.379122:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae800 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae7f9 expref: 24 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.379136:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae800 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae7f9 expref: 24 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.379145:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b200) 00010000:00010000:3.0:1787739073.379148:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739073.379151:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab400, rc 0) 00000100:00100000:0.0:1787739073.379269:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1407 00000100:00100000:0.0:1787739073.379279:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432e940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+34:2408:x1874579652542080:12345-192.168.203.15@tcp:103:rm.0 00010000:00010000:0.0:1787739073.379290:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739073.379307:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1adfa2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1467:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09528e expref: 34 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.379323:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1467:0x0].0x0 (ffffa0730608b000) 00010000:00010000:0.0:1787739073.379331:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739073.379392:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432e940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+33:2408:x1874579652542080:12345-192.168.203.15@tcp:103:rm.0 Request processed in 115us (2292us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739073.379400:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1407 00000100:00100000:3.0:1787739073.381377:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+24:6524:x1874579665773056:12345-0@lo:101:mdt00_001.0 Request processed in 2318us (3548us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739073.381387:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3595 00000100:00100000:1.0:1787739073.382403:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730252bc00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665773056:0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739073.382815:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae7f9 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae800 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.382830:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae7f9 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae800 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739073.382853:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:1.0:1787739073.382942:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1ae807 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.382958:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1ae807 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.382968:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1ae807 lrc: 3/0,1 mode: --/PW res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739073.382991:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356aed00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665773184:0@lo:101:mdt00_001.0 00000100:00100000:1.0:1787739073.383045:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739073.384367:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288c00 going to sleep for 11 seconds 00000100:00100000:0.0:1787739073.384853:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665773184 00000100:00100000:0.0:1787739073.384864:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665773184 00000100:00100000:0.0:1787739073.384967:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3596 00000100:00100000:0.0:1787739073.384977:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+24:6524:x1874579665773184:12345-0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739073.384994:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739073.385014:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae80e lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae807 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.385041:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae80e lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae807 expref: 25 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.385054:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae80e lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae807 expref: 25 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.385063:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x4ac2f730 (ffffa07305ca2b00) 00010000:00010000:0.0:1787739073.385067:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739073.385070:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306088a00, rc 0) 00000100:00100000:0.0:1787739073.385141:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ad680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+25:6524:x1874579665773184:12345-0@lo:101:mdt00_001.0 Request processed in 165us (2095us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739073.385148:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3596 00000100:00100000:1.0:1787739073.385229:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356aed00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665773184:0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739073.385246:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1ae807 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae80e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.385261:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1ae807 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae80e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739073.385282:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00000100:00100000:1.0:1787739073.385326:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356aed00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665773312:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1787739073.385360:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739073.386871:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288b40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739073.389105:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665773312 00000100:00100000:3.0:1787739073.389120:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665773312 00000100:00100000:3.0:1787739073.389197:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3597 00000100:00100000:3.0:1787739073.389207:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+25:6524:x1874579665773312:12345-0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1787739073.389348:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ad2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+25:6524:x1874579665773312:12345-0@lo:1000:mdt00_001.0 Request processed in 142us (3986us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739073.389355:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3597 00000100:00100000:1.0:1787739073.389655:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356aed00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665773312:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739073.390070:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356aed00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665773440:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1787739073.390135:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739073.390200:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062340c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739073.391754:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665773440 00000100:00100000:2.0:1787739073.391768:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665773440 00000100:00100000:2.0:1787739073.392042:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3598 00000100:00100000:2.0:1787739073.392051:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d843c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+25:6524:x1874579665773440:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1787739073.392192:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d843c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+25:6524:x1874579665773440:12345-0@lo:1000:mdt00_001.0 Request processed in 142us (2056us total) trans 0 rc 1/1 00000100:00100000:2.0:1787739073.392199:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3598 00000100:00100000:0.0:1787739073.392352:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356aed00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665773440:0@lo:1000:mdt00_001.0 00010000:00010000:0.0:1787739073.392498:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1ae815 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.392509:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1ae815 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.392520:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1ae815 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739073.392538:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d870c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665773568:0@lo:101:mdt00_001.0 00000100:00100000:0.0:1787739073.392576:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739073.392609:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062489c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739073.392740:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665773568 00000100:00100000:2.0:1787739073.392746:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665773568 00000100:00100000:2.0:1787739073.392779:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3599 00000100:00100000:2.0:1787739073.392985:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d87c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+25:6524:x1874579665773568:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739073.392997:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739073.393015:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae81c lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae815 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.393039:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae81c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae815 expref: 26 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.393059:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae81c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae815 expref: 26 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.393067:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1467:0x0].0x0 (ffffa074419d5900) 00010000:00010000:2.0:1787739073.393070:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739073.393074:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa0e00, rc 0) 00000100:00100000:2.0:1787739073.393119:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d87c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+26:6524:x1874579665773568:12345-0@lo:101:mdt00_001.0 Request processed in 333us (542us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739073.393126:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3599 00000100:00100000:0.0:1787739073.393201:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d870c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665773568:0@lo:101:mdt00_001.0 00010000:00010000:0.0:1787739073.393215:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1ae815 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae81c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.393225:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1ae815 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae81c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.393251:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae823 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.393260:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae823 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.393268:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae823 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.393287:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae82a lrc: 3/0,1 mode: --/EX res: [0x200000400:0x6e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.393301:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1adfcc lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952a3 expref: 60 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.393310:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae82a lrc: 4/0,1 mode: --/EX res: [0x200000400:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.393332:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1adfcc lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952a3 expref: 60 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.393345:0:6524:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1adfcc lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952a3 expref: 60 pid: 6525 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739073.393356:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d870c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665773696:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739073.393531:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062489c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739073.403612:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d870c0 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6524:1874579665773696:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739073.403646:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae82a lrc: 3/0,1 mode: --/EX res: [0x200000400:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739073.403880:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739073.404550:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652542464 00000100:00100000:3.0:1787739073.404559:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652542464 00010000:00010000:3.0:1787739073.406295:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1adfcc lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952a3 expref: 61 pid: 6525 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739073.406357:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1594 00000100:00100000:3.0:1787739073.406373:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba2450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:2408:x1874579652542464:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787739073.406389:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739073.406395:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1adfcc lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952a3 expref: 61 pid: 6525 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.406407:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1adfcc lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952a3 expref: 61 pid: 6525 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.406454:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1adfcc lrc: 0/0,0 mode: --/PR res: [0x200000400:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952a3 expref: 61 pid: 6525 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.406473:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x6e:0x0].0x0 (ffffa0730713a800) 00010000:00010000:3.0:1787739073.406478:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae82a lrc: 3/0,1 mode: --/EX res: [0x200000400:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.406488:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae82a lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.406496:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae82a lrc: 4/0,1 mode: EX/EX res: [0x200000400:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.406585:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739073.406657:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba2450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:2408:x1874579652542464:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 292us (2778us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739073.406665:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1594 00010000:00010000:2.0:1787739073.408362:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae82a lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.408377:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae82a lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.408444:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae831 lrc: 3/0,1 mode: --/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.410921:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae831 lrc: 3/0,1 mode: --/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.410959:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae831 lrc: 3/0,1 mode: --/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739073.410991:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665773824:0@lo:101:mdt00_001.0 00000100:00100000:2.0:1787739073.412015:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739073.412265:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665773824 00000100:00100000:3.0:1787739073.412273:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665773824 00000100:00100000:3.0:1787739073.412332:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3600 00000100:00100000:3.0:1787739073.412341:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+26:6524:x1874579665773824:12345-0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787739073.412359:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739073.412380:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ae838 lrc: 2/0,0 mode: --/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae831 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.412409:0:12410:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adfda lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952aa expref: 32 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.412460:0:12410:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ae838 lrc: 4/0,0 mode: --/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae831 expref: 27 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.412495:0:12410:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adfda lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952aa expref: 32 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.412508:0:12410:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adfda lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952aa expref: 32 pid: 7944 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739073.412521:0:12410:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306393480 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001_UUID:12410:1874579665773952:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739073.412700:0:12410:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739073.413739:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4dd80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739073.419026:0:12410:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306393480 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001_UUID:12410:1874579665773952:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739073.419065:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ae838 lrc: 3/0,0 mode: --/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae831 expref: 27 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.419145:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x6e:0x0].0x0 (ffffa07403eeb200) 00010000:00010000:3.0:1787739073.419149:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739073.419153:0:12410:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ae838 lrc: 3/0,0 mode: --/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae831 expref: 27 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.419162:0:12410:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ae838 lrc: 3/0,0 mode: --/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae831 expref: 27 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.419240:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aac00, rc 0) 00000100:00100000:3.0:1787739073.419303:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+27:6524:x1874579665773824:12345-0@lo:101:mdt00_001.0 Request processed in 6963us (7291us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739073.419313:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3600 00000100:00100000:2.0:1787739073.419455:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665773824:0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739073.419474:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae831 lrc: 4/0,1 mode: --/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae838 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.419485:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae831 lrc: 4/0,1 mode: --/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae838 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739073.419717:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739073.419781:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652542592 00000100:00100000:2.0:1787739073.419788:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652542592 00010000:00010000:2.0:1787739073.419828:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adfda lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952aa expref: 33 pid: 7944 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739073.419859:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1595 00000100:00100000:2.0:1787739073.419868:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356afc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+33:2409:x1874579652542592:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739073.419881:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739073.419887:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adfda lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952aa expref: 33 pid: 7944 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.419910:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adfda lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952aa expref: 33 pid: 7944 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.419931:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730633ce00/0x7ee4453e4a1adfda lrc: 0/0,0 mode: --/PR res: [0x240000401:0x6e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952aa expref: 33 pid: 7944 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.419953:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x6e:0x0].0x0 (ffffa07403eeb200) 00010000:00010000:2.0:1787739073.419960:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ae838 lrc: 2/0,0 mode: --/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae831 expref: 26 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.419973:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ae838 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae831 expref: 26 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.419982:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ae838 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae831 expref: 26 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.420012:0:10340:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ae838 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae831 expref: 26 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739073.420025:0:10340:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665774080:0@lo:105: 00000100:00100000:0.0:1787739073.420690:0:10340:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739073.420737:0:10340:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4dd80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739073.420918:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665774080 00000100:00100000:3.0:1787739073.420924:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665774080 00000100:00100000:3.0:1787739073.420975:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1213 00000100:00100000:3.0:1787739073.420983:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665774080:12345-0@lo:105: 00010000:00010000:3.0:1787739073.420994:0:6507:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae831 lrc: 5/0,1 mode: --/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae838 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.421003:0:6507:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae831 lrc: 5/0,1 mode: --/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae838 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.421010:0:6507:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae831 lrc: 5/0,1 mode: --/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae838 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.421017:0:6507:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae831 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae838 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.421024:0:6507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae831 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae838 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.421032:0:6507:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae831 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ae838 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.421206:0:6507:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07306aa1e00) 00010000:00010000:0.0:1787739073.421288:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae831 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae838 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.421303:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae831 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae838 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739073.421988:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d86940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665774208:0@lo:403:mdt00_001.0 00000100:00100000:0.0:1787739073.422038:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739073.423310:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665774080:12345-0@lo:105: Request processed in 2322us (2615us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739073.423321:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1213 00000100:00100000:0.0:1787739073.423491:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306248780 going to sleep for 11 seconds 00000100:00100000:0.0:1787739073.424651:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665774208 00000100:00100000:0.0:1787739073.424664:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665774208 00000100:00100000:1.0:1787739073.424813:0:10340:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665774080:0@lo:105: 00010000:00010000:1.0:1787739073.424842:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739073.424899:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356afc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+32:2409:x1874579652542592:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 5030us (5181us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739073.424908:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1595 00000100:00100000:0.0:1787739073.425093:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3601 00000100:00100000:0.0:1787739073.425105:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d861c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+27:6524:x1874579665774208:12345-0@lo:403:mdt00_001.0 00000100:00100000:0.0:1787739073.426863:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d861c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+27:6524:x1874579665774208:12345-0@lo:403:mdt00_001.0 Request processed in 1752us (4816us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739073.426878:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3601 00000100:00100000:1.0:1787739073.427006:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d86940 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665774208:0@lo:403:mdt00_001.0 00000040:00080000:1.0:1787739073.428005:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].400, 952 off601792 00000040:00080000:1.0:1787739073.428188:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].399, 1032 off651776 00000004:00080000:1.0:1787739073.428194:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306288b40 with version 753 00000004:00080000:3.0:1787739073.428831:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 753 rpc_version 753 00000020:00100000:3.0:1787739073.428869:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739073.428879:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306393480 x1874579665774464/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 2352/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739073.428905:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306393480 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665774464:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739073.428974:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739073.429685:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665774464 00000100:00100000:2.0:1787739073.429694:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665774464 00000100:00100000:0.0:1787739073.429835:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3602 00000100:00100000:0.0:1787739073.429848:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+27:7740:x1874579665774464:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739073.430507:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739073.430565:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306393480 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665774464:0@lo:1000:osp_up1-0.0 00000004:00200000:3.0:1787739073.430646:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1467:0x0]: returning size 4096 00010000:00010000:3.0:1787739073.430671:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae82a lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.430679:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae82a lrc: 2/0,0 mode: EX/EX res: [0x200000400:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.430686:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae82a lrc: 3/0,0 mode: EX/EX res: [0x200000400:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.430692:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306088600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739073.430697:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae82a lrc: 3/0,0 mode: EX/EX res: [0x200000400:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.430703:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae82a lrc: 4/0,0 mode: EX/EX res: [0x200000400:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.430718:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x6e:0x0].0x0 (ffffa0730713a800) 00010000:00010000:3.0:1787739073.430722:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739073.430725:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae82a lrc: 2/0,0 mode: --/EX res: [0x200000400:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.430731:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae82a lrc: 0/0,0 mode: --/EX res: [0x200000400:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.430751:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae831 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae838 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.430760:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae831 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae838 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.430767:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae831 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae838 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.430789:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739073.430797:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x6e:0x0] 00010000:00010000:3.0:1787739073.430806:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae823 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.430813:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae823 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.430818:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae823 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.430823:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306089600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739073.430825:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae823 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.430831:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae823 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.430838:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1467:0x0].0x0 (ffffa07305ca2900) 00010000:00010000:3.0:1787739073.430840:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739073.430843:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae823 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.430848:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae823 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.430860:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1ae815 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae81c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.430866:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1ae815 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae81c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.430872:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1ae815 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae81c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.430893:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00080000:3.0:1787739073.430901:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739073.430906:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae7f9 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae800 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.430912:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae7f9 lrc: 4/0,1 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae800 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.430918:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae7f9 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae800 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.430928:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000004:00080000:3.0:1787739073.430931:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:3.0:1787739073.430935:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1ae807 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae80e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.430952:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1ae807 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae80e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739073.431025:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:17726:x1874579652542336:12345-192.168.203.15@tcp:36:rm.0 Request processed in 53468us (53821us total) trans 4294983486 rc 0/0 00000100:00100000:3.0:1787739073.431034:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23687 00010000:00010000:1.0:1787739073.431096:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1787739073.431102:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae831 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae838 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.431115:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1e00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739073.431121:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae831 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae838 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739073.431129:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae831 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae838 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.431159:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae831 lrc: 2/0,0 mode: --/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae838 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.431167:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739073.431201:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae831 lrc: 1/0,0 mode: --/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae838 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.431208:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1ae831 lrc: 0/0,0 mode: --/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae838 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.431221:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1787739073.431224:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1ae815 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae81c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.431229:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306088e00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739073.431233:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1ae815 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae81c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739073.431240:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1ae815 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae81c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.431260:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1ae815 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae81c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.431267:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739073.431272:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1ae815 lrc: 1/0,0 mode: --/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae81c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.431279:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1ae815 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae81c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.431289:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739073.431292:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae7f9 lrc: 2/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae800 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.431298:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1600 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739073.431302:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae7f9 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae800 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739073.431308:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae7f9 lrc: 3/0,0 mode: CW/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae800 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.431323:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae7f9 lrc: 2/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae800 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.431329:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739073.431341:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae7f9 lrc: 1/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae800 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.431611:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1600/0x7ee4453e4a1ae7f9 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae800 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.431624:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739073.431625:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739073.431627:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739073.431634:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739073.431641:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787739073.431654:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665774848:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739073.431703:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739073.431783:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+28:7740:x1874579665774464:12345-0@lo:1000:osp_up1-0.0 Request processed in 500us (1369us total) trans 4294968467 rc 0/0 00000100:00100000:0.0:1787739073.431798:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3602 00010000:00010000:0.0:1787739073.431826:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739073.431839:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665774848 00000100:00100000:0.0:1787739073.431846:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665774848 00000100:00100000:3.0:1787739073.432750:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356af480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665774592:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739073.432801:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739073.432974:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356aed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665774720:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739073.433789:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739073.433981:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665774592 00000100:00100000:0.0:1787739073.433993:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665774592 00000100:00100000:0.0:1787739073.434042:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1408 00000100:00100000:0.0:1787739073.434051:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305654b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+29:3653:x1874579665774848:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739073.434068:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739073.434092:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae800 lrc: 0/0,0 mode: --/CW res: [0x240000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae7f9 expref: 29 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.434109:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa0730608b200) 00010000:00010000:0.0:1787739073.434119:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739073.434256:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665774720 00000100:00100000:2.0:1787739073.434261:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665774720 00000100:00100000:2.0:1787739073.434286:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1409 00000100:00100000:2.0:1787739073.434292:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730432e940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+29:3650:x1874579665774592:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739073.434300:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739073.434312:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aac00/0x7ee4453e4a1ae838 lrc: 0/0,0 mode: --/EX res: [0x240000401:0x6e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae831 expref: 29 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.434323:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x6e:0x0].0x0 (ffffa07403eeb200) 00010000:00010000:2.0:1787739073.434329:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739073.434530:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305654b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+28:3653:x1874579665774848:12345-0@lo:103:ldlm_bl.0 Request processed in 478us (2824us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739073.434537:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1408 00000100:00100000:0.0:1787739073.434557:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1410 00000100:00100000:0.0:1787739073.434563:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be61c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+27:3651:x1874579665774720:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739073.434571:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739073.434583:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae81c lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1467:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae815 expref: 27 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.434593:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1467:0x0].0x0 (ffffa074419d5900) 00010000:00010000:0.0:1787739073.434599:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739073.434630:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be61c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+26:3651:x1874579665774720:12345-0@lo:103:ldlm_bl.0 Request processed in 67us (844us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739073.434637:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1410 00000100:00100000:0.0:1787739073.434679:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356aed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665774720:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739073.434816:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665774848:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739073.434931:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730432e940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+25:3650:x1874579665774592:12345-0@lo:103:ldlm_bl.0 Request processed in 638us (2128us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739073.434939:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1409 00000100:00100000:3.0:1787739073.435983:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356af480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665774592:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739073.443798:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739073.443831:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652542848 00000100:00100000:2.0:1787739073.443841:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652542848 00000100:00100000:2.0:1787739073.443900:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23688 00000100:00100000:2.0:1787739073.443909:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305656580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:17726:x1874579652542848:12345-192.168.203.15@tcp:101:rm.0 00010000:00010000:2.0:1787739073.443932:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739073.443963:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae83f lrc: 2/0,0 mode: --/CR res: [0x200000402:0x13a5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0954cc expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739073.444131:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x13a5:0x0]: returning size 4096 00010000:00010000:2.0:1787739073.444227:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae83f lrc: 1/0,0 mode: --/CR res: [0x200000402:0x13a5:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954cc expref: 62 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.444248:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13a5:0x0].0x0 (ffffa0730608b800) 00010000:00010000:2.0:1787739073.444251:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739073.444256:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae83f lrc: 0/0,0 mode: --/CR res: [0x200000402:0x13a5:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954cc expref: 62 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.444269:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa0e00, rc 0) 00000100:00100000:2.0:1787739073.444334:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305656580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:17726:x1874579652542848:12345-192.168.203.15@tcp:101:rm.0 Request processed in 426us (545us total) trans 4294983487 rc 301/301 00000100:00100000:2.0:1787739073.444341:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23688 00000100:00100000:2.0:1787739073.453279:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739073.453377:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652543104 00000100:00100000:0.0:1787739073.453388:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652543104 00000100:00100000:0.0:1787739073.453492:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5071 00000100:00100000:0.0:1787739073.453502:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:17726:x1874579652543104:12345-192.168.203.15@tcp:35:rm.0 00000100:00100000:0.0:1787739073.453722:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+62:17726:x1874579652543104:12345-192.168.203.15@tcp:35:rm.0 Request processed in 221us (451us total) trans 4294983488 rc 0/0 00000100:00100000:0.0:1787739073.453730:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5071 00000100:00100000:1.0:1787739073.467346:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739073.467447:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652543360 00000100:00100000:2.0:1787739073.467458:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652543360 00000100:00100000:2.0:1787739073.467519:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23689 00000100:00100000:2.0:1787739073.467528:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305654000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+62:17726:x1874579652543360:12345-192.168.203.15@tcp:36:rm.0 00010000:00010000:2.0:1787739073.467582:0:7944:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1787739073.467617:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae433 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x13a5:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09545c expref: 62 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.467639:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ae058 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x13a5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952db expref: 61 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.467650:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13a5:0x0].0x0 (ffffa0730608b800) 00010000:00010000:2.0:1787739073.467658:0:7944:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739073.467681:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae84d lrc: 3/0,1 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.467692:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae84d lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.467701:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae84d lrc: 3/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.467725:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae854 lrc: 3/0,1 mode: --/PW res: [0x200000400:0x5:0x0].0x649afea4 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.467732:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae854 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.467739:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae854 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.467797:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1ae85b lrc: 3/0,1 mode: --/EX res: [0x200000402:0x13a5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.467804:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1ae85b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x13a5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.467810:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1ae85b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x13a5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.467824:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1ae862 lrc: 3/0,1 mode: --/EX res: [0x200000400:0x11:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.467833:0:7944:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ae066 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0x11:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952e2 expref: 60 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.467840:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1ae862 lrc: 4/0,1 mode: --/EX res: [0x200000400:0x11:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.467874:0:7944:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ae066 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x11:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952e2 expref: 60 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.467886:0:7944:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ae066 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x11:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952e2 expref: 60 pid: 7944 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739073.467900:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665774976:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739073.467992:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4dcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739073.471471:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7944:1874579665774976:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739073.471500:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1ae862 lrc: 3/0,1 mode: --/EX res: [0x200000400:0x11:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739073.474052:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739073.474146:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652543488 00000100:00100000:0.0:1787739073.474155:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652543488 00010000:00010000:0.0:1787739073.474193:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ae066 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x11:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952e2 expref: 61 pid: 7944 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739073.474234:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1596 00000100:00100000:0.0:1787739073.474244:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305654f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:2408:x1874579652543488:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739073.474259:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739073.474266:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ae066 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0x11:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952e2 expref: 61 pid: 7944 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.474279:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ae066 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0x11:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952e2 expref: 61 pid: 7944 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.474301:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa0730633c600/0x7ee4453e4a1ae066 lrc: 0/0,0 mode: --/PR res: [0x200000400:0x11:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952e2 expref: 61 pid: 7944 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.474315:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x11:0x0].0x0 (ffffa07403eeb400) 00010000:00010000:0.0:1787739073.474320:0:9525:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1ae862 lrc: 3/0,1 mode: --/EX res: [0x200000400:0x11:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.474329:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1ae862 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x11:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.474337:0:9525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1ae862 lrc: 4/0,1 mode: EX/EX res: [0x200000400:0x11:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.474371:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739073.474449:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305654f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:2408:x1874579652543488:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 207us (406us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739073.474456:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1596 00010000:00010000:2.0:1787739073.474698:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1ae862 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.474707:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1ae862 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.474750:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae869 lrc: 3/0,1 mode: --/EX res: [0x240000401:0x11:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.474758:0:7944:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae869 lrc: 3/0,1 mode: --/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.474768:0:7944:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae869 lrc: 3/0,1 mode: --/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739073.474787:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665775104:0@lo:101:mdt00_003.0 00000100:00100000:2.0:1787739073.474828:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739073.474856:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4dcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739073.474880:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665775104 00000100:00100000:2.0:1787739073.474884:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665775104 00000100:00100000:2.0:1787739073.474917:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3603 00000100:00100000:2.0:1787739073.474924:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730252bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+25:7944:x1874579665775104:12345-0@lo:101:mdt00_003.0 00010000:00010000:2.0:1787739073.474940:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739073.474963:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1ae870 lrc: 2/0,0 mode: --/EX res: [0x240000401:0x11:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae869 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.474983:0:12410:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ae074 lrc: 2/0,0 mode: PR/PR res: [0x240000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952e9 expref: 31 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.474992:0:12410:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1ae870 lrc: 4/0,0 mode: --/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae869 expref: 26 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.475014:0:12410:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ae074 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952e9 expref: 31 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.475033:0:12410:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ae074 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952e9 expref: 31 pid: 6524 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739073.475043:0:12410:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304262940 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001_UUID:12410:1874579665775232:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739073.475083:0:12410:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d180 going to sleep for 11 seconds 00000100:00100000:2.0:1787739073.483023:0:12410:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304262940 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001_UUID:12410:1874579665775232:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739073.483072:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1ae870 lrc: 3/0,0 mode: --/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae869 expref: 26 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.483087:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x11:0x0].0x0 (ffffa07305ca2300) 00010000:00010000:2.0:1787739073.483089:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739073.483094:0:12410:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1ae870 lrc: 3/0,0 mode: --/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae869 expref: 26 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.483114:0:12410:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1ae870 lrc: 3/0,0 mode: --/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae869 expref: 26 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.483122:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa1200, rc 0) 00000100:00100000:2.0:1787739073.483186:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730252bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+26:7944:x1874579665775104:12345-0@lo:101:mdt00_003.0 Request processed in 8262us (8356us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739073.483195:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3603 00000100:00100000:0.0:1787739073.483380:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665775104:0@lo:101:mdt00_003.0 00010000:00010000:0.0:1787739073.483397:0:7944:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae869 lrc: 4/0,1 mode: --/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae870 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.483407:0:7944:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae869 lrc: 4/0,1 mode: --/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae870 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739073.485317:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739073.485712:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652543616 00000100:00100000:2.0:1787739073.485723:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652543616 00010000:00010000:2.0:1787739073.485766:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ae074 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952e9 expref: 32 pid: 6524 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739073.485811:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1597 00000100:00100000:2.0:1787739073.485820:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305656d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+32:2409:x1874579652543616:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739073.485835:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739073.485841:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ae074 lrc: 4/0,0 mode: PR/PR res: [0x240000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952e9 expref: 32 pid: 6524 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.485853:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ae074 lrc: 3/0,0 mode: PR/PR res: [0x240000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952e9 expref: 32 pid: 6524 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.485875:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa0730609aa00/0x7ee4453e4a1ae074 lrc: 0/0,0 mode: --/PR res: [0x240000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0952e9 expref: 32 pid: 6524 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.485891:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x11:0x0].0x0 (ffffa07305ca2300) 00010000:00010000:2.0:1787739073.485897:0:10340:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1ae870 lrc: 2/0,0 mode: --/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae869 expref: 25 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.485908:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1ae870 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae869 expref: 25 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.485916:0:10340:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1ae870 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1ae869 expref: 25 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.485962:0:10340:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1ae870 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae869 expref: 25 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739073.485975:0:10340:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304262940 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665775360:0@lo:105: 00000100:00100000:2.0:1787739073.486018:0:10340:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739073.486293:0:10340:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4d180 going to sleep for 11 seconds 00000100:00100000:3.0:1787739073.490629:0:6506:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665775360 00000100:00100000:3.0:1787739073.490645:0:6506:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665775360 00000100:00100000:3.0:1787739073.490737:0:6506:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1214 00000100:00100000:3.0:1787739073.490748:0:6506:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304260f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665775360:12345-0@lo:105: 00010000:00010000:3.0:1787739073.490766:0:6506:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae869 lrc: 5/0,1 mode: --/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae870 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.490778:0:6506:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae869 lrc: 5/0,1 mode: --/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae870 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.490787:0:6506:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae869 lrc: 5/0,1 mode: --/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae870 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.490797:0:6506:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae869 lrc: 5/0,1 mode: EX/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae870 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.490806:0:6506:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae869 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1ae870 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.490815:0:6506:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae869 lrc: 6/0,1 mode: EX/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1ae870 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.490838:0:6506:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07306aa1000) 00000100:00100000:3.0:1787739073.490926:0:6506:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304260f00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:lustre-MDT0000-mdtlov_UUID+4:10340:x1874579665775360:12345-0@lo:105: Request processed in 180us (4906us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739073.490933:0:6506:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1214 00010000:00010000:0.0:1787739073.491024:0:7944:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae869 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae870 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739073.491037:0:7944:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae869 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae870 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739073.492904:0:10340:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304262940 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001_UUID:10340:1874579665775360:0@lo:105: 00010000:00010000:2.0:1787739073.492928:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739073.493000:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305656d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+31:2409:x1874579652543616:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 7179us (7689us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739073.493011:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1597 00000100:00100000:0.0:1787739073.493142:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d84b40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665775488:0@lo:403:mdt00_003.0 00000100:00100000:0.0:1787739073.493202:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739073.493241:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062489c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739073.493518:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665775488 00000100:00100000:2.0:1787739073.493525:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665775488 00000100:00100000:2.0:1787739073.493562:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3604 00000100:00100000:2.0:1787739073.493570:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d87480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+26:7944:x1874579665775488:12345-0@lo:403:mdt00_003.0 00000100:00100000:2.0:1787739073.496552:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d87480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+26:7944:x1874579665775488:12345-0@lo:403:mdt00_003.0 Request processed in 2974us (3342us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739073.496567:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3604 00000100:00100000:0.0:1787739073.496956:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d84b40 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665775488:0@lo:403:mdt00_003.0 00000040:00080000:0.0:1787739073.501130:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].401, 952 off602744 00000040:00080000:0.0:1787739073.501317:0:7944:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].400, 1032 off652808 00000004:00080000:0.0:1787739073.501323:0:7944:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa073062489c0 with version 754 00000004:00080000:1.0:1787739073.503296:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 754 rpc_version 754 00000020:00100000:1.0:1787739073.503341:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739073.503353:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07305654780 x1874579665775744/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1992/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739073.503377:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305654780 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665775744:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739073.503523:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739073.503602:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288480 going to sleep for 11 seconds 00000100:00100000:2.0:1787739073.503785:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665775744 00000100:00100000:2.0:1787739073.503793:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665775744 00000100:00100000:2.0:1787739073.503952:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3605 00000100:00100000:2.0:1787739073.503961:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305656940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+26:7740:x1874579665775744:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739073.504221:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305656940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+27:7740:x1874579665775744:12345-0@lo:1000:osp_up1-0.0 Request processed in 260us (696us total) trans 4294968468 rc 0/0 00000100:00100000:2.0:1787739073.504229:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3605 00000100:00100000:1.0:1787739073.504321:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305654780 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665775744:0@lo:1000:osp_up1-0.0 00000004:00200000:3.0:1787739073.504463:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x13a5:0x0]: returning size 4096 00010000:00010000:3.0:1787739073.504486:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1ae862 lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504496:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1ae862 lrc: 2/0,0 mode: EX/EX res: [0x200000400:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504504:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1ae862 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504510:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0c00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739073.504515:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1ae862 lrc: 3/0,0 mode: EX/EX res: [0x200000400:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504522:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1ae862 lrc: 4/0,0 mode: EX/EX res: [0x200000400:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504537:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x11:0x0].0x0 (ffffa07403eeb400) 00010000:00010000:3.0:1787739073.504539:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739073.504544:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1ae862 lrc: 2/0,0 mode: --/EX res: [0x200000400:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504550:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1ae862 lrc: 0/0,0 mode: --/EX res: [0x200000400:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504570:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae869 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae870 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504578:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae869 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae870 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504586:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae869 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae870 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504606:0:7944:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739073.504614:0:7944:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x11:0x0] 00010000:00010000:3.0:1787739073.504643:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1ae85b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x13a5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504650:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1ae85b lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x13a5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504655:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1ae85b lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x13a5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504667:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1e00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739073.504670:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1ae85b lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x13a5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504675:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1ae85b lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x13a5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504683:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13a5:0x0].0x0 (ffffa074419d5a00) 00010000:00010000:3.0:1787739073.504685:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1787739073.504689:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1ae85b lrc: 2/0,0 mode: --/EX res: [0x200000402:0x13a5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504694:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1ae85b lrc: 0/0,0 mode: --/EX res: [0x200000402:0x13a5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504711:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae84d lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504717:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae84d lrc: 4/0,1 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504722:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae84d lrc: 3/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504728:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae84d lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504732:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0e00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739073.504735:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae84d lrc: 4/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504741:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae84d lrc: 5/0,0 mode: CW/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504749:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa074419d5400) 00010000:00010000:3.0:1787739073.504751:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739073.504754:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae84d lrc: 3/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504759:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1ae84d lrc: 0/0,0 mode: --/CW res: [0x200000400:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739073.504771:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000755f4155 reply state 0000000021e2ed8e transno 4294983489 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae854 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504782:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae854 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.504789:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x649afea4 (ffffa074419d5900) 00010000:00010000:3.0:1787739073.504791:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739073.504794:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1787739073.504865:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305654000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:17726:x1874579652543360:12345-192.168.203.15@tcp:36:rm.0 Request processed in 37337us (37527us total) trans 4294983489 rc 0/0 00000100:00100000:3.0:1787739073.504874:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23689 00010000:00010000:1.0:1787739073.505253:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739073.505260:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae869 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae870 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.505273:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739073.505278:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae869 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae870 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739073.505286:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae869 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae870 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.505315:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae869 lrc: 2/0,0 mode: --/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae870 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.505322:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739073.505342:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae869 lrc: 1/0,0 mode: --/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae870 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.505349:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1000/0x7ee4453e4a1ae869 lrc: 0/0,0 mode: --/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae870 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.505364:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739073.505365:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739073.505367:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739073.505375:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787739073.505387:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073056552c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665775872:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739073.505450:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739073.505486:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665775872 00000100:00100000:1.0:1787739073.505491:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665775872 00000100:00100000:1.0:1787739073.505535:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1411 00000100:00100000:1.0:1787739073.505543:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba3c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+27:3652:x1874579665775872:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739073.505557:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739073.505574:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1ae870 lrc: 0/0,0 mode: --/EX res: [0x240000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae869 expref: 27 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.505586:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x11:0x0].0x0 (ffffa07305ca2300) 00010000:00010000:1.0:1787739073.505594:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739073.505623:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba3c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+26:3652:x1874579665775872:12345-0@lo:103:ldlm_bl.0 Request processed in 81us (172us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739073.505642:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1411 00000100:00100000:1.0:1787739073.505707:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073056552c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665775872:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739073.516342:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739073.516902:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652543872 00000100:00100000:2.0:1787739073.516911:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652543872 00000100:00100000:2.0:1787739073.516985:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5072 00000100:00100000:2.0:1787739073.516995:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074332ab850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+61:17726:x1874579652543872:12345-192.168.203.15@tcp:35:rm.0 00000040:00080000:2.0:1787739073.517258:0:6526:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].402, 344 off603088 00000040:00080000:2.0:1787739073.517302:0:6526:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].401, 424 off653232 00000004:00080000:2.0:1787739073.517307:0:6526:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07410b4d180 with version 755 00000004:00080000:1.0:1787739073.517386:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 755 rpc_version 755 00000020:00100000:1.0:1787739073.517448:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739073.517457:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073056552c0 x1874579665776128/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1304/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739073.517482:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073056552c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665776128:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739073.517538:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739073.517563:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306288180 going to sleep for 11 seconds 00000100:00100000:2.0:1787739073.517835:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665776128 00000100:00100000:2.0:1787739073.517841:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665776128 00000100:00100000:2.0:1787739073.517882:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3606 00000100:00100000:2.0:1787739073.517890:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305654b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+26:7740:x1874579665776128:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739073.518044:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305654b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+27:7740:x1874579665776128:12345-0@lo:1000:osp_up1-0.0 Request processed in 154us (504us total) trans 4294968469 rc 0/0 00000100:00100000:2.0:1787739073.518052:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3606 00000100:00100000:3.0:1787739073.518157:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073056552c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665776128:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739073.518325:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074332ab850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+62:17726:x1874579652543872:12345-192.168.203.15@tcp:35:rm.0 Request processed in 1330us (1990us total) trans 4294983490 rc 0/0 00000100:00100000:2.0:1787739073.518333:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5072 00000100:00100000:1.0:1787739073.536065:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739073.536174:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652544128 00000100:00100000:3.0:1787739073.536185:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652544128 00000100:00100000:3.0:1787739073.536273:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23690 00000100:00100000:3.0:1787739073.536284:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305654f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+62:17726:x1874579652544128:12345-192.168.203.15@tcp:36:rm.0 00010000:00010000:3.0:1787739073.536403:0:6524:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 4 locks, starting at 0 00010000:00010000:3.0:1787739073.536481:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd200/0x7ee4453e4a199c75 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x138f:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09139c expref: 62 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.536510:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc400/0x7ee4453e4a1998fc lrc: 0/0,0 mode: --/PR res: [0x200000402:0x138f:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0912d1 expref: 61 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.536532:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa074355b1000/0x7ee4453e4a1998bd lrc: 0/0,0 mode: --/PR res: [0x200000402:0x138f:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0912b5 expref: 60 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.536544:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x138f:0x0].0x0 (ffffa0730608be00) 00010000:00010000:3.0:1787739073.536567:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc200/0x7ee4453e4a1998a1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0912ae expref: 59 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.536576:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa07401795300) 00010000:00010000:3.0:1787739073.536579:0:6524:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739073.536796:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae877 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.536817:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae877 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.536827:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae877 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.536841:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ae87e lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xca5a1f93 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.536847:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ae87e lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xca5a1f93 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.536852:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ae87e lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xca5a1f93 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.536920:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae885 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x138f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.536926:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae885 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x138f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.536932:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae885 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x138f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.536963:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ae88c lrc: 3/0,1 mode: --/EX res: [0x200000400:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.536973:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ae88c lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.536980:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ae88c lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.537011:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ae893 lrc: 3/0,1 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.537017:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ae893 lrc: 3/0,1 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.537027:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ae893 lrc: 3/0,1 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739073.537047:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665776256:0@lo:101:mdt00_001.0 00000100:00100000:3.0:1787739073.537099:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739073.537183:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739073.544839:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665776256 00000100:00100000:2.0:1787739073.544854:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665776256 00000100:00100000:2.0:1787739073.544928:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3607 00000100:00100000:2.0:1787739073.544937:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+27:6524:x1874579665776256:12345-0@lo:101:mdt00_001.0 00010000:00010000:2.0:1787739073.544973:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739073.544997:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1ae89a lrc: 2/0,0 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae893 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.545026:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1ae89a lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae893 expref: 28 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.545041:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1ae89a lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae893 expref: 28 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.545048:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074419d5700) 00010000:00010000:2.0:1787739073.545052:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739073.545055:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa0400, rc 0) 00000100:00100000:2.0:1787739073.545182:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+28:6524:x1874579665776256:12345-0@lo:101:mdt00_001.0 Request processed in 247us (8082us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739073.545190:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3607 00000100:00100000:3.0:1787739073.547586:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665776256:0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787739073.548732:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ae893 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae89a expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.548747:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ae893 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae89a expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739073.549819:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304262d00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665776384:0@lo:403:mdt00_001.0 00000100:00100000:2.0:1787739073.549891:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739073.550056:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b4dcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739073.551301:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665776384 00000100:00100000:1.0:1787739073.551312:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665776384 00000100:00100000:1.0:1787739073.551575:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3608 00000100:00100000:1.0:1787739073.551594:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+28:6524:x1874579665776384:12345-0@lo:403:mdt00_001.0 00000100:00100000:1.0:1787739073.557128:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+28:6524:x1874579665776384:12345-0@lo:403:mdt00_001.0 Request processed in 5530us (7222us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739073.557148:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3608 00000100:00100000:3.0:1787739073.560021:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304262d00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665776384:0@lo:403:mdt00_001.0 00000040:00080000:2.0:1787739073.561881:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].403, 936 off604024 00000040:00080000:2.0:1787739073.562055:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].402, 1016 off654248 00000004:00080000:2.0:1787739073.562061:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07410b4dcc0 with version 756 00000004:00080000:3.0:1787739073.562140:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 756 rpc_version 756 00000020:00100000:3.0:1787739073.562174:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739073.562183:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07306be4f00 x1874579665776640/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 1976/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739073.562206:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be4f00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665776640:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739073.562270:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739073.562327:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0730610e000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739073.562600:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665776640 00000100:00100000:2.0:1787739073.562606:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665776640 00000100:00100000:2.0:1787739073.562661:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3609 00000100:00100000:2.0:1787739073.562670:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f770c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+28:7740:x1874579665776640:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739073.566054:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f770c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+29:7740:x1874579665776640:12345-0@lo:1000:osp_up1-0.0 Request processed in 3378us (3776us total) trans 4294968470 rc 0/0 00000100:00100000:0.0:1787739073.566071:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3609 00000100:00100000:3.0:1787739073.566578:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be4f00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665776640:0@lo:1000:osp_up1-0.0 00000004:00200000:2.0:1787739073.566610:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x138f:0x0]: returning size 4096 00010000:00010000:2.0:1787739073.566640:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ae88c lrc: 3/0,1 mode: EX/EX res: [0x200000400:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566652:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ae88c lrc: 2/0,0 mode: EX/EX res: [0x200000400:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566660:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ae88c lrc: 3/0,0 mode: EX/EX res: [0x200000400:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566666:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aac00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739073.566670:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ae88c lrc: 3/0,0 mode: EX/EX res: [0x200000400:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566677:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ae88c lrc: 4/0,0 mode: EX/EX res: [0x200000400:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566693:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x0 (ffffa0730608b800) 00010000:00010000:2.0:1787739073.566696:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739073.566719:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ae88c lrc: 2/0,0 mode: --/EX res: [0x200000400:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566741:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1ae88c lrc: 0/0,0 mode: --/EX res: [0x200000400:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566762:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ae893 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae89a expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566770:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ae893 lrc: 4/0,1 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae89a expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566782:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ae893 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1ae89a expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566807:0:6524:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739073.566816:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0x5:0x0] 00010000:00010000:2.0:1787739073.566829:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae885 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x138f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566835:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae885 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x138f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566840:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae885 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x138f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566844:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739073.566847:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae885 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x138f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566852:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae885 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x138f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566860:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x138f:0x0].0x0 (ffffa0730608bf00) 00010000:00010000:2.0:1787739073.566862:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1787739073.566865:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae885 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x138f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566871:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1ae885 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x138f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566889:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae877 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566895:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae877 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566900:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae877 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566905:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae877 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566910:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739073.566912:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae877 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566918:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae877 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566924:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa07401795300) 00010000:00010000:2.0:1787739073.566927:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739073.566930:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae877 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566936:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1ae877 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739073.566958:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003fcce350 reply state 000000001b67490b transno 4294983491 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ae87e lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xca5a1f93 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566969:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ae87e lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xca5a1f93 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739073.566976:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xca5a1f93 (ffffa0730608bb00) 00010000:00010000:2.0:1787739073.566979:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739073.566982:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1787739073.567060:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305654f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+60:17726:x1874579652544128:12345-192.168.203.15@tcp:36:rm.0 Request processed in 30776us (31012us total) trans 4294983491 rc 0/0 00000100:00100000:2.0:1787739073.567068:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23690 00010000:00010000:1.0:1787739073.567249:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739073.567256:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ae893 lrc: 2/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1ae89a expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.567263:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab800 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739073.567267:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ae893 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae89a expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739073.567275:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ae893 lrc: 3/0,0 mode: EX/EX res: [0x240000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae89a expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.567306:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ae893 lrc: 2/0,0 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae89a expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.567312:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739073.567342:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ae893 lrc: 1/0,0 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae89a expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.567349:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069ab800/0x7ee4453e4a1ae893 lrc: 0/0,0 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae89a expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739073.567370:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739073.567372:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739073.567374:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739073.567475:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787739073.567491:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305656d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665776768:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739073.567535:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739073.567624:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665776768 00000100:00100000:3.0:1787739073.567631:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665776768 00000100:00100000:3.0:1787739073.567687:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1412 00000100:00100000:3.0:1787739073.567753:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+29:3651:x1874579665776768:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739073.567768:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739073.567785:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1ae89a lrc: 0/0,0 mode: --/EX res: [0x240000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae893 expref: 29 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739073.567797:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x0 (ffffa074419d5700) 00010000:00010000:3.0:1787739073.567804:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739073.567845:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84b40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+28:3651:x1874579665776768:12345-0@lo:103:ldlm_bl.0 Request processed in 150us (309us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739073.567851:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1412 00000100:00100000:0.0:1787739073.567919:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305656d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665776768:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739073.859032:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739073.986752:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665776896:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1787739073.986827:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739073.986970:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665776896 00000100:00100000:0.0:1787739073.986978:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665776896 00000100:00100000:0.0:1787739073.987039:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 366 00000100:00100000:0.0:1787739073.987049:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665776896:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1787739073.987076:0:8441:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8646080 pending 0 free 3923529728 avail 3692007424 00000100:00100000:0.0:1787739073.987131:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665776896:12345-0@lo:13:osp-pre-0-1.0 Request processed in 84us (304us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739073.987139:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 366 00000100:00100000:1.0:1787739073.987316:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305657c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665777024:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1787739073.987356:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1787739073.987535:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787739073.987548:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665777024 00000100:00100000:2.0:1787739073.987552:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665777024 00000100:00100000:2.0:1787739073.987599:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 367 00000100:00100000:2.0:1787739073.987606:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305655e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874579665777024:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787739073.987616:0:8440:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8646080 pending 0 free 3923529728 avail 3692007424 00000100:00100000:2.0:1787739073.987649:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305655e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3652:x1874579665777024:12345-0@lo:13:osp-pre-0-0.0 Request processed in 44us (292us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739073.987656:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 367 00000100:00100000:2.0:1787739073.987694:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305657c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665777024:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787739073.987723:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665776896:0@lo:13:osp-pre-0-1.0 00000100:00080000:0.0:1787739073.987780:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1787739074.002215:0:6502:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1ae807 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae80e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739074.002238:0:6502:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1ae807 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae80e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739074.002285:0:6502:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1ae807 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae80e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739074.002292:0:6502:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739074.002328:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1ae807 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae80e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739074.002356:0:6502:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1ae75f lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae766 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739074.002363:0:6502:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1ae75f lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae766 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739074.002379:0:6502:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1ae75f lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae766 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739074.002385:0:6502:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739074.002396:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1ae75f lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae766 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739074.002407:0:6502:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1c00/0x7ee4453e4a1ae67f lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae686 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739074.002449:0:6502:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1c00/0x7ee4453e4a1ae67f lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae686 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739074.002465:0:6502:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1c00/0x7ee4453e4a1ae67f lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae686 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739074.002470:0:6502:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739074.002475:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1c00/0x7ee4453e4a1ae67f lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae686 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739074.002487:0:6502:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0600/0x7ee4453e4a1ae5f3 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x44749f58 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae5fa expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739074.002497:0:6502:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0600/0x7ee4453e4a1ae5f3 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x44749f58 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae5fa expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739074.002509:0:6502:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0600/0x7ee4453e4a1ae5f3 lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x44749f58 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae5fa expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739074.002514:0:6502:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739074.002518:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa0600/0x7ee4453e4a1ae5f3 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x44749f58 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae5fa expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739074.002527:0:6502:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841a00/0x7ee4453e4a1ae59f lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae5a6 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739074.002533:0:6502:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841a00/0x7ee4453e4a1ae59f lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae5a6 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739074.002544:0:6502:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841a00/0x7ee4453e4a1ae59f lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae5a6 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739074.002549:0:6502:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739074.002553:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841a00/0x7ee4453e4a1ae59f lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae5a6 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739074.002563:0:6502:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089200/0x7ee4453e4a1ae54b lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae552 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739074.002569:0:6502:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089200/0x7ee4453e4a1ae54b lrc: 3/0,0 mode: PW/PW res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae552 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739074.002598:0:6502:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089200/0x7ee4453e4a1ae54b lrc: 2/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae552 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739074.002605:0:6502:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739074.002609:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089200/0x7ee4453e4a1ae54b lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae552 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1787739074.002620:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983491 is committed 00000001:00080000:0.0:1787739074.002652:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983483 is committed 00000100:00100000:2.0:1787739074.002685:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665777536:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787739074.002749:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739074.002804:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1787739074.002811:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665777152:0@lo:103:jbd2/dm-0-8.0 00010000:00010000:1.0:1787739074.002830:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1ae3f4 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739074.002832:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787739074.002842:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1ae3f4 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739074.002848:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665777664:0@lo:103:jbd2/dm-0-8.0 00010000:00010000:1.0:1787739074.002849:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1ae3f4 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739074.002855:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0a00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739074.002859:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1ae3f4 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739074.002864:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1787739074.002867:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1ae3f4 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739074.002880:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xac50785c (ffffa074419d5000) 00010000:00010000:1.0:1787739074.002883:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739074.002887:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1ae3f4 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739074.002892:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665777536 00010000:00010000:1.0:1787739074.002893:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1ae3f4 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0xac50785c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739074.002897:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665777536 00010000:00010000:1.0:1787739074.002910:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1ae640 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739074.002916:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1ae640 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739074.002921:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1ae640 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739074.002925:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739074.002929:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1ae640 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739074.002934:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1ae640 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739074.002948:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665777152 00000100:00100000:2.0:1787739074.002951:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665777152 00010000:00010000:1.0:1787739074.002955:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x3fb0b00e (ffffa074419d5300) 00010000:00010000:1.0:1787739074.002958:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739074.002961:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1ae640 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739074.002967:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1ae640 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x3fb0b00e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739074.002968:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665777664 00000100:00100000:2.0:1787739074.002971:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665777664 00010000:00010000:1.0:1787739074.002980:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae854 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739074.002985:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1413 00010000:00010000:1.0:1787739074.002987:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae854 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739074.002992:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042470c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+30:3652:x1874579665777536:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:1.0:1787739074.002993:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae854 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739074.002998:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739074.003000:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae854 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739074.003004:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739074.003006:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae854 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739074.003012:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x649afea4 (ffffa074419d5900) 00010000:00010000:1.0:1787739074.003014:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739074.003017:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae854 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739074.003020:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841400/0x7ee4453e4a1ae5fa lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x44749f58 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae5f3 expref: 30 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739074.003023:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae854 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0x5:0x0].0x649afea4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739074.003031:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x44749f58 (ffffa0730713a600) 00010000:00010000:1.0:1787739074.003035:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ae87e lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xca5a1f93 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739074.003037:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739074.003040:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ae87e lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xca5a1f93 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739074.003045:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ae87e lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xca5a1f93 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739074.003050:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739074.003052:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ae87e lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xca5a1f93 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739074.003058:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ae87e lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xca5a1f93 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739074.003066:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042470c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+29:3652:x1874579665777536:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 74us (317us total) trans 0 rc 0/0 00010000:00010000:1.0:1787739074.003066:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xca5a1f93 (ffffa0730608bb00) 00010000:00010000:1.0:1787739074.003068:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739074.003071:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ae87e lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0xca5a1f93 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739074.003072:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1413 00010000:00010000:1.0:1787739074.003076:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1ae87e lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0xca5a1f93 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739074.003086:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1414 00000100:00100000:2.0:1787739074.003093:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304245e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+28:3652:x1874579665777152:12345-0@lo:103:jbd2/dm-0-8.0 00000100:00100000:1.0:1787739074.003097:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1415 00010000:00010000:2.0:1787739074.003099:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787739074.003103:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304247c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+28:3652:x1874579665777664:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:2.0:1787739074.003109:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae80e lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x4ac2f730 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae807 expref: 28 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739074.003112:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739074.003121:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1ae5a6 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x32e1d295 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae59f expref: 28 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739074.003130:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x32e1d295 (ffffa0730608b300) 00010000:00010000:2.0:1787739074.003131:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x4ac2f730 (ffffa07305ca2b00) 00010000:00010000:1.0:1787739074.003135:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739074.003136:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739074.003156:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304245e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+26:3652:x1874579665777152:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 64us (324us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739074.003157:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304247c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+26:3652:x1874579665777664:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 55us (293us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739074.003162:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1414 00000020:00080000:0.0:1787739074.003182:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 394 batchid updated 00000020:00080000:0.0:1787739074.003186:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 393 batchid updated 00000020:00080000:0.0:1787739074.003189:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 392 batchid updated 00000100:00100000:2.0:1787739074.003197:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665777152:0@lo:103:jbd2/dm-0-8.0 00000020:00080000:0.0:1787739074.003203:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 391 batchid updated 00000100:00100000:2.0:1787739074.003205:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665777664:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787739074.003212:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665777536:0@lo:103:jbd2/dm-0-8.0 00000020:00080000:0.0:1787739074.003220:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 394 00000100:00100000:2.0:1787739074.003234:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d870c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665777408:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787739074.003255:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739074.003271:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000004:00080000:0.0:1787739074.003273:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07410b95240 with version 757 00000100:00100000:2.0:1787739074.003276:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d86940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665777280:0@lo:103:jbd2/dm-0-8.0 00000020:00080000:0.0:1787739074.003281:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 391 cancel update log [0x240000401:0x3:0x0].388: rc = 0 00000100:00100000:2.0:1787739074.003292:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739074.003305:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665777792:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787739074.003322:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739074.003336:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665777408 00000100:00100000:2.0:1787739074.003339:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665777408 00000020:00080000:0.0:1787739074.003351:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 391 cancel update log [0x200000400:0x3:0x0].389: rc = 0 00000100:00100000:3.0:1787739074.003590:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665777280 00000100:00100000:3.0:1787739074.003594:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665777280 00000004:00080000:0.0:1787739074.003680:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07410b95840 with version 758 00000020:00080000:0.0:1787739074.003684:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 393 cancel update log [0x240000401:0x3:0x0].389: rc = 0 00000020:00080000:0.0:1787739074.004673:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 393 cancel update log [0x200000400:0x3:0x0].390: rc = 0 00000020:00080000:0.0:1787739074.004693:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 394 00000100:00100000:0.0:1787739074.004770:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665777792 00000100:00100000:0.0:1787739074.004776:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665777792 00000100:00100000:1.0:1787739074.005070:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1415 00000100:00100000:2.0:1787739074.007438:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1416 00000100:00100000:2.0:1787739074.007461:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302529680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+27:3651:x1874579665777408:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:2.0:1787739074.007479:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739074.007508:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088200/0x7ee4453e4a1ae686 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0xda636067 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae67f expref: 27 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739074.007528:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0xda636067 (ffffa07305ca2f00) 00010000:00010000:2.0:1787739074.007537:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739074.007605:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302529680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+26:3651:x1874579665777408:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 154us (4348us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739074.007612:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1416 00000100:00100000:1.0:1787739074.016100:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1417 00000100:00100000:1.0:1787739074.016125:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07302528000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+25:3651:x1874579665777280:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:1.0:1787739074.016147:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739074.016175:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae766 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x23e4280e bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae75f expref: 25 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739074.016197:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x23e4280e (ffffa074419d5d00) 00010000:00010000:1.0:1787739074.016209:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739074.016278:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07302528000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+24:3651:x1874579665777280:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 164us (12983us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739074.016286:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1417 00000100:00100000:2.0:1787739074.019134:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d86940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665777280:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787739074.019152:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d870c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665777408:0@lo:103:jbd2/dm-0-8.0 00000004:00080000:2.0:1787739074.019179:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 757 rpc_version 757 00000020:00100000:2.0:1787739074.019214:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739074.019222:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304d870c0 x1874579665778048/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739074.019244:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d870c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665778048:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739074.019300:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739074.019347:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a5f6e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739074.025608:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665778048 00000100:00100000:2.0:1787739074.025623:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665778048 00000100:00100000:2.0:1787739074.025666:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3610 00000100:00100000:2.0:1787739074.025676:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d86940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+24:7740:x1874579665778048:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739074.025847:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d86940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+25:7740:x1874579665778048:12345-0@lo:1000:osp_up1-0.0 Request processed in 171us (6545us total) trans 4294968471 rc 0/0 00000100:00100000:2.0:1787739074.025855:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3610 00000100:00100000:2.0:1787739074.025903:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d870c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665778048:0@lo:1000:osp_up1-0.0 00000004:00080000:2.0:1787739074.025917:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 758 rpc_version 758 00000020:00100000:2.0:1787739074.025949:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739074.025956:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304d852c0 x1874579665778304/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739074.025976:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665778304:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739074.026013:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739074.026048:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0740a5f6e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787739074.026060:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665778304 00000100:00100000:2.0:1787739074.026064:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665778304 00000100:00100000:2.0:1787739074.026081:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3611 00000100:00100000:2.0:1787739074.026087:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306004000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+25:7740:x1874579665778304:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739074.026173:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306004000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+26:7740:x1874579665778304:12345-0@lo:1000:osp_up1-0.0 Request processed in 86us (159us total) trans 4294968472 rc 0/0 00000100:00100000:2.0:1787739074.026180:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3611 00000100:00100000:2.0:1787739074.026208:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304d852c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665778304:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739074.030923:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1418 00000100:00100000:2.0:1787739074.030957:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba3850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+25:3651:x1874579665777792:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:2.0:1787739074.030979:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739074.031010:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd000/0x7ee4453e4a1ae552 lrc: 0/0,0 mode: --/PW res: [0x240000401:0x5:0x0].0x4a1b0d34 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae54b expref: 25 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739074.031031:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0x5:0x0].0x4a1b0d34 (ffffa0730608b600) 00010000:00010000:2.0:1787739074.031043:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739074.031137:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba3850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+24:3651:x1874579665777792:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 202us (27813us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739074.031145:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1418 00000100:00100000:0.0:1787739074.031547:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074356ae580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665777792:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:2.0:1787739074.498667:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787739074.498691:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665778432:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787739074.498773:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739074.498836:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665778432 00000100:00100000:3.0:1787739074.498844:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665778432 00000100:00100000:3.0:1787739074.498900:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3612 00000100:00100000:3.0:1787739074.498910:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304d84b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+24:3650:x1874579665778432:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1787739074.498955:0:6529:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1445134336 avail 1298894848 00000100:00100000:3.0:1787739074.499015:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304d84b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+24:3650:x1874579665778432:12345-0@lo:41:osp-pre-1-0.0 Request processed in 101us (237us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739074.499023:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3612 00000100:00100000:2.0:1787739074.502170:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665778432:0@lo:41:osp-pre-1-0.0 00010000:00010000:0.0:1787739074.883750:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739075.010804:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042470c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665778560:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1787739075.010878:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739075.011065:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665778560 00000100:00100000:2.0:1787739075.011073:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665778560 00000100:00100000:2.0:1787739075.011128:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3613 00000100:00100000:2.0:1787739075.011137:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305657c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:3652:x1874579665778560:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1787739075.011175:0:12410:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1444085760 avail 1297846272 00000100:00100000:2.0:1787739075.011220:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305657c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:3652:x1874579665778560:12345-0@lo:41:osp-pre-0-1.0 Request processed in 84us (342us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739075.011227:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3613 00000100:00100000:1.0:1787739075.011361:0:3652:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 4294983483 gen 2 00000004:00080000:1.0:1787739075.011368:0:3652:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983451 committed transno 4294983483 00000004:00080000:1.0:1787739075.011389:0:3652:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983456 committed transno 4294983483 00000004:00080000:1.0:1787739075.011399:0:3652:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983457 committed transno 4294983483 00000004:00080000:1.0:1787739075.011409:0:3652:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983461 committed transno 4294983483 00000004:00080000:1.0:1787739075.011448:0:3652:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983462 committed transno 4294983483 00000004:00080000:1.0:1787739075.011458:0:3652:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983463 committed transno 4294983483 00000004:00080000:1.0:1787739075.011477:0:3652:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983479 committed transno 4294983483 00000004:00080000:1.0:1787739075.011486:0:3652:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294983483 committed transno 4294983483 00000100:00100000:1.0:1787739075.011502:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042470c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665778560:0@lo:41:osp-pre-0-1.0 00010000:00010000:2.0:1787739075.906751:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739076.930570:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739077.058839:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304247c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3650:1874579665778816:0@lo:400:kworker.0 00000100:00100000:2.0:1787739077.058909:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739077.058972:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3650:1874579665779328:0@lo:400:kworker.0 00000100:00100000:2.0:1787739077.058995:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739077.059010:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3650:1874579665779840:0@lo:400:kworker.0 00000100:00100000:2.0:1787739077.059031:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739077.059066:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:2.0:1787739077.059072:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042470c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665778944:0@lo:400:kworker.0 00000100:00100000:2.0:1787739077.059095:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739077.059111:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3650:1874579665779456:0@lo:400:kworker.0 00000100:00100000:2.0:1787739077.059130:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739077.059144:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3650:1874579665779968:0@lo:400:kworker.0 00000100:00100000:2.0:1787739077.059165:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739077.059194:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665778944 00000100:00100000:2.0:1787739077.059201:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665778944 00000100:00100000:2.0:1787739077.059242:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3614 00000100:00100000:2.0:1787739077.059249:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+13:3650:x1874579665778944:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739077.059298:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+13:3650:x1874579665778944:12345-0@lo:400:kworker.0 Request processed in 49us (201us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739077.059305:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3614 00000100:00100000:2.0:1787739077.059325:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665778816 00000100:00100000:2.0:1787739077.059329:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665778816 00000100:00100000:2.0:1787739077.059346:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665779328 00000100:00100000:2.0:1787739077.059348:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665779328 00000100:00100000:2.0:1787739077.059365:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665779840 00000100:00100000:2.0:1787739077.059368:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665779840 00000100:00100000:2.0:1787739077.059379:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665779456 00000100:00100000:2.0:1787739077.059381:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665779456 00000100:00100000:2.0:1787739077.059391:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665779968 00000100:00100000:2.0:1787739077.059393:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665779968 00000100:00100000:2.0:1787739077.059406:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 674 00000100:00100000:2.0:1787739077.059437:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1874579665778816:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739077.059469:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1874579665778816:12345-0@lo:400:kworker.0 Request processed in 57us (560us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739077.059475:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 674 00000100:00100000:2.0:1787739077.059488:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 675 00000100:00100000:2.0:1787739077.059494:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1874579665779328:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739077.059519:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1874579665779328:12345-0@lo:400:kworker.0 Request processed in 26us (524us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739077.059525:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 675 00000100:00100000:2.0:1787739077.059537:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 676 00000100:00100000:2.0:1787739077.059543:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3650:x1874579665779840:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739077.059569:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3650:x1874579665779840:12345-0@lo:400:kworker.0 Request processed in 27us (537us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739077.059574:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 676 00000100:00100000:2.0:1787739077.059584:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 677 00000100:00100000:2.0:1787739077.059590:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3650:x1874579665779456:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739077.059613:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce21c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3650:x1874579665779456:12345-0@lo:400:kworker.0 Request processed in 24us (483us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739077.059618:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 677 00000100:00100000:2.0:1787739077.059628:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 678 00000100:00100000:2.0:1787739077.059634:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1874579665779968:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1787739077.059657:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce1e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1874579665779968:12345-0@lo:400:kworker.0 Request processed in 24us (492us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739077.059662:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 678 00000100:00100000:2.0:1787739077.059704:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042470c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665778944:0@lo:400:kworker.0 00000100:00100000:2.0:1787739077.059715:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3650:1874579665779456:0@lo:400:kworker.0 00000100:00100000:2.0:1787739077.059723:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3650:1874579665779968:0@lo:400:kworker.0 00000100:00100000:2.0:1787739077.059730:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304247c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3650:1874579665778816:0@lo:400:kworker.0 00000100:00100000:2.0:1787739077.059738:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3650:1874579665779328:0@lo:400:kworker.0 00000100:00100000:2.0:1787739077.059746:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3650:1874579665779840:0@lo:400:kworker.0 00000100:00100000:0.0:1787739077.059959:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302529680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874579665779072:0@lo:400:kworker.0 00000100:00100000:0.0:1787739077.059990:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739077.060021:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665779584:0@lo:400:kworker.0 00000100:00100000:0.0:1787739077.060057:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739077.060075:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665780096:0@lo:400:kworker.0 00000100:00100000:0.0:1787739077.060096:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739077.060117:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [3->2] 00000100:00100000:0.0:1787739077.060128:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304245e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2:3652:1874579665778688:0@lo:400:kworker.0 00000100:00100000:0.0:1787739077.060164:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739077.060186:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665779200:0@lo:400:kworker.0 00000100:00100000:0.0:1787739077.060207:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739077.060223:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665779712:0@lo:400:kworker.0 00000100:00100000:0.0:1787739077.060243:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739077.060258:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665780224:0@lo:400:kworker.0 00000100:00100000:0.0:1787739077.060276:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739077.060302:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665779072 00000100:00100000:0.0:1787739077.060305:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665779072 00000100:00100000:0.0:1787739077.060334:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 679 00000100:00100000:0.0:1787739077.060342:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ae580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874579665779072:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787739077.060378:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ae580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3652:x1874579665779072:12345-0@lo:400:kworker.0 Request processed in 37us (387us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739077.060384:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 679 00000100:00080000:0.0:1787739077.060401:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1787739077.060405:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787739077.060441:0:14663:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665779584 00000100:00100000:0.0:1787739077.060445:0:14663:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665779584 00000100:00100000:0.0:1787739077.060473:0:14663:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665780096 00000100:00100000:0.0:1787739077.060476:0:14663:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665780096 00000100:00100000:0.0:1787739077.060493:0:14663:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665779712 00000100:00100000:0.0:1787739077.060496:0:14663:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665779712 00000100:00100000:0.0:1787739077.060513:0:14663:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665780224 00000100:00100000:0.0:1787739077.060516:0:14663:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665780224 00000100:00100000:0.0:1787739077.060536:0:14663:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 363 00000100:00100000:0.0:1787739077.060542:0:14663:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356afc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3652:x1874579665779584:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787739077.060572:0:14663:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356afc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3652:x1874579665779584:12345-0@lo:400:kworker.0 Request processed in 31us (515us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739077.060577:0:14663:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 363 00000100:00100000:0.0:1787739077.060588:0:14663:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 364 00000100:00100000:0.0:1787739077.060594:0:14663:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3652:x1874579665780096:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787739077.060619:0:14663:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+5:3652:x1874579665780096:12345-0@lo:400:kworker.0 Request processed in 26us (522us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739077.060624:0:14663:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 364 00000100:00100000:0.0:1787739077.060633:0:14663:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 365 00000100:00100000:0.0:1787739077.060639:0:14663:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356aed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3652:x1874579665779712:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787739077.060663:0:14663:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356aed00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3652:x1874579665779712:12345-0@lo:400:kworker.0 Request processed in 25us (419us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739077.060669:0:14663:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 365 00000100:00100000:0.0:1787739077.060678:0:14663:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 366 00000100:00100000:0.0:1787739077.060685:0:14663:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a31a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3652:x1874579665780224:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787739077.060709:0:14663:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a31a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3652:x1874579665780224:12345-0@lo:400:kworker.0 Request processed in 25us (432us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739077.060715:0:14663:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 366 00000100:00100000:0.0:1787739077.060730:0:6517:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665778688 00000100:00100000:0.0:1787739077.060734:0:6517:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665778688 00000100:00100000:0.0:1787739077.060756:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 454 00000100:00100000:0.0:1787739077.060762:0:6517:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0740ea12850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2+24:3652:x1874579665778688:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787739077.060789:0:6517:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0740ea12850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2+24:3652:x1874579665778688:12345-0@lo:400:kworker.0 Request processed in 28us (625us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739077.060795:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 454 00000100:00100000:0.0:1787739077.060818:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665779200 00000100:00100000:0.0:1787739077.060822:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665779200 00000100:00100000:0.0:1787739077.060845:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3615 00000100:00100000:0.0:1787739077.060851:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ae940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+24:3652:x1874579665779200:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787739077.060880:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ae940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+24:3652:x1874579665779200:12345-0@lo:400:kworker.0 Request processed in 29us (672us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739077.060886:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3615 00000100:00080000:0.0:1787739077.060933:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1787739077.060936:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787739077.060957:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304245e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2:3652:1874579665778688:0@lo:400:kworker.0 00000100:00100000:0.0:1787739077.060969:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302528000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665779200:0@lo:400:kworker.0 00000100:00100000:0.0:1787739077.060978:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665779712:0@lo:400:kworker.0 00000100:00100000:0.0:1787739077.060986:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce12c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665780224:0@lo:400:kworker.0 00000100:00100000:0.0:1787739077.060995:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07302529680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3652:1874579665779072:0@lo:400:kworker.0 00000100:00100000:0.0:1787739077.061003:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665779584:0@lo:400:kworker.0 00000100:00100000:0.0:1787739077.061012:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665780096:0@lo:400:kworker.0 00000100:00100000:3.0:1787739077.061947:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787739077.061960:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303f743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665780352:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787739077.062004:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739077.062061:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665780352 00000100:00100000:3.0:1787739077.062067:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665780352 00000100:00100000:3.0:1787739077.062108:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 368 00000100:00100000:3.0:1787739077.062115:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665780352:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1787739077.062138:0:8441:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 25431488 pending 0 free 3923496960 avail 3691974656 00000100:00100000:3.0:1787739077.062175:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665780352:12345-0@lo:13:osp-pre-1-0.0 Request processed in 62us (171us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739077.062182:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 368 00000100:00100000:3.0:1787739077.062224:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303f743c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665780352:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1787739077.065377:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a30f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665780480:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1787739077.065461:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739077.065796:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665780480 00000100:00100000:0.0:1787739077.065804:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665780480 00000100:00100000:0.0:1787739077.065852:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 369 00000100:00100000:0.0:1787739077.065861:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f43c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665780480:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1787739077.065884:0:8440:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 25431488 pending 0 free 3923496960 avail 3691974656 00000100:00100000:3.0:1787739077.066598:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a30f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665780480:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1787739077.066897:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f43c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665780480:12345-0@lo:13:osp-pre-1-1.0 Request processed in 1035us (1435us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739077.066907:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 369 00010000:00010000:2.0:1787739077.067212:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ae7b3 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x98a090de bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae7ba expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739077.067228:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ae7b3 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x98a090de bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae7ba expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739077.067264:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ae7b3 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x98a090de bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae7ba expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739077.067272:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739077.067300:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab600/0x7ee4453e4a1ae7b3 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x98a090de bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae7ba expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739077.067321:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089400/0x7ee4453e4a1ae6d3 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae6da expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739077.067329:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089400/0x7ee4453e4a1ae6d3 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae6da expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739077.067344:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089400/0x7ee4453e4a1ae6d3 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae6da expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739077.067351:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739077.067357:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089400/0x7ee4453e4a1ae6d3 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae6da expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739077.067369:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ae4f7 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae4fe expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739077.067376:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ae4f7 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae4fe expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739077.067389:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ae4f7 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae4fe expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739077.067399:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739077.067409:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa073069ab200/0x7ee4453e4a1ae4f7 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae4fe expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739077.067444:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089800/0x7ee4453e4a1ae4a3 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xe1386388 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae4aa expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739077.067451:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089800/0x7ee4453e4a1ae4a3 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xe1386388 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae4aa expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739077.067467:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089800/0x7ee4453e4a1ae4a3 lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xe1386388 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae4aa expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739077.067474:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739077.067479:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089800/0x7ee4453e4a1ae4a3 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xe1386388 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae4aa expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739077.067491:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5e00/0x7ee4453e4a1ae44f lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae456 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739077.067498:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5e00/0x7ee4453e4a1ae44f lrc: 3/0,0 mode: PW/PW res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae456 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739077.067513:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5e00/0x7ee4453e4a1ae44f lrc: 2/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae456 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739077.067518:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739077.067522:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07404cc5e00/0x7ee4453e4a1ae44f lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae456 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1787739077.067534:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968472 is committed 00000001:00080000:2.0:1787739077.067550:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968465 is committed 00000020:00080000:2.0:1787739077.067555:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa074078cb480 refcount 2 committed 0 result 0 batchid 388 00000020:00080000:2.0:1787739077.067559:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00400 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa074078cbf00 00000020:00080000:2.0:1787739077.067563:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].379 00000020:00080000:2.0:1787739077.067577:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00900 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa074034d5200 00000020:00080000:2.0:1787739077.067580:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].379 00000020:00080000:2.0:1787739077.067600:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa0730610e480 refcount 2 committed 0 result 0 batchid 387 00000020:00080000:2.0:1787739077.067604:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2b00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730610ef00 00000020:00080000:2.0:1787739077.067607:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].378 00000020:00080000:2.0:1787739077.067609:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2000 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306a10000 00000020:00080000:2.0:1787739077.067612:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].378 00000020:00080000:2.0:1787739077.067627:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa0730610e600 refcount 2 committed 0 result 0 batchid 386 00000020:00080000:2.0:1787739077.067631:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2d00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa0730610e6c0 00000020:00080000:2.0:1787739077.067633:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].377 00000020:00080000:2.0:1787739077.067636:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073072c2c00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306a10800 00000020:00080000:2.0:1787739077.067639:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].377 00000020:00080000:2.0:1787739077.067646:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa0740a459000 refcount 2 committed 0 result 0 batchid 385 00000020:00080000:2.0:1787739077.067649:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07302edd700 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa0740a459180 00000020:00080000:2.0:1787739077.067651:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].376 00000020:00080000:2.0:1787739077.067654:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07302edd500 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073027da000 00000020:00080000:2.0:1787739077.067656:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].376 00000020:00080000:2.0:1787739077.067673:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa074118e0480 refcount 2 committed 0 result 0 batchid 384 00000020:00080000:2.0:1787739077.067677:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c691600 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07306887e40 00000020:00080000:2.0:1787739077.067679:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].375 00000020:00080000:2.0:1787739077.067681:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c691100 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306aad800 00000020:00080000:2.0:1787739077.067683:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].375 00000020:00080000:2.0:1787739077.067699:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07302e89780 refcount 2 committed 0 result 0 batchid 383 00000020:00080000:2.0:1787739077.067702:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00d00 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07302e89f00 00000020:00080000:2.0:1787739077.067704:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].374 00000020:00080000:2.0:1787739077.067706:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07302f96e00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073040e9800 00000020:00080000:2.0:1787739077.067709:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].374 00000020:00080000:2.0:1787739077.067715:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07435539240 refcount 2 committed 0 result 0 batchid 382 00000020:00080000:2.0:1787739077.067717:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07302f96700 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07435539a80 00000020:00080000:2.0:1787739077.067719:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].373 00000020:00080000:2.0:1787739077.067721:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07302f96500 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073040e9000 00000020:00080000:2.0:1787739077.067724:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].373 00000020:00080000:2.0:1787739077.067731:0:7453:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt ffffa07306234240 refcount 2 committed 0 result 0 batchid 381 00000020:00080000:2.0:1787739077.067734:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c691400 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th ffffa07306234180 00000020:00080000:2.0:1787739077.067736:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000401:0x3:0x0].372 00000020:00080000:2.0:1787739077.067738:0:7453:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c691f00 obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304df9600 00000020:00080000:2.0:1787739077.067741:0:7453:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000400:0x3:0x0].372 00000020:00080000:2.0:1787739077.067760:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 388 committed batchid 380 00000020:00080000:2.0:1787739077.067817:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 380 00000100:00100000:3.0:1787739077.068235:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665780736:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:3.0:1787739077.068298:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739077.068347:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1787739077.068353:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665780608:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:3.0:1787739077.068373:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739077.068400:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665781120:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:3.0:1787739077.068443:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739077.068477:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665780992:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:3.0:1787739077.068503:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739077.068522:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787739077.068527:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665780864:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:3.0:1787739077.068547:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739077.068590:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665780736 00000100:00100000:3.0:1787739077.068597:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665780736 00000100:00100000:3.0:1787739077.068636:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665780608 00000100:00100000:3.0:1787739077.068640:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665780608 00000100:00100000:3.0:1787739077.068653:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665781120 00000100:00100000:3.0:1787739077.068655:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665781120 00000100:00100000:3.0:1787739077.068665:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665780992 00000100:00100000:3.0:1787739077.068667:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665780992 00000100:00100000:3.0:1787739077.068676:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665780864 00000100:00100000:3.0:1787739077.068678:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665780864 00000100:00100000:3.0:1787739077.068690:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1419 00000100:00100000:3.0:1787739077.068708:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f70c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+17:3653:x1874579665780736:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:3.0:1787739077.068721:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739077.068744:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0800/0x7ee4453e4a1ae6da lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xd1c5f96a bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae6d3 expref: 17 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739077.068759:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xd1c5f96a (ffffa074419d5600) 00010000:00010000:3.0:1787739077.068768:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739077.068805:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f70c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+16:3653:x1874579665780736:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 98us (506us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739077.068813:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1419 00000100:00100000:3.0:1787739077.068829:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1420 00000100:00100000:3.0:1787739077.068835:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f6580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+15:3653:x1874579665780608:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:3.0:1787739077.068842:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739077.068853:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088000/0x7ee4453e4a1ae7ba lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x98a090de bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae7b3 expref: 15 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739077.068863:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x98a090de (ffffa07305ca2800) 00010000:00010000:3.0:1787739077.068870:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739077.068893:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f6580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+14:3653:x1874579665780608:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 59us (519us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739077.068899:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1420 00000100:00100000:3.0:1787739077.068910:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1421 00000100:00100000:3.0:1787739077.068915:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+13:3653:x1874579665781120:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:3.0:1787739077.068932:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1787739077.069211:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1422 00000100:00100000:1.0:1787739077.069220:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f4780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+13:3651:x1874579665780992:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:1.0:1787739077.069229:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739077.069241:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1ae4aa lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xe1386388 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae4a3 expref: 13 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739077.069252:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xe1386388 (ffffa074419d5500) 00010000:00010000:1.0:1787739077.069258:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739077.069292:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f4780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+12:3651:x1874579665780992:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 73us (788us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739077.069298:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1422 00000100:00100000:1.0:1787739077.069313:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1423 00000100:00100000:1.0:1787739077.069319:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+11:3651:x1874579665780864:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:1.0:1787739077.069326:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739077.069336:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841200/0x7ee4453e4a1ae4fe lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0x6ae86992 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae4f7 expref: 11 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739077.069348:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0x6ae86992 (ffffa07305eae300) 00010000:00010000:1.0:1787739077.069354:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739077.069375:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0001-mdtlov_UUID+10:3651:x1874579665780864:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 57us (828us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739077.069382:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1423 00010000:00010000:3.0:1787739077.069874:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088400/0x7ee4453e4a1ae456 lrc: 0/0,0 mode: --/PW res: [0x200000400:0x5:0x0].0xce9d12a2 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae44f expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739077.069886:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0x5:0x0].0xce9d12a2 (ffffa07305ca2700) 00010000:00010000:3.0:1787739077.069893:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739077.069997:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0001-mdtlov_UUID+8:3653:x1874579665781120:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 1020us (1492us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739077.070005:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1421 00000100:00100000:3.0:1787739077.070112:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665780608:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:3.0:1787739077.070126:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce1680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665781120:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:3.0:1787739077.070135:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665780736:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:3.0:1787739077.070169:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce0780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665780864:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:3.0:1787739077.070179:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce0b40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665780992:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:2.0:1787739077.186369:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739077.186452:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739077.186479:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739077.186500:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739077.186528:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739077.186557:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652544512 00000100:00100000:2.0:1787739077.186566:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652544512 00000100:00100000:2.0:1787739077.186618:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652544384 00000100:00100000:2.0:1787739077.186621:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652544384 00000100:00100000:2.0:1787739077.186643:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 680 00000100:00100000:2.0:1787739077.186652:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2408:x1874579652544512:12345-192.168.203.15@tcp:400:kworker.0 00000100:00100000:2.0:1787739077.186724:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2408:x1874579652544512:12345-192.168.203.15@tcp:400:kworker.0 Request processed in 72us (270us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739077.186737:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 680 00000100:00100000:2.0:1787739077.186754:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 681 00000100:00100000:2.0:1787739077.186760:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:2408:x1874579652544384:12345-192.168.203.15@tcp:400:kworker.0 00000100:00100000:2.0:1787739077.186788:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce0b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:2408:x1874579652544384:12345-192.168.203.15@tcp:400:kworker.0 Request processed in 28us (287us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739077.186793:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 681 00000100:00100000:0.0:1787739077.186972:0:6516:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652544256 00000100:00100000:0.0:1787739077.186976:0:6516:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652544256 00000100:00100000:0.0:1787739077.187006:0:6516:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 455 00000100:00100000:0.0:1787739077.187013:0:6516:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0740ea10050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:53427813-ebb7-48fc-94b6-ef03330d7ee5+10:2408:x1874579652544256:12345-192.168.203.15@tcp:400:kworker.0 00000100:00100000:0.0:1787739077.187041:0:6516:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0740ea10050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:53427813-ebb7-48fc-94b6-ef03330d7ee5+10:2408:x1874579652544256:12345-192.168.203.15@tcp:400:kworker.0 Request processed in 28us (682us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739077.187047:0:6516:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 455 00000100:00080000:0.0:1787739077.187073:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1787739077.187077:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1787739077.187103:0:14677:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652544768 00000100:00100000:0.0:1787739077.187106:0:14677:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652544768 00000100:00100000:0.0:1787739077.187133:0:14677:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652544640 00000100:00100000:0.0:1787739077.187135:0:14677:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652544640 00000100:00100000:0.0:1787739077.187154:0:14677:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 367 00000100:00100000:0.0:1787739077.187173:0:14677:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652544768:12345-192.168.203.15@tcp:400:kworker.0 00000100:00100000:0.0:1787739077.187200:0:14677:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce0f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2408:x1874579652544768:12345-192.168.203.15@tcp:400:kworker.0 Request processed in 40us (721us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739077.187206:0:14677:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 367 00000100:00100000:0.0:1787739077.187217:0:14677:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 368 00000100:00100000:0.0:1787739077.187223:0:14677:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c406c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2408:x1874579652544640:12345-192.168.203.15@tcp:400:kworker.0 00000100:00100000:0.0:1787739077.187243:0:14677:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c406c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:2408:x1874579652544640:12345-192.168.203.15@tcp:400:kworker.0 Request processed in 21us (716us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739077.187250:0:14677:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 368 00000100:00080000:0.0:1787739077.187475:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1787739077.187478:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1787739077.955303:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739078.978653:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739079.106832:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a31a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665781248:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1787739079.106928:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739079.107062:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665781248 00000100:00100000:0.0:1787739079.107070:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665781248 00000100:00100000:0.0:1787739079.107140:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 370 00000100:00100000:0.0:1787739079.107151:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f52c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874579665781248:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1787739079.107180:0:8441:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8646080 pending 0 free 3923529728 avail 3692007424 00000100:00100000:0.0:1787739079.107224:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f52c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3653:x1874579665781248:12345-0@lo:13:osp-pre-0-1.0 Request processed in 75us (298us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739079.107231:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 370 00000100:00080000:3.0:1787739079.107785:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787739079.107833:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a31a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665781248:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1787739079.108322:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787739079.108333:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305656d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665781376:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787739079.108373:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739079.108465:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665781376 00000100:00100000:2.0:1787739079.108471:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665781376 00000100:00100000:2.0:1787739079.108554:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 371 00000100:00100000:2.0:1787739079.108562:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a312c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665781376:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1787739079.108575:0:8440:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 8646080 pending 0 free 3923529728 avail 3692007424 00000100:00100000:2.0:1787739079.108614:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a312c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665781376:12345-0@lo:13:osp-pre-0-0.0 Request processed in 54us (240us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739079.108620:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 371 00000100:00100000:2.0:1787739079.108649:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305656d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665781376:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1787739079.114118:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1787739079.618741:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787739079.618765:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665781504:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1787739079.618834:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739079.619149:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665781504 00000100:00100000:3.0:1787739079.619156:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665781504 00000100:00100000:3.0:1787739079.619199:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3616 00000100:00100000:3.0:1787739079.619206:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305656d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:3650:x1874579665781504:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1787739079.619235:0:12410:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1445142528 avail 1298903040 00000100:00100000:3.0:1787739079.619294:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305656d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:3650:x1874579665781504:12345-0@lo:41:osp-pre-1-0.0 Request processed in 88us (460us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739079.619301:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3616 00000100:00100000:1.0:1787739079.619367:0:3650:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968472 gen 2 00000004:00080000:1.0:1787739079.619373:0:3650:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968433 committed transno 4294968472 00000020:00080000:1.0:1787739079.619378:0:3650:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07306241900 refcount 2 committed 0 result 0 batchid 395 00000020:00080000:1.0:1787739079.619383:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00200 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07306241600 00000020:00080000:1.0:1787739079.619386:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].390 00000020:00080000:1.0:1787739079.619389:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00100 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07305406600 00000020:00080000:1.0:1787739079.619392:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].391 00000004:00080000:1.0:1787739079.619444:0:3650:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968444 committed transno 4294968472 00000020:00080000:1.0:1787739079.619447:0:3650:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07306241300 refcount 2 committed 0 result 0 batchid 396 00000020:00080000:1.0:1787739079.619450:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00a00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa074118e0d80 00000020:00080000:1.0:1787739079.619453:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].391 00000020:00080000:1.0:1787739079.619455:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00e00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073040e8e00 00000020:00080000:1.0:1787739079.619457:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].392 00000004:00080000:1.0:1787739079.619466:0:3650:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968454 committed transno 4294968472 00000020:00080000:1.0:1787739079.619469:0:3650:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07303803000 refcount 2 committed 0 result 0 batchid 397 00000020:00080000:1.0:1787739079.619472:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c691000 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07303803f00 00000020:00080000:1.0:1787739079.619474:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].392 00000020:00080000:1.0:1787739079.619477:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c691e00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306aada00 00000020:00080000:1.0:1787739079.619479:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].393 00000004:00080000:1.0:1787739079.619488:0:3650:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968455 committed transno 4294968472 00000020:00080000:1.0:1787739079.619491:0:3650:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa073062886c0 refcount 2 committed 0 result 0 batchid 398 00000020:00080000:1.0:1787739079.619493:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07302edd400 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07306288540 00000020:00080000:1.0:1787739079.619496:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].393 00000020:00080000:1.0:1787739079.619499:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07302eddf00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07303a40600 00000020:00080000:1.0:1787739079.619502:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].394 00000004:00080000:1.0:1787739079.619511:0:3650:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968456 committed transno 4294968472 00000020:00080000:1.0:1787739079.619515:0:3650:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa073062480c0 refcount 2 committed 0 result 0 batchid 399 00000020:00080000:1.0:1787739079.619517:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c691900 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa0740a46c0c0 00000020:00080000:1.0:1787739079.619519:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].394 00000020:00080000:1.0:1787739079.619521:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c691a00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306aadc00 00000020:00080000:1.0:1787739079.619524:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].395 00000004:00080000:1.0:1787739079.619532:0:3650:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968457 committed transno 4294968472 00000020:00080000:1.0:1787739079.619535:0:3650:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07302e89e40 refcount 2 committed 0 result 0 batchid 400 00000020:00080000:1.0:1787739079.619537:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00600 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07302e89600 00000020:00080000:1.0:1787739079.619540:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].395 00000020:00080000:1.0:1787739079.619542:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00500 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07303fe0400 00000020:00080000:1.0:1787739079.619545:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].396 00000004:00080000:1.0:1787739079.619553:0:3650:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968458 committed transno 4294968472 00000020:00080000:1.0:1787739079.619556:0:3650:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa073062483c0 refcount 2 committed 0 result 0 batchid 401 00000020:00080000:1.0:1787739079.619558:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c691800 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07306248540 00000020:00080000:1.0:1787739079.619561:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].396 00000020:00080000:1.0:1787739079.619562:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c691500 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304010a00 00000020:00080000:1.0:1787739079.619565:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].397 00000004:00080000:1.0:1787739079.619573:0:3650:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968462 committed transno 4294968472 00000020:00080000:1.0:1787739079.619576:0:3650:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa0740352a6c0 refcount 2 committed 0 result 0 batchid 402 00000020:00080000:1.0:1787739079.619579:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00c00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa0740352a840 00000020:00080000:1.0:1787739079.619581:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].397 00000020:00080000:1.0:1787739079.619583:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00700 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073055b8200 00000020:00080000:1.0:1787739079.619586:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].398 00000004:00080000:1.0:1787739079.619595:0:3650:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968466 committed transno 4294968472 00000020:00080000:1.0:1787739079.619598:0:3650:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07410b4d0c0 refcount 2 committed 0 result 0 batchid 403 00000020:00080000:1.0:1787739079.619600:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c693700 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07410b4d840 00000020:00080000:1.0:1787739079.619603:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].398 00000020:00080000:1.0:1787739079.619605:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07305c00300 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa073055b9800 00000020:00080000:1.0:1787739079.619607:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].399 00000004:00080000:1.0:1787739079.619616:0:3650:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968467 committed transno 4294968472 00000020:00080000:1.0:1787739079.619620:0:3650:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa073062883c0 refcount 2 committed 0 result 0 batchid 404 00000020:00080000:1.0:1787739079.619622:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa073060b2b00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07306288b40 00000020:00080000:1.0:1787739079.619625:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].399 00000020:00080000:1.0:1787739079.619627:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07306871500 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07304d5ba00 00000020:00080000:1.0:1787739079.619629:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].400 00000004:00080000:1.0:1787739079.619637:0:3650:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968468 committed transno 4294968472 00000020:00080000:1.0:1787739079.619641:0:3650:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa073062340c0 refcount 2 committed 0 result 0 batchid 405 00000020:00080000:1.0:1787739079.619643:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07410c8d300 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa073062489c0 00000020:00080000:1.0:1787739079.619646:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].400 00000020:00080000:1.0:1787739079.619647:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0741877da00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07411bdac00 00000020:00080000:1.0:1787739079.619650:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].401 00000004:00080000:1.0:1787739079.619657:0:3650:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968469 committed transno 4294968472 00000020:00080000:1.0:1787739079.619660:0:3650:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07410b4dd80 refcount 2 committed 0 result 0 batchid 406 00000020:00080000:1.0:1787739079.619662:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c693800 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07410b4d180 00000020:00080000:1.0:1787739079.619665:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].401 00000020:00080000:1.0:1787739079.619668:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa0743c693b00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306a2ce00 00000020:00080000:1.0:1787739079.619670:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].402 00000004:00080000:1.0:1787739079.619679:0:3650:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968470 committed transno 4294968472 00000020:00080000:1.0:1787739079.619682:0:3650:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa07306248780 refcount 2 committed 0 result 0 batchid 407 00000020:00080000:1.0:1787739079.619685:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07302e62300 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07410b4dcc0 00000020:00080000:1.0:1787739079.619687:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].402 00000020:00080000:1.0:1787739079.619689:0:3650:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa07302e62b00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07306a2dc00 00000020:00080000:1.0:1787739079.619692:0:3650:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].403 00000004:00080000:1.0:1787739079.619702:0:3650:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968471 committed transno 4294968472 00000004:00080000:1.0:1787739079.619711:0:3650:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968472 committed transno 4294968472 00000100:00100000:1.0:1787739079.619724:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f6d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665781504:0@lo:41:osp-pre-1-0.0 00000020:00080000:3.0:1787739079.619792:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 407 committed batchid 394 00000020:00080000:3.0:1787739079.619852:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 394 00000020:00080000:1.0:1787739079.624327:0:6502:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 407 batchid updated 00000020:00080000:0.0:1787739079.626243:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 407 00000004:00080000:0.0:1787739079.626310:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306107000 with version 759 00000020:00080000:0.0:1787739079.626329:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 395 cancel update log [0x240000401:0x3:0x0].390: rc = 0 00000020:00080000:0.0:1787739079.626404:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 395 cancel update log [0x200000400:0x3:0x0].391: rc = 0 00000004:00080000:0.0:1787739079.626452:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306107d80 with version 760 00000020:00080000:0.0:1787739079.626455:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 396 cancel update log [0x240000401:0x3:0x0].391: rc = 0 00000020:00080000:0.0:1787739079.626474:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 396 cancel update log [0x200000400:0x3:0x0].392: rc = 0 00000004:00080000:0.0:1787739079.626491:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306107540 with version 761 00000020:00080000:0.0:1787739079.626494:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 397 cancel update log [0x240000401:0x3:0x0].392: rc = 0 00000020:00080000:0.0:1787739079.626514:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 397 cancel update log [0x200000400:0x3:0x0].393: rc = 0 00000004:00080000:0.0:1787739079.626535:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306107cc0 with version 762 00000020:00080000:0.0:1787739079.626538:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 398 cancel update log [0x240000401:0x3:0x0].393: rc = 0 00000020:00080000:0.0:1787739079.626557:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 398 cancel update log [0x200000400:0x3:0x0].394: rc = 0 00000004:00080000:0.0:1787739079.626576:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07303c6e0c0 with version 763 00000020:00080000:0.0:1787739079.626578:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 399 cancel update log [0x240000401:0x3:0x0].394: rc = 0 00000020:00080000:0.0:1787739079.626596:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 399 cancel update log [0x200000400:0x3:0x0].395: rc = 0 00000004:00080000:0.0:1787739079.626614:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07303c6e9c0 with version 764 00000020:00080000:0.0:1787739079.626617:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 400 cancel update log [0x240000401:0x3:0x0].395: rc = 0 00000020:00080000:0.0:1787739079.626636:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 400 cancel update log [0x200000400:0x3:0x0].396: rc = 0 00000004:00080000:0.0:1787739079.626655:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0741042ab40 with version 765 00000020:00080000:0.0:1787739079.626657:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 401 cancel update log [0x240000401:0x3:0x0].396: rc = 0 00000020:00080000:0.0:1787739079.626674:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 401 cancel update log [0x200000400:0x3:0x0].397: rc = 0 00000004:00080000:0.0:1787739079.626691:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0741042a900 with version 766 00000020:00080000:0.0:1787739079.626694:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 402 cancel update log [0x240000401:0x3:0x0].397: rc = 0 00000020:00080000:0.0:1787739079.626712:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 402 cancel update log [0x200000400:0x3:0x0].398: rc = 0 00000004:00080000:0.0:1787739079.626731:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0741042a780 with version 767 00000020:00080000:0.0:1787739079.626734:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 403 cancel update log [0x240000401:0x3:0x0].398: rc = 0 00000020:00080000:0.0:1787739079.626751:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 403 cancel update log [0x200000400:0x3:0x0].399: rc = 0 00000004:00080000:0.0:1787739079.626770:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0741042a600 with version 768 00000020:00080000:0.0:1787739079.626773:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 404 cancel update log [0x240000401:0x3:0x0].399: rc = 0 00000020:00080000:0.0:1787739079.626791:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 404 cancel update log [0x200000400:0x3:0x0].400: rc = 0 00000004:00080000:0.0:1787739079.626810:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa074052af0c0 with version 769 00000020:00080000:0.0:1787739079.626812:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 405 cancel update log [0x240000401:0x3:0x0].400: rc = 0 00000020:00080000:0.0:1787739079.626829:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 405 cancel update log [0x200000400:0x3:0x0].401: rc = 0 00000004:00080000:0.0:1787739079.626870:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa074037136c0 with version 770 00000020:00080000:0.0:1787739079.626874:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 406 cancel update log [0x240000401:0x3:0x0].401: rc = 0 00000020:00080000:0.0:1787739079.626978:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 406 cancel update log [0x200000400:0x3:0x0].402: rc = 0 00000004:00080000:0.0:1787739079.627000:0:6555:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07403713e40 with version 771 00000020:00080000:0.0:1787739079.627003:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osp-MDT0000: batchid 407 cancel update log [0x240000401:0x3:0x0].402: rc = 0 00000020:00080000:0.0:1787739079.627024:0:6555:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osd: batchid 407 cancel update log [0x200000400:0x3:0x0].403: rc = 0 00000020:00080000:0.0:1787739079.627030:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 407 00000004:00080000:2.0:1787739079.627097:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 759 rpc_version 759 00000020:00100000:2.0:1787739079.627129:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739079.627140:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304ce0780 x1874579665781760/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739079.627162:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce0780 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665781760:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739079.627223:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739079.627283:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b95e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739079.627375:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665781760 00000100:00100000:0.0:1787739079.627383:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665781760 00000100:00100000:0.0:1787739079.627538:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3617 00000100:00100000:0.0:1787739079.627548:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:7740:x1874579665781760:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739079.627684:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665781760:12345-0@lo:1000:osp_up1-0.0 Request processed in 137us (460us total) trans 4294968473 rc 0/0 00000100:00100000:0.0:1787739079.627692:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3617 00000100:00100000:2.0:1787739079.627768:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce0780 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665781760:0@lo:1000:osp_up1-0.0 00000004:00080000:2.0:1787739079.627778:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 760 rpc_version 760 00000020:00100000:2.0:1787739079.627789:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739079.627794:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304ce1680 x1874579665782016/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739079.627807:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce1680 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665782016:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739079.627831:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739079.635528:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b95e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739079.646566:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665782016 00000100:00100000:3.0:1787739079.646580:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665782016 00000100:00100000:3.0:1787739079.646638:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3618 00000100:00100000:3.0:1787739079.646647:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+10:7740:x1874579665782016:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739079.646846:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+11:7740:x1874579665782016:12345-0@lo:1000:osp_up1-0.0 Request processed in 200us (19013us total) trans 4294968474 rc 0/0 00000100:00100000:3.0:1787739079.646854:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3618 00000100:00100000:2.0:1787739079.647054:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce1680 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665782016:0@lo:1000:osp_up1-0.0 00000004:00080000:2.0:1787739079.647067:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 761 rpc_version 761 00000020:00100000:2.0:1787739079.647087:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739079.647093:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304ce3480 x1874579665782272/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739079.647112:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce3480 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665782272:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739079.647149:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739079.647188:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b95e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739079.647294:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665782272 00000100:00100000:0.0:1787739079.647299:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665782272 00000100:00100000:0.0:1787739079.647331:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3619 00000100:00100000:0.0:1787739079.647337:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+11:7740:x1874579665782272:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739079.647410:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+12:7740:x1874579665782272:12345-0@lo:1000:osp_up1-0.0 Request processed in 73us (260us total) trans 4294968475 rc 0/0 00000100:00100000:0.0:1787739079.647433:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3619 00000100:00100000:2.0:1787739079.647518:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce3480 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665782272:0@lo:1000:osp_up1-0.0 00000004:00080000:2.0:1787739079.647525:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 762 rpc_version 762 00000020:00100000:2.0:1787739079.647537:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739079.647542:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304ce12c0 x1874579665782528/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739079.647555:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce12c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665782528:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739079.647582:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739079.647609:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b95e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739079.647662:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665782528 00000100:00100000:0.0:1787739079.647666:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665782528 00000100:00100000:0.0:1787739079.647685:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3620 00000100:00100000:0.0:1787739079.647691:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+12:7740:x1874579665782528:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739079.647768:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+13:7740:x1874579665782528:12345-0@lo:1000:osp_up1-0.0 Request processed in 78us (185us total) trans 4294968476 rc 0/0 00000100:00100000:0.0:1787739079.647775:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3620 00000100:00100000:2.0:1787739079.647842:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce12c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665782528:0@lo:1000:osp_up1-0.0 00000004:00080000:2.0:1787739079.647849:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 763 rpc_version 763 00000020:00100000:2.0:1787739079.647861:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739079.647866:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304ce0f00 x1874579665782784/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739079.647879:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce0f00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665782784:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739079.647922:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739079.647942:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b95e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739079.648046:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665782784 00000100:00100000:0.0:1787739079.648050:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665782784 00000100:00100000:0.0:1787739079.648067:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3621 00000100:00100000:0.0:1787739079.648072:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042452c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+13:7740:x1874579665782784:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739079.648142:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042452c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+14:7740:x1874579665782784:12345-0@lo:1000:osp_up1-0.0 Request processed in 69us (219us total) trans 4294968477 rc 0/0 00000100:00100000:0.0:1787739079.648149:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3621 00000100:00100000:2.0:1787739079.648246:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce0f00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665782784:0@lo:1000:osp_up1-0.0 00000004:00080000:2.0:1787739079.648254:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 764 rpc_version 764 00000020:00100000:2.0:1787739079.648264:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739079.648269:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304245e00 x1874579665783040/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739079.648281:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304245e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665783040:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739079.648304:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739079.648324:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b95e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739079.648376:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665783040 00000100:00100000:3.0:1787739079.648380:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665783040 00000100:00100000:3.0:1787739079.648397:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3622 00000100:00100000:3.0:1787739079.648403:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304247c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+14:7740:x1874579665783040:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739079.648496:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304247c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+15:7740:x1874579665783040:12345-0@lo:1000:osp_up1-0.0 Request processed in 92us (190us total) trans 4294968478 rc 0/0 00000100:00100000:3.0:1787739079.648506:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3622 00000100:00100000:2.0:1787739079.648574:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304245e00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665783040:0@lo:1000:osp_up1-0.0 00000004:00080000:2.0:1787739079.648581:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 765 rpc_version 765 00000020:00100000:2.0:1787739079.648592:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739079.648597:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa073042470c0 x1874579665783296/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739079.648614:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042470c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665783296:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739079.648641:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739079.648668:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b95e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739079.648775:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665783296 00000100:00100000:1.0:1787739079.648780:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665783296 00000100:00100000:1.0:1787739079.648802:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3623 00000100:00100000:1.0:1787739079.648808:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+15:7740:x1874579665783296:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739079.648966:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+16:7740:x1874579665783296:12345-0@lo:1000:osp_up1-0.0 Request processed in 158us (323us total) trans 4294968479 rc 0/0 00000100:00100000:1.0:1787739079.648974:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3623 00000100:00100000:2.0:1787739079.649149:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042470c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665783296:0@lo:1000:osp_up1-0.0 00000004:00080000:2.0:1787739079.649156:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 766 rpc_version 766 00000020:00100000:2.0:1787739079.649167:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739079.649172:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07409fb52c0 x1874579665783552/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739079.649184:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb52c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665783552:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739079.649206:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739079.649225:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b95e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739079.649271:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665783552 00000100:00100000:3.0:1787739079.649275:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665783552 00000100:00100000:3.0:1787739079.649292:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3624 00000100:00100000:3.0:1787739079.649297:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+16:7740:x1874579665783552:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739079.649369:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+17:7740:x1874579665783552:12345-0@lo:1000:osp_up1-0.0 Request processed in 71us (161us total) trans 4294968480 rc 0/0 00000100:00100000:3.0:1787739079.649375:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3624 00000100:00100000:2.0:1787739079.649455:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb52c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665783552:0@lo:1000:osp_up1-0.0 00000004:00080000:2.0:1787739079.649461:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 767 rpc_version 767 00000020:00100000:2.0:1787739079.649472:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739079.649477:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07409fb6580 x1874579665783808/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739079.649488:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb6580 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665783808:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739079.649512:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739079.649528:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b95e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739079.649583:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665783808 00000100:00100000:0.0:1787739079.649588:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665783808 00000100:00100000:0.0:1787739079.649608:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3625 00000100:00100000:0.0:1787739079.649614:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb7c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+17:7740:x1874579665783808:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739079.649697:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb7c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+18:7740:x1874579665783808:12345-0@lo:1000:osp_up1-0.0 Request processed in 82us (182us total) trans 4294968481 rc 0/0 00000100:00100000:0.0:1787739079.649703:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3625 00000100:00100000:2.0:1787739079.649812:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb6580 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665783808:0@lo:1000:osp_up1-0.0 00000004:00080000:2.0:1787739079.649820:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 768 rpc_version 768 00000020:00100000:2.0:1787739079.649831:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739079.649836:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07409fb4780 x1874579665784064/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739079.649854:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb4780 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665784064:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739079.649898:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739079.649924:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b95e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739079.650030:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665784064 00000100:00100000:0.0:1787739079.650035:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665784064 00000100:00100000:0.0:1787739079.650065:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3626 00000100:00100000:0.0:1787739079.650072:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410b09c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+18:7740:x1874579665784064:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739079.650157:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410b09c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+19:7740:x1874579665784064:12345-0@lo:1000:osp_up1-0.0 Request processed in 85us (258us total) trans 4294968482 rc 0/0 00000100:00100000:0.0:1787739079.650164:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3626 00000100:00100000:2.0:1787739079.650489:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb4780 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665784064:0@lo:1000:osp_up1-0.0 00000004:00080000:2.0:1787739079.650498:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 769 rpc_version 769 00000020:00100000:2.0:1787739079.650510:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739079.650516:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07409fb4b40 x1874579665784320/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739079.650528:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb4b40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665784320:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739079.650603:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739079.650633:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b95e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739079.650689:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665784320 00000100:00100000:1.0:1787739079.650693:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665784320 00000100:00100000:1.0:1787739079.650727:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3627 00000100:00100000:1.0:1787739079.650733:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+19:7740:x1874579665784320:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739079.650814:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+20:7740:x1874579665784320:12345-0@lo:1000:osp_up1-0.0 Request processed in 81us (210us total) trans 4294968483 rc 0/0 00000100:00100000:1.0:1787739079.650824:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3627 00000100:00100000:2.0:1787739079.651001:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb4b40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665784320:0@lo:1000:osp_up1-0.0 00000004:00080000:2.0:1787739079.651009:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 770 rpc_version 770 00000020:00100000:2.0:1787739079.651034:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739079.651039:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07409fb4f00 x1874579665784576/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739079.651051:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb4f00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665784576:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739079.651076:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739079.651097:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b95e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739079.651207:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665784576 00000100:00100000:1.0:1787739079.651211:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665784576 00000100:00100000:1.0:1787739079.651242:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3628 00000100:00100000:1.0:1787739079.651249:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+20:7740:x1874579665784576:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739079.651327:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+21:7740:x1874579665784576:12345-0@lo:1000:osp_up1-0.0 Request processed in 78us (249us total) trans 4294968484 rc 0/0 00000100:00100000:1.0:1787739079.651333:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3628 00000100:00100000:2.0:1787739079.651455:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb4f00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665784576:0@lo:1000:osp_up1-0.0 00000004:00080000:2.0:1787739079.651462:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 771 rpc_version 771 00000020:00100000:2.0:1787739079.651474:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739079.651478:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07409fb5680 x1874579665784832/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739079.651495:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb5680 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665784832:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739079.651519:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739079.651536:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07410b95e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739079.651589:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665784832 00000100:00100000:0.0:1787739079.651594:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665784832 00000100:00100000:0.0:1787739079.651624:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3629 00000100:00100000:0.0:1787739079.651632:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+21:7740:x1874579665784832:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739079.651717:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+22:7740:x1874579665784832:12345-0@lo:1000:osp_up1-0.0 Request processed in 85us (196us total) trans 4294968485 rc 0/0 00000100:00100000:0.0:1787739079.651722:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3629 00000100:00100000:2.0:1787739079.651822:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb5680 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665784832:0@lo:1000:osp_up1-0.0 00010000:00010000:1.0:1787739080.003006:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739080.132270:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665784960:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1787739080.132339:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739080.132505:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665784960 00000100:00100000:1.0:1787739080.132514:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665784960 00000100:00100000:1.0:1787739080.132568:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3630 00000100:00100000:1.0:1787739080.132576:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:3652:x1874579665784960:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1787739080.132609:0:6528:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1444106240 avail 1297866752 00000100:00100000:1.0:1787739080.134393:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+8:3652:x1874579665784960:12345-0@lo:41:osp-pre-0-1.0 Request processed in 1810us (2046us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739080.134406:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3630 00000100:00100000:0.0:1787739080.134677:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb6940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665784960:0@lo:41:osp-pre-0-1.0 00000001:02000400:2.0:1787739080.841902:0:20881:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity test 60h: striped directory with missing stripes can be accessed ========================================================== 06:11:19 (1787739079) 00010000:00010000:2.0:1787739081.026886:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739081.897874:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739081.898942:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652544896 00000100:00100000:3.0:1787739081.898954:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652544896 00000100:00100000:3.0:1787739081.899024:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23691 00000100:00100000:3.0:1787739081.899034:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb7c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:18315:x1874579652544896:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739081.899055:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739081.899075:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae8a1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0954d3 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.899135:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ae8a8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.899148:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ae8a8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.899160:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ae8a8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739081.899205:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 262144 00000004:00200000:3.0:1787739081.899209:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1787739081.899229:0:6525:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ae8a8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.899254:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1ae8a1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954d3 expref: 11 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.899270:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ae8a8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954d3 expref: 10 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.899278:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa07401795300) 00010000:00010000:3.0:1787739081.899281:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1787739081.899284:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aae00, rc 0) 00000100:00100000:3.0:1787739081.899354:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb7c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:18315:x1874579652544896:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 321us (1489us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739081.899361:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23691 00000100:00080000:2.0:1787739081.899802:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1787739081.902713:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739081.902780:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652545024 00000100:00100000:3.0:1787739081.902789:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652545024 00000100:00100000:3.0:1787739081.902846:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23692 00000100:00100000:3.0:1787739081.902854:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:18315:x1874579652545024:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739081.902873:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739081.902904:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ae8af lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0954da expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.902964:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae8b6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.902978:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae8b6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.902987:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae8b6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.903001:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae8bd lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.903007:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae8bd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.903012:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae8bd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.903062:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae8b6 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.903068:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae8b6 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.903073:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae8b6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.903079:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae8b6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.903085:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739081.903089:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae8b6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.903096:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae8b6 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.903106:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa07401795300) 00010000:00010000:3.0:1787739081.903109:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739081.903113:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae8b6 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.903120:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae8b6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.903132:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae8bd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.903138:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae8bd lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.903144:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae8bd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.903148:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739081.903151:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae8bd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.903157:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae8bd lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.903163:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xf5d6a2c1 (ffffa074331c2300) 00010000:00010000:3.0:1787739081.903165:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739081.903167:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae8bd lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.903172:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae8bd lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.903201:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ae8af lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954da expref: 11 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.903215:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa07401795300) 00010000:00010000:3.0:1787739081.903217:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739081.903221:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1ae8af lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954da expref: 11 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.903229:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa400, rc 0) 00000100:00100000:3.0:1787739081.903286:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb6940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:18315:x1874579652545024:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 433us (581us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739081.903294:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23692 00000100:00100000:3.0:1787739081.912146:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739081.912632:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652545280 00000100:00100000:0.0:1787739081.912641:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652545280 00000100:00100000:0.0:1787739081.912703:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23693 00000100:00100000:0.0:1787739081.912713:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:18315:x1874579652545280:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:0.0:1787739081.912766:0:6524:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739081.912791:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1ae8a8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954d3 expref: 10 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.912807:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa07401795300) 00010000:00010000:0.0:1787739081.912811:0:6524:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739081.912878:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae8c4 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.912907:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae8c4 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.912917:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae8c4 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.912932:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae8cb lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.912943:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae8cb lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.912949:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae8cb lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:02000000:0.0:1787739081.913555:0:6524:0:(lod_object.c:4506:lod_xattr_set_lmv()) *** cfs_fail_loc=188, val=0*** 00000040:00080000:0.0:1787739081.915336:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].404, 1760 off605784 00000040:00080000:0.0:1787739081.923770:0:6524:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].404, 1928 off657288 00000004:00080000:0.0:1787739081.923785:0:6524:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07406701180 with version 772 00000004:00080000:3.0:1787739081.923826:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 772 rpc_version 772 00000020:00100000:3.0:1787739081.923860:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739081.923870:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074074f7c00 x1874579665785216/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 3984/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739081.923957:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665785216:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739081.924023:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739081.924078:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739081.924120:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665785216 00000100:00100000:1.0:1787739081.924128:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665785216 00000100:00100000:1.0:1787739081.924207:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3631 00000100:00100000:1.0:1787739081.924216:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f5a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+22:7740:x1874579665785216:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739081.924564:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f5a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+23:7740:x1874579665785216:12345-0@lo:1000:osp_up1-0.0 Request processed in 348us (539us total) trans 4294968486 rc 0/0 00000100:00100000:1.0:1787739081.924572:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3631 00000100:00100000:0.0:1787739081.924681:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f7c00 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665785216:0@lo:1000:osp_up1-0.0 00010000:00010000:0.0:1787739081.924787:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1ae8d2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x152b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.924803:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1ae8d2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x152b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.924814:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1ae8d2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x152b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.924832:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1ae8d9 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.924838:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1ae8d9 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.924844:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1ae8d9 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.924865:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089200/0x7ee4453e4a1ae8e0 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.924872:0:6524:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089200/0x7ee4453e4a1ae8e0 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.924880:0:6524:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089200/0x7ee4453e4a1ae8e0 lrc: 3/0,1 mode: --/PW res: [0x240000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739081.927667:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a30780 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665785344:0@lo:101:mdt00_001.0 00000100:00100000:0.0:1787739081.927731:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739081.932639:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665785344 00000100:00100000:3.0:1787739081.932662:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665785344 00000100:00100000:3.0:1787739081.932717:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3632 00000100:00100000:3.0:1787739081.932726:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a330c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+23:6524:x1874579665785344:12345-0@lo:101:mdt00_001.0 00010000:00010000:3.0:1787739081.932749:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739081.932770:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ae8e7 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xce:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1ae8e0 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.932803:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ae8e7 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae8e0 expref: 24 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.932817:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ae8e7 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1ae8e0 expref: 24 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.932824:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xce:0x0].0x0 (ffffa074331c2300) 00010000:00010000:3.0:1787739081.932827:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739081.932831:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa400, rc 0) 00000100:00100000:3.0:1787739081.933032:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a330c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+24:6524:x1874579665785344:12345-0@lo:101:mdt00_001.0 Request processed in 307us (5300us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739081.933040:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3632 00000100:00100000:1.0:1787739081.933561:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07406701480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739081.933596:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a30780 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:6524:1874579665785344:0@lo:101:mdt00_001.0 00010000:00010000:1.0:1787739081.933616:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089200/0x7ee4453e4a1ae8e0 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae8e7 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.933628:0:6524:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089200/0x7ee4453e4a1ae8e0 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1ae8e7 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739081.933763:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a7552d91 reply state 00000000d0219484 transno 4294983492 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1ae8d9 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.933780:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1ae8d9 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.933788:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa074126f6600) 00010000:00010000:1.0:1787739081.933791:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739081.933794:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787739081.933801:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089200/0x7ee4453e4a1ae8e0 lrc: 4/0,1 mode: PW/PW res: [0x240000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae8e7 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.933810:0:6524:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089200/0x7ee4453e4a1ae8e0 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1ae8e7 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739081.933822:0:6524:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000401:0xce:0x0] 00000004:00010000:1.0:1787739081.933828:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a7552d91 reply state 00000000d0219484 transno 4294983492 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1ae8d2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x152b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.933836:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1ae8d2 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x152b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.933842:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152b:0x0].0x0 (ffffa074126f6200) 00010000:00010000:1.0:1787739081.933844:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739081.933847:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00200000:1.0:1787739081.933852:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x152b:0x0]: returning size 4096 00010000:00010000:1.0:1787739081.933863:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae8c4 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.933870:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae8c4 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.933877:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae8c4 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.933955:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae8c4 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.933962:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306089000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739081.933967:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae8c4 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.933975:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae8c4 lrc: 5/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.933985:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa07401795300) 00010000:00010000:1.0:1787739081.933988:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739081.933991:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae8c4 lrc: 3/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.933998:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae8c4 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739081.934012:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a7552d91 reply state 00000000d0219484 transno 4294983492 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae8cb lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.934020:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae8cb lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.934027:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xf5d6a2c1 (ffffa074126f6800) 00010000:00010000:1.0:1787739081.934029:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739081.934031:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 2 00000100:00100000:1.0:1787739081.934111:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f7840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:18315:x1874579652545280:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 21397us (21969us total) trans 4294983492 rc 0/0 00000100:00100000:1.0:1787739081.934119:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23693 00000100:00100000:3.0:1787739081.938933:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739081.939167:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652545408 00000100:00100000:0.0:1787739081.939177:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652545408 00000100:00100000:0.0:1787739081.939239:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23694 00000100:00100000:0.0:1787739081.939249:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f6d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:18160:x1874579652545408:12345-192.168.203.15@tcp:101:bash.0 00010000:00010000:0.0:1787739081.939273:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739081.939293:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088200/0x7ee4453e4a1ae8ee lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0954e1 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.939356:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae8f5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.939370:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae8f5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.939382:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae8f5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739081.939455:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 262144 00000004:00200000:0.0:1787739081.939459:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787739081.939480:0:13793:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae8f5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.939508:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088200/0x7ee4453e4a1ae8ee lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954e1 expref: 13 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.939525:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae8f5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954e1 expref: 12 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.939543:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa07401795300) 00010000:00010000:0.0:1787739081.939546:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787739081.939550:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306088a00, rc 0) 00000100:00100000:0.0:1787739081.939604:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f6d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18160:x1874579652545408:12345-192.168.203.15@tcp:101:bash.0 Request processed in 356us (678us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739081.939611:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23694 00000100:00100000:3.0:1787739081.944664:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739081.944776:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652545536 00000100:00100000:0.0:1787739081.944784:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652545536 00000100:00100000:0.0:1787739081.944841:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23695 00000100:00100000:0.0:1787739081.944851:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f770c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18160:x1874579652545536:12345-192.168.203.15@tcp:101:bash.0 00010000:00010000:0.0:1787739081.944869:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739081.944902:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae8fc lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0954e8 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.944963:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae903 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.944976:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae903 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.944986:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae903 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.944996:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae90a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.945003:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae90a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.945009:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae90a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.945071:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae911 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x152b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.945079:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae911 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.945086:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae911 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739081.945153:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x152b:0x0]: returning size 4096 00000004:00200000:0.0:1787739081.945158:0:6523:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000004:00010000:0.0:1787739081.945176:0:6523:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae911 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.945188:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae903 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.945195:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae903 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.945201:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae903 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.945211:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae903 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.945217:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306088e00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739081.945221:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae903 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.945228:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae903 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.945239:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa07401795300) 00010000:00010000:0.0:1787739081.945241:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739081.945246:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae903 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.945251:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae903 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.945263:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae90a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.945269:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae90a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.945274:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae90a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.945279:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306089600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739081.945282:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae90a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.945293:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae90a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.945299:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xf5d6a2c1 (ffffa074126f6800) 00010000:00010000:0.0:1787739081.945301:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739081.945304:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae90a lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.945309:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae90a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.945333:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae8fc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954e8 expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.945345:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae911 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x152b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954e8 expref: 13 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.945351:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152b:0x0].0x0 (ffffa074126f6200) 00010000:00010000:0.0:1787739081.945353:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1787739081.945355:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306088600, rc 0) 00000100:00100000:0.0:1787739081.945432:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f770c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:18160:x1874579652545536:12345-192.168.203.15@tcp:101:bash.0 Request processed in 560us (764us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739081.945440:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23695 00000100:00100000:2.0:1787739081.953736:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739081.953853:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652545664 00000100:00100000:3.0:1787739081.953862:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652545664 00000100:00100000:3.0:1787739081.953954:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23696 00000100:00100000:3.0:1787739081.953964:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+13:18160:x1874579652545664:12345-192.168.203.15@tcp:101:bash.0 00010000:00010000:3.0:1787739081.953982:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739081.954000:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae918 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0954ef expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.954055:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae91f lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.954068:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae91f lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.954084:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae91f lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739081.954128:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xce:0x0]: returning size 4096 00000004:00200000:3.0:1787739081.954133:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00010000:00010000:3.0:1787739081.954168:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1ae918 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954ef expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.954184:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae91f lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954ef expref: 14 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739081.954193:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa074126f6600) 00010000:00010000:3.0:1787739081.954196:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739081.954199:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab200, rc 0) 00000100:00100000:3.0:1787739081.954249:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561c780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:18160:x1874579652545664:12345-192.168.203.15@tcp:101:bash.0 Request processed in 287us (522us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739081.954256:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23696 00000100:00100000:0.0:1787739081.961081:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739081.961142:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652545920 00000100:00100000:1.0:1787739081.961150:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652545920 00000100:00100000:1.0:1787739081.961199:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23697 00000100:00100000:1.0:1787739081.961208:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:18160:x1874579652545920:12345-192.168.203.15@tcp:101:bash.0 00010000:00010000:1.0:1787739081.961226:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739081.961229:0:7944:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1787739081.961252:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab200/0x7ee4453e4a1ae91f lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954ef expref: 14 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.961266:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa074126f6600) 00010000:00010000:1.0:1787739081.961270:0:7944:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1787739081.961282:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae926 lrc: 2/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0954f6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.961398:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1ae92d lrc: 3/0,1 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.961438:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1ae92d lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.961450:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1ae92d lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.961471:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae934 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.961478:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae934 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.961484:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae934 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739081.962025:0:7944:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x9c8:0x0], index 0: 0 00010000:00010000:1.0:1787739081.962151:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840600/0x7ee4453e4a1ae93b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x152c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.962160:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840600/0x7ee4453e4a1ae93b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.962167:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840600/0x7ee4453e4a1ae93b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.962197:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1ae92d lrc: 4/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.962205:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1ae92d lrc: 4/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.962211:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1ae92d lrc: 3/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.962217:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1ae92d lrc: 4/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.962223:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306841800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739081.962227:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1ae92d lrc: 4/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.962234:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1ae92d lrc: 5/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.962242:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa074126f6600) 00010000:00010000:1.0:1787739081.962245:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739081.962248:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1ae92d lrc: 3/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.962253:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1ae92d lrc: 0/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739081.962263:0:7944:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ef31d4dd reply state 0000000074905236 transno 4294983493 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae934 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.962272:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae934 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.962277:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0xdcc87686 (ffffa07433235000) 00010000:00010000:1.0:1787739081.962280:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739081.962282:0:7944:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787739081.962303:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1ae926 lrc: 0/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954f6 expref: 16 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.962315:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840600/0x7ee4453e4a1ae93b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x152c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954f6 expref: 15 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.962322:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152c:0x0].0x0 (ffffa07433235100) 00010000:00010000:1.0:1787739081.962324:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1787739081.962326:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306840600, rc 0) 00000100:00100000:1.0:1787739081.962386:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:18160:x1874579652545920:12345-192.168.203.15@tcp:101:bash.0 Request processed in 1179us (1313us total) trans 4294983493 rc 0/0 00000100:00100000:1.0:1787739081.962395:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23697 00000100:00100000:3.0:1787739081.967629:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739081.967803:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652546048 00000100:00100000:1.0:1787739081.967813:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652546048 00000100:00100000:1.0:1787739081.967880:0:8436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5189 00000100:00100000:1.0:1787739081.967901:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:18160:x1874579652546048:12345-192.168.203.15@tcp:101:bash.0 00010000:00010000:1.0:1787739081.967919:0:8436:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739081.967938:0:8436:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e0be3a3/0x7ee4453e4a1ae949 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0954fd expref: -99 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787739081.967961:0:8436:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x9c8:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:1.0:1787739081.967981:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9c8:0x0].0x0 since=668s v=(0 0) 00010000:00010000:1.0:1787739081.967986:0:8436:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9c8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787739081.967995:0:8436:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e0be3a3/0x7ee4453e4a1ae949 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954fd expref: 7 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.968012:0:8436:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000e0be3a3/0x7ee4453e4a1ae949 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954fd expref: 7 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.968025:0:8436:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306840800, rc 0) 00000100:00100000:1.0:1787739081.968073:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:18160:x1874579652546048:12345-192.168.203.15@tcp:101:bash.0 Request processed in 173us (452us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739081.968080:0:8436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5189 00000100:00080000:3.0:1787739081.969196:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787739081.973765:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739081.974544:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652546304 00000100:00100000:1.0:1787739081.974556:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652546304 00000100:00100000:1.0:1787739081.974613:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5073 00000100:00100000:1.0:1787739081.974622:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:18160:x1874579652546304:12345-192.168.203.15@tcp:35:bash.0 00000100:00100000:1.0:1787739081.974915:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561da40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:18160:x1874579652546304:12345-192.168.203.15@tcp:35:bash.0 Request processed in 293us (1158us total) trans 4294983494 rc 0/0 00000100:00100000:1.0:1787739081.974923:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5073 00000100:00100000:3.0:1787739081.980811:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739081.980894:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652546560 00000100:00100000:0.0:1787739081.980904:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652546560 00000100:00100000:0.0:1787739081.980960:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23698 00000100:00100000:0.0:1787739081.980971:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f761c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:18160:x1874579652546560:12345-192.168.203.15@tcp:101:bash.0 00010000:00010000:0.0:1787739081.980995:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739081.981015:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae950 lrc: 2/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095504 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.981117:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae957 lrc: 3/0,1 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.981131:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae957 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.981141:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae957 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.981153:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae95e lrc: 3/0,1 mode: --/PW res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.981160:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae95e lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.981166:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae95e lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739081.981482:0:6525:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x2c0000401:0x9c8:0x0], index 1: 0 00010000:00010000:0.0:1787739081.981623:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088200/0x7ee4453e4a1ae965 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x152d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.981633:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088200/0x7ee4453e4a1ae965 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.981640:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088200/0x7ee4453e4a1ae965 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.981673:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae957 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.981680:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae957 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.981686:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae957 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.981693:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae957 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.981700:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306089600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739081.981705:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae957 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.981712:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae957 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.981722:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa074126f6600) 00010000:00010000:0.0:1787739081.981725:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739081.981729:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae957 lrc: 3/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.981735:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae957 lrc: 0/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739081.981748:0:6525:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000cc5175f3 reply state 000000004429b1f5 transno 4294983495 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae95e lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.981757:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae95e lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.981765:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0xbc0ca3a9 (ffffa074126f6500) 00010000:00010000:0.0:1787739081.981767:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739081.981769:0:6525:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739081.981796:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae950 lrc: 0/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095504 expref: 20 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.981810:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088200/0x7ee4453e4a1ae965 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x152d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095504 expref: 19 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739081.981817:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152d:0x0].0x0 (ffffa074126f6000) 00010000:00010000:0.0:1787739081.981819:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787739081.981821:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306088200, rc 0) 00000100:00100000:0.0:1787739081.981892:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f761c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+20:18160:x1874579652546560:12345-192.168.203.15@tcp:101:bash.0 Request processed in 914us (1082us total) trans 4294983495 rc 0/0 00000100:00100000:0.0:1787739081.981900:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23698 00000100:00100000:3.0:1787739081.988645:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739081.988716:0:14680:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652546688 00000100:00100000:1.0:1787739081.988725:0:14680:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652546688 00000100:00100000:1.0:1787739081.988787:0:14680:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5190 00000100:00100000:1.0:1787739081.988798:0:14680:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f77480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_011:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:18160:x1874579652546688:12345-192.168.203.15@tcp:101:bash.0 00010000:00010000:1.0:1787739081.988815:0:14680:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739081.988834:0:14680:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004c07339b/0x7ee4453e4a1ae973 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09550b expref: -99 pid: 14680 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787739081.988857:0:14680:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000401:0x9c8:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:1.0:1787739081.988877:0:14680:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x9c8:0x0].0x0 since=668s v=(0 0) 00010000:00010000:1.0:1787739081.988891:0:14680:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x9c8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787739081.988901:0:14680:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004c07339b/0x7ee4453e4a1ae973 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09550b expref: 8 pid: 14680 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.988918:0:14680:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004c07339b/0x7ee4453e4a1ae973 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09550b expref: 8 pid: 14680 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739081.988929:0:14680:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306840c00, rc 0) 00000100:00100000:1.0:1787739081.988974:0:14680:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f77480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_011:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:18160:x1874579652546688:12345-192.168.203.15@tcp:101:bash.0 Request processed in 179us (339us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739081.988980:0:14680:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5190 00000100:00080000:3.0:1787739081.989179:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1787739081.997292:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739081.998598:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652546944 00000100:00100000:0.0:1787739081.998611:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652546944 00000100:00100000:0.0:1787739081.998664:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5074 00000100:00100000:0.0:1787739081.998673:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f77c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+20:18160:x1874579652546944:12345-192.168.203.15@tcp:35:bash.0 00000100:00100000:0.0:1787739081.998961:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f77c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:18160:x1874579652546944:12345-192.168.203.15@tcp:35:bash.0 Request processed in 288us (1678us total) trans 4294983496 rc 0/0 00000100:00100000:0.0:1787739081.998968:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5074 00000100:00100000:1.0:1787739082.020279:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.020610:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652547200 00000100:00100000:0.0:1787739082.020620:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652547200 00000100:00100000:0.0:1787739082.020678:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23699 00000100:00100000:0.0:1787739082.020687:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073056552c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:18160:x1874579652547200:12345-192.168.203.15@tcp:101:bash.0 00010000:00010000:0.0:1787739082.020710:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739082.020730:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae97a lrc: 2/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095512 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.020839:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae981 lrc: 3/0,1 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.020852:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae981 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.020863:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae981 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.020876:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1ae988 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xce:0x0].0x391d5834 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.021045:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1ae988 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.021055:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1ae988 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739082.021649:0:6524:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x9c9:0x0], index 0: 0 00010000:00010000:0.0:1787739082.021792:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae98f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x152e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.021803:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae98f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.021811:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae98f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.021851:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae981 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.021859:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae981 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.021865:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae981 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.021872:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae981 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.021878:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306089a00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739082.023653:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae981 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.023674:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae981 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.023692:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa074126f6600) 00010000:00010000:0.0:1787739082.023695:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739082.023700:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae981 lrc: 3/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.023707:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae981 lrc: 0/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739082.023726:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002daca3cc reply state 0000000089405e0c transno 4294983497 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1ae988 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.023739:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1ae988 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.023747:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x391d5834 (ffffa074126f6e00) 00010000:00010000:0.0:1787739082.023749:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739082.023751:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739082.023799:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae97a lrc: 0/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095512 expref: 24 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.023815:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae98f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x152e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095512 expref: 23 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.023822:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152e:0x0].0x0 (ffffa074126f6300) 00010000:00010000:0.0:1787739082.023824:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787739082.023827:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306089000, rc 0) 00000100:00100000:0.0:1787739082.024449:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073056552c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+24:18160:x1874579652547200:12345-192.168.203.15@tcp:101:bash.0 Request processed in 3758us (4175us total) trans 4294983497 rc 0/0 00000100:00100000:0.0:1787739082.024459:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23699 00000100:00100000:1.0:1787739082.036756:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739082.037048:0:14660:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652547328 00000100:00100000:2.0:1787739082.037057:0:14660:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652547328 00000100:00100000:2.0:1787739082.037111:0:14660:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5191 00000100:00100000:2.0:1787739082.037119:0:14660:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305654780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:18160:x1874579652547328:12345-192.168.203.15@tcp:101:bash.0 00010000:00010000:2.0:1787739082.037137:0:14660:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739082.037157:0:14660:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf6e65f9/0x7ee4453e4a1ae99d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095519 expref: -99 pid: 14660 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787739082.037183:0:14660:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x9c9:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:2.0:1787739082.037204:0:14660:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9c9:0x0].0x0 since=669s v=(0 0) 00010000:00010000:2.0:1787739082.037209:0:14660:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9c9:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1787739082.037219:0:14660:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf6e65f9/0x7ee4453e4a1ae99d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095519 expref: 8 pid: 14660 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.037243:0:14660:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cf6e65f9/0x7ee4453e4a1ae99d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095519 expref: 8 pid: 14660 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.037255:0:14660:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa1c00, rc 0) 00000100:00100000:2.0:1787739082.037377:0:14660:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305654780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:18160:x1874579652547328:12345-192.168.203.15@tcp:101:bash.0 Request processed in 258us (630us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.037384:0:14660:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5191 00000100:00100000:1.0:1787739082.043754:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739082.043816:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652547584 00000100:00100000:3.0:1787739082.043824:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652547584 00000100:00100000:3.0:1787739082.043874:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5075 00000100:00100000:3.0:1787739082.043975:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073056561c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+24:18160:x1874579652547584:12345-192.168.203.15@tcp:35:bash.0 00000100:00100000:3.0:1787739082.044261:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073056561c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+25:18160:x1874579652547584:12345-192.168.203.15@tcp:35:bash.0 Request processed in 379us (515us total) trans 4294983498 rc 0/0 00000100:00100000:3.0:1787739082.044268:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5075 00010000:00010000:1.0:1787739082.050809:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739082.054217:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.054346:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652547840 00000100:00100000:0.0:1787739082.054354:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652547840 00000100:00100000:0.0:1787739082.054490:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23700 00000100:00100000:0.0:1787739082.054501:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f74f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+25:18160:x1874579652547840:12345-192.168.203.15@tcp:101:bash.0 00010000:00010000:0.0:1787739082.054525:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739082.054545:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae9a4 lrc: 2/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095520 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.054644:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae9ab lrc: 3/0,1 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.054658:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae9ab lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.054669:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae9ab lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.054684:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1ae9b2 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xce:0x0].0x18618557 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.054691:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1ae9b2 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.054697:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1ae9b2 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739082.056202:0:6523:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x2c0000401:0x9c9:0x0], index 1: 0 00010000:00010000:0.0:1787739082.057108:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089800/0x7ee4453e4a1ae9b9 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x152f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.057125:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089800/0x7ee4453e4a1ae9b9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.057137:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089800/0x7ee4453e4a1ae9b9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.057174:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae9ab lrc: 4/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.057182:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae9ab lrc: 4/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.057188:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae9ab lrc: 3/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.057195:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae9ab lrc: 4/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.057202:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306089a00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739082.057206:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae9ab lrc: 4/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.057213:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae9ab lrc: 5/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.057224:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa074126f6600) 00010000:00010000:0.0:1787739082.057227:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739082.057231:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae9ab lrc: 3/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.057237:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae9ab lrc: 0/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739082.057253:0:6523:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000184e623 reply state 0000000062f90fd2 transno 4294983499 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1ae9b2 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.057263:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1ae9b2 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.057269:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x18618557 (ffffa074126f6700) 00010000:00010000:0.0:1787739082.057272:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739082.057275:0:6523:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739082.057583:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089600/0x7ee4453e4a1ae9a4 lrc: 0/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095520 expref: 28 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.057599:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089800/0x7ee4453e4a1ae9b9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x152f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095520 expref: 27 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.057611:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152f:0x0].0x0 (ffffa074126f6100) 00010000:00010000:0.0:1787739082.057614:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1787739082.057616:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306089800, rc 0) 00000100:00100000:0.0:1787739082.057749:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f74f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+28:18160:x1874579652547840:12345-192.168.203.15@tcp:101:bash.0 Request processed in 3249us (3540us total) trans 4294983499 rc 0/0 00000100:00100000:0.0:1787739082.057758:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23700 00000100:00100000:1.0:1787739082.071075:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.072023:0:14659:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652547968 00000100:00100000:0.0:1787739082.072035:0:14659:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652547968 00000100:00100000:0.0:1787739082.072107:0:14659:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5192 00000100:00100000:0.0:1787739082.072116:0:14659:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073056570c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:18160:x1874579652547968:12345-192.168.203.15@tcp:101:bash.0 00010000:00010000:0.0:1787739082.072133:0:14659:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739082.072155:0:14659:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002bbfd57a/0x7ee4453e4a1ae9c7 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095527 expref: -99 pid: 14659 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787739082.072179:0:14659:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000401:0x9c9:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:0.0:1787739082.072200:0:14659:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x9c9:0x0].0x0 since=669s v=(0 0) 00010000:00010000:0.0:1787739082.072204:0:14659:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x9c9:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787739082.072213:0:14659:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002bbfd57a/0x7ee4453e4a1ae9c7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095527 expref: 9 pid: 14659 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.072228:0:14659:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002bbfd57a/0x7ee4453e4a1ae9c7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095527 expref: 9 pid: 14659 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.072240:0:14659:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306089600, rc 0) 00000100:00100000:0.0:1787739082.072352:0:14659:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073056570c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:18160:x1874579652547968:12345-192.168.203.15@tcp:101:bash.0 Request processed in 237us (1287us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.072360:0:14659:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5192 00000100:00100000:1.0:1787739082.082163:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.082255:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652548224 00000100:00100000:0.0:1787739082.082265:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652548224 00000100:00100000:0.0:1787739082.082331:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5076 00000100:00100000:0.0:1787739082.082342:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305655a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+28:18160:x1874579652548224:12345-192.168.203.15@tcp:35:bash.0 00000100:00100000:0.0:1787739082.082658:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305655a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+29:18160:x1874579652548224:12345-192.168.203.15@tcp:35:bash.0 Request processed in 317us (503us total) trans 4294983500 rc 0/0 00000100:00100000:0.0:1787739082.082666:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5076 00000100:00100000:2.0:1787739082.098163:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.098672:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652548480 00000100:00100000:0.0:1787739082.098681:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652548480 00000100:00100000:0.0:1787739082.098747:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23701 00000100:00100000:0.0:1787739082.098757:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+29:18160:x1874579652548480:12345-192.168.203.15@tcp:101:bash.0 00010000:00010000:0.0:1787739082.098780:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739082.098801:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae9ce lrc: 2/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09552e expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.099204:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088000/0x7ee4453e4a1ae9d5 lrc: 3/0,1 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.099218:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088000/0x7ee4453e4a1ae9d5 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.099573:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088000/0x7ee4453e4a1ae9d5 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.099689:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae9dc lrc: 3/0,1 mode: --/PW res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.099698:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae9dc lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.099705:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae9dc lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739082.100209:0:13793:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x280000401:0x9ca:0x0], index 0: 0 00010000:00010000:2.0:1787739082.100704:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1400/0x7ee4453e4a1ae9e3 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1530:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.100715:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1400/0x7ee4453e4a1ae9e3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1530:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.100722:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1400/0x7ee4453e4a1ae9e3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1530:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.100756:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088000/0x7ee4453e4a1ae9d5 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.100765:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088000/0x7ee4453e4a1ae9d5 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.100771:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088000/0x7ee4453e4a1ae9d5 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.100777:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088000/0x7ee4453e4a1ae9d5 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.100783:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306088000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739082.100787:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088000/0x7ee4453e4a1ae9d5 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.100794:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088000/0x7ee4453e4a1ae9d5 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.100806:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa074126f6600) 00010000:00010000:2.0:1787739082.100809:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739082.100814:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088000/0x7ee4453e4a1ae9d5 lrc: 3/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.100819:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088000/0x7ee4453e4a1ae9d5 lrc: 0/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739082.100834:0:13793:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000041780257 reply state 00000000f870baf1 transno 4294983501 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae9dc lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.100845:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae9dc lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.100851:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0xb62e0cbf (ffffa07403eeb400) 00010000:00010000:2.0:1787739082.100854:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739082.100857:0:13793:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787739082.100946:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1ae9ce lrc: 0/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09552e expref: 32 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.100962:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1400/0x7ee4453e4a1ae9e3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1530:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09552e expref: 31 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.100968:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1530:0x0].0x0 (ffffa07403eeb600) 00010000:00010000:2.0:1787739082.100970:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787739082.100972:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa1400, rc 0) 00000100:00100000:2.0:1787739082.101183:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+32:18160:x1874579652548480:12345-192.168.203.15@tcp:101:bash.0 Request processed in 2426us (3028us total) trans 4294983501 rc 0/0 00000100:00100000:2.0:1787739082.101190:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23701 00000100:00100000:1.0:1787739082.110543:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739082.111357:0:14683:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652548608 00000100:00100000:3.0:1787739082.111367:0:14683:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652548608 00000100:00100000:1.0:1787739082.111582:0:14683:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5193 00000100:00100000:1.0:1787739082.111595:0:14683:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305657840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:18160:x1874579652548608:12345-192.168.203.15@tcp:101:bash.0 00010000:00010000:1.0:1787739082.111613:0:14683:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739082.111635:0:14683:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e81f861/0x7ee4453e4a1ae9f1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095535 expref: -99 pid: 14683 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787739082.111661:0:14683:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x9ca:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:1.0:1787739082.111682:0:14683:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9ca:0x0].0x0 since=669s v=(0 0) 00010000:00010000:1.0:1787739082.111687:0:14683:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9ca:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787739082.111696:0:14683:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e81f861/0x7ee4453e4a1ae9f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095535 expref: 9 pid: 14683 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.111713:0:14683:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e81f861/0x7ee4453e4a1ae9f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095535 expref: 9 pid: 14683 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.111979:0:14683:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841400, rc 0) 00000100:00100000:1.0:1787739082.112048:0:14683:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305657840 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:18160:x1874579652548608:12345-192.168.203.15@tcp:101:bash.0 Request processed in 456us (1520us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.112056:0:14683:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5193 00000100:00100000:1.0:1787739082.118066:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739082.118309:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652548864 00000100:00100000:3.0:1787739082.118319:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652548864 00000100:00100000:3.0:1787739082.118381:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5077 00000100:00100000:3.0:1787739082.118393:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305655680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+32:18160:x1874579652548864:12345-192.168.203.15@tcp:35:bash.0 00000100:00100000:3.0:1787739082.118739:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305655680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+33:18160:x1874579652548864:12345-192.168.203.15@tcp:35:bash.0 Request processed in 349us (682us total) trans 4294983502 rc 0/0 00000100:00100000:3.0:1787739082.118747:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5077 00000100:00100000:2.0:1787739082.127147:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.127244:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652549120 00000100:00100000:0.0:1787739082.127253:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652549120 00000100:00100000:0.0:1787739082.127309:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23702 00000100:00100000:0.0:1787739082.127320:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+33:18160:x1874579652549120:12345-192.168.203.15@tcp:101:bash.0 00010000:00010000:0.0:1787739082.127351:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739082.127373:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088400/0x7ee4453e4a1ae9f8 lrc: 2/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09553c expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.127793:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae9ff lrc: 3/0,1 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.127810:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae9ff lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.127821:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae9ff lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.127836:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aea06 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xce:0x0].0x84832a57 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.127844:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aea06 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.127850:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aea06 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739082.128166:0:6524:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x2c0000401:0x9ca:0x0], index 1: 0 00010000:00010000:2.0:1787739082.128300:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0800/0x7ee4453e4a1aea0d lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1531:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.128308:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0800/0x7ee4453e4a1aea0d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1531:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.128317:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0800/0x7ee4453e4a1aea0d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1531:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.128349:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae9ff lrc: 4/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.128355:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae9ff lrc: 4/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.128361:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae9ff lrc: 3/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.128367:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae9ff lrc: 4/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.128372:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739082.128377:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae9ff lrc: 4/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.128385:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae9ff lrc: 5/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.128397:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa074126f6600) 00010000:00010000:2.0:1787739082.128400:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739082.128405:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae9ff lrc: 3/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.128437:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1ae9ff lrc: 0/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739082.128454:0:6524:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001d4d9556 reply state 00000000bb248a6e transno 4294983503 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aea06 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.128463:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aea06 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.128469:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x84832a57 (ffffa07403eeb800) 00010000:00010000:2.0:1787739082.128471:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739082.128473:0:6524:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787739082.128500:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088400/0x7ee4453e4a1ae9f8 lrc: 0/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09553c expref: 36 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.128514:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0800/0x7ee4453e4a1aea0d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1531:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09553c expref: 35 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.128520:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1531:0x0].0x0 (ffffa07403eeb100) 00010000:00010000:2.0:1787739082.128522:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1787739082.128524:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa0800, rc 0) 00000100:00100000:2.0:1787739082.128591:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+36:18160:x1874579652549120:12345-192.168.203.15@tcp:101:bash.0 Request processed in 1272us (1452us total) trans 4294983503 rc 0/0 00000100:00100000:2.0:1787739082.128599:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23702 00000100:00100000:1.0:1787739082.138941:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.139645:0:8439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652549248 00000100:00100000:0.0:1787739082.139656:0:8439:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652549248 00000100:00100000:0.0:1787739082.139719:0:8439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5194 00000100:00100000:0.0:1787739082.139728:0:8439:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073056543c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:18160:x1874579652549248:12345-192.168.203.15@tcp:101:bash.0 00010000:00010000:0.0:1787739082.139746:0:8439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739082.139777:0:8439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e6a9f713/0x7ee4453e4a1aea1b lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095543 expref: -99 pid: 8439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787739082.139803:0:8439:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x2c0000401:0x9ca:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:0.0:1787739082.139826:0:8439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x9ca:0x0].0x0 since=669s v=(0 0) 00010000:00010000:0.0:1787739082.139831:0:8439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x9ca:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787739082.139841:0:8439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e6a9f713/0x7ee4453e4a1aea1b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095543 expref: 10 pid: 8439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.139857:0:8439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e6a9f713/0x7ee4453e4a1aea1b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095543 expref: 10 pid: 8439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.139870:0:8439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306088400, rc 0) 00000100:00100000:0.0:1787739082.140044:0:8439:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073056543c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:18160:x1874579652549248:12345-192.168.203.15@tcp:101:bash.0 Request processed in 317us (1111us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.140055:0:8439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5194 00000100:00100000:1.0:1787739082.147532:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739082.147694:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652549504 00000100:00100000:2.0:1787739082.147704:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652549504 00000100:00100000:2.0:1787739082.147758:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5078 00000100:00100000:2.0:1787739082.147768:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074332a9850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+36:18160:x1874579652549504:12345-192.168.203.15@tcp:35:bash.0 00000100:00100000:2.0:1787739082.148066:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074332a9850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+37:18160:x1874579652549504:12345-192.168.203.15@tcp:35:bash.0 Request processed in 299us (542us total) trans 4294983504 rc 0/0 00000100:00100000:2.0:1787739082.148074:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5078 00000100:00100000:1.0:1787739082.178766:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3650:1874579665785600:0@lo:400:kworker.0 00000100:00100000:1.0:1787739082.178844:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.178920:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3650:1874579665786112:0@lo:400:kworker.0 00000100:00100000:1.0:1787739082.178945:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.178962:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3650:1874579665786624:0@lo:400:kworker.0 00000100:00100000:1.0:1787739082.178981:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.179005:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:1.0:1787739082.179011:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665785728:0@lo:400:kworker.0 00000100:00100000:1.0:1787739082.179043:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.179059:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a30780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3650:1874579665786240:0@lo:400:kworker.0 00000100:00100000:1.0:1787739082.179078:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.179105:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3650:1874579665786752:0@lo:400:kworker.0 00000100:00100000:1.0:1787739082.179126:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.179155:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665785600 00000100:00100000:1.0:1787739082.179161:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665785600 00000100:00100000:1.0:1787739082.179197:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665786112 00000100:00100000:1.0:1787739082.179200:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665786112 00000100:00100000:1.0:1787739082.179231:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665786624 00000100:00100000:1.0:1787739082.179234:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665786624 00000100:00100000:1.0:1787739082.179249:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665786240 00000100:00100000:1.0:1787739082.179252:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665786240 00000100:00100000:1.0:1787739082.179261:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665786752 00000100:00100000:1.0:1787739082.179263:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665786752 00000100:00100000:1.0:1787739082.179277:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 682 00000100:00100000:1.0:1787739082.179284:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073056561c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1874579665785600:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787739082.179285:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2:3653:1874579665785472:0@lo:400:kworker.0 00000100:00100000:3.0:1787739082.179319:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.179335:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073056561c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3650:x1874579665785600:12345-0@lo:400:kworker.0 Request processed in 51us (491us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.179342:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 682 00000100:00100000:1.0:1787739082.179357:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 683 00000100:00100000:1.0:1787739082.179363:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305655a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1874579665786112:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787739082.179364:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a33480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665785984:0@lo:400:kworker.0 00000100:00100000:3.0:1787739082.179385:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.179388:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305655a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3650:x1874579665786112:12345-0@lo:400:kworker.0 Request processed in 26us (442us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.179393:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 683 00000100:00100000:3.0:1787739082.179402:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a30f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665786496:0@lo:400:kworker.0 00000100:00100000:1.0:1787739082.179404:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 684 00000100:00100000:1.0:1787739082.179429:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305655680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3650:x1874579665786624:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787739082.179441:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739082.179457:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665787008:0@lo:400:kworker.0 00000100:00100000:1.0:1787739082.179457:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305655680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3650:x1874579665786624:12345-0@lo:400:kworker.0 Request processed in 48us (474us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.179468:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 684 00000100:00100000:3.0:1787739082.179477:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.179480:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 685 00000100:00100000:1.0:1787739082.179486:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305654b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3650:x1874579665786240:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787739082.179496:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:3.0:1787739082.179502:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a321c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3653:1874579665785856:0@lo:400:kworker.0 00000100:00100000:1.0:1787739082.179512:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305654b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3650:x1874579665786240:12345-0@lo:400:kworker.0 Request processed in 26us (432us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.179517:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 685 00000100:00100000:3.0:1787739082.179520:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.179526:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665785856 00000100:00100000:1.0:1787739082.179529:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665785856 00000100:00100000:3.0:1787739082.179532:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a31a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665786368:0@lo:400:kworker.0 00000100:00100000:1.0:1787739082.179542:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 686 00000100:00100000:1.0:1787739082.179546:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305657c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1874579665786752:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787739082.179550:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739082.179568:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665786880:0@lo:400:kworker.0 00000100:00100000:1.0:1787739082.179569:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305657c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3650:x1874579665786752:12345-0@lo:400:kworker.0 Request processed in 23us (442us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.179574:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 686 00000100:00100000:3.0:1787739082.179585:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.179585:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 687 00000100:00100000:1.0:1787739082.179590:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c383c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3653:x1874579665785856:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787739082.179610:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665785728 00000100:00100000:3.0:1787739082.179613:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665785728 00000100:00100000:1.0:1787739082.179618:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c383c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3653:x1874579665785856:12345-0@lo:400:kworker.0 Request processed in 28us (97us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.179623:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 687 00000100:00100000:3.0:1787739082.179635:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665785984 00000100:00100000:3.0:1787739082.179638:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665785984 00000100:00080000:1.0:1787739082.179638:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1787739082.179641:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1787739082.179651:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3633 00000100:00100000:3.0:1787739082.179656:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305656940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:3650:x1874579665785728:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787739082.179666:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a30780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3650:1874579665786240:0@lo:400:kworker.0 00000100:00100000:1.0:1787739082.179676:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3650:1874579665786752:0@lo:400:kworker.0 00000100:00100000:1.0:1787739082.179685:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3650:1874579665785600:0@lo:400:kworker.0 00000100:00100000:3.0:1787739082.179687:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305656940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:3650:x1874579665785728:12345-0@lo:400:kworker.0 Request processed in 31us (650us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.179692:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3633 00000100:00100000:1.0:1787739082.179693:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3650:1874579665786112:0@lo:400:kworker.0 00000100:00100000:1.0:1787739082.179701:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce2940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3650:1874579665786624:0@lo:400:kworker.0 00000100:00100000:3.0:1787739082.179704:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3634 00000100:00100000:3.0:1787739082.179709:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f743c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+24:3653:x1874579665785984:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1787739082.179725:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3650:1874579665785728:0@lo:400:kworker.0 00000100:00100000:3.0:1787739082.179734:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f743c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+24:3653:x1874579665785984:12345-0@lo:400:kworker.0 Request processed in 25us (349us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.179740:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3634 00000100:00100000:3.0:1787739082.179795:0:6518:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665785472 00000100:00100000:3.0:1787739082.179798:0:6518:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665785472 00000100:00100000:3.0:1787739082.179822:0:6518:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 456 00000100:00100000:3.0:1787739082.179829:0:6518:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0740ea11c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2+24:3653:x1874579665785472:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787739082.179859:0:6518:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0740ea11c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2+24:3653:x1874579665785472:12345-0@lo:400:kworker.0 Request processed in 31us (539us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.179864:0:6518:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 456 00000100:00100000:3.0:1787739082.179903:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665786496 00000100:00100000:3.0:1787739082.179907:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665786496 00000100:00100000:3.0:1787739082.179932:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665787008 00000100:00100000:3.0:1787739082.179934:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665787008 00000100:00100000:3.0:1787739082.179948:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665786368 00000100:00100000:3.0:1787739082.179951:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665786368 00000100:00100000:3.0:1787739082.179965:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665786880 00000100:00100000:3.0:1787739082.179968:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665786880 00000100:00100000:3.0:1787739082.179982:0:8436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 369 00000100:00100000:3.0:1787739082.179987:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f77c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3653:x1874579665786496:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787739082.180019:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f77c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3653:x1874579665786496:12345-0@lo:400:kworker.0 Request processed in 32us (578us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.180025:0:8436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 369 00000100:00100000:3.0:1787739082.180037:0:8436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 370 00000100:00100000:3.0:1787739082.180043:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c380b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3653:x1874579665787008:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787739082.180071:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c380b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3653:x1874579665787008:12345-0@lo:400:kworker.0 Request processed in 28us (593us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.180077:0:8436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 370 00000100:00100000:3.0:1787739082.180088:0:8436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 371 00000100:00100000:3.0:1787739082.180094:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c3830c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3653:x1874579665786368:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1787739082.180120:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c3830c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3653:x1874579665786368:12345-0@lo:400:kworker.0 Request processed in 27us (569us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.180123:0:8885:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 372 00000100:00100000:3.0:1787739082.180127:0:8436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 371 00000100:00100000:0.0:1787739082.180131:0:8885:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c381a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3653:x1874579665786880:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1787739082.180162:0:8885:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c381a40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3653:x1874579665786880:12345-0@lo:400:kworker.0 Request processed in 33us (576us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.180168:0:8885:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 372 00000100:00100000:3.0:1787739082.180171:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a321c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3653:1874579665785856:0@lo:400:kworker.0 00000100:00100000:3.0:1787739082.180182:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a31a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665786368:0@lo:400:kworker.0 00000100:00100000:3.0:1787739082.180191:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce30c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665786880:0@lo:400:kworker.0 00000100:00100000:3.0:1787739082.180198:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:571cde1e-8b5a-497d-8da6-8ec9d7eaf5d2:3653:1874579665785472:0@lo:400:kworker.0 00000100:00100000:3.0:1787739082.180210:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a33480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665785984:0@lo:400:kworker.0 00000100:00100000:3.0:1787739082.180218:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a30f00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665786496:0@lo:400:kworker.0 00000100:00100000:3.0:1787739082.180227:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce03c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665787008:0@lo:400:kworker.0 00000100:00100000:2.0:1787739082.180449:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665787264:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1787739082.180481:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.180502:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787739082.180508:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665787136:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1787739082.180534:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.181173:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665787264 00000100:00100000:0.0:1787739082.181180:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665787264 00000100:00100000:0.0:1787739082.181232:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665787136 00000100:00100000:0.0:1787739082.181235:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665787136 00000100:00100000:0.0:1787739082.181254:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 372 00000100:00100000:0.0:1787739082.181261:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665787264:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1787739082.181283:0:8441:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 25431488 pending 0 free 3923496960 avail 3691974656 00000100:00100000:0.0:1787739082.181329:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3651:x1874579665787264:12345-0@lo:13:osp-pre-1-1.0 Request processed in 68us (847us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.181336:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 372 00000100:00100000:0.0:1787739082.181351:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 373 00000100:00100000:0.0:1787739082.181358:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665787136:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1787739082.181367:0:8441:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 25431488 pending 0 free 3923496960 avail 3691974656 00000100:00100000:0.0:1787739082.181390:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3651:x1874579665787136:12345-0@lo:13:osp-pre-1-0.0 Request processed in 34us (856us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.181396:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 373 00000100:00100000:0.0:1787739082.181461:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383f0c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665787136:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1787739082.181472:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce1a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665787264:0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1787739082.181997:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1787739082.182003:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1787739082.186549:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739082.186697:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652549760 00000100:00100000:1.0:1787739082.186706:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652549760 00000100:00100000:1.0:1787739082.186771:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23703 00000100:00100000:1.0:1787739082.186782:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+37:18316:x1874579652549760:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739082.186808:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739082.186829:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841200/0x7ee4453e4a1aea22 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09554a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739082.187234:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x152b:0x0]: returning size 4096 00010000:00010000:1.0:1787739082.187332:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841200/0x7ee4453e4a1aea22 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09554a expref: 39 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.187347:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa07401795300) 00010000:00010000:1.0:1787739082.187351:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739082.187356:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841200/0x7ee4453e4a1aea22 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09554a expref: 39 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.187369:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841200, rc 0) 00000100:00100000:1.0:1787739082.187457:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+38:18316:x1874579652549760:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 677us (927us total) trans 4294983505 rc 301/301 00000100:00100000:1.0:1787739082.187465:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23703 00000100:00100000:1.0:1787739082.195309:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.195384:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652549888 00000100:00100000:0.0:1787739082.195394:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652549888 00000100:00100000:0.0:1787739082.195527:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5079 00000100:00100000:0.0:1787739082.195537:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305656d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+38:18316:x1874579652549888:12345-192.168.203.15@tcp:33:lfs.0 00000004:00200000:0.0:1787739082.195663:0:6526:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x152b:0x0]: returning size 4096 00000004:00200000:0.0:1787739082.195668:0:6526:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 2 MDT count 2 00000100:00100000:0.0:1787739082.195751:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305656d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+38:18316:x1874579652549888:12345-192.168.203.15@tcp:33:lfs.0 Request processed in 215us (451us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.195767:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5079 00000100:00100000:1.0:1787739082.203868:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739082.207736:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652550144 00000100:00100000:3.0:1787739082.207750:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652550144 00000100:00100000:3.0:1787739082.207873:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5080 00000100:00100000:3.0:1787739082.207899:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+38:18316:x1874579652550144:12345-192.168.203.15@tcp:35:lfs.0 00000100:00100000:3.0:1787739082.208804:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+39:18316:x1874579652550144:12345-192.168.203.15@tcp:35:lfs.0 Request processed in 904us (4942us total) trans 4294983506 rc 0/0 00000100:00100000:3.0:1787739082.208814:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5080 00000100:00100000:1.0:1787739082.256507:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.256808:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652550400 00000100:00100000:0.0:1787739082.256818:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652550400 00000100:00100000:0.0:1787739082.256893:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23704 00000100:00100000:0.0:1787739082.256904:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+39:18317:x1874579652550400:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739082.256926:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739082.256946:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1aea30 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095551 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.257096:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088000/0x7ee4453e4a1aea37 lrc: 3/1,0 mode: --/CR res: [0x200000402:0x152b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.257111:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088000/0x7ee4453e4a1aea37 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x152b:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.257121:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088000/0x7ee4453e4a1aea37 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x152b:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739082.257132:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x152b:0x0]: returning size 4096 00010000:00010000:0.0:1787739082.257224:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1aea30 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095551 expref: 42 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.257248:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088000/0x7ee4453e4a1aea37 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x152b:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095551 expref: 41 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.257257:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152b:0x0].0x0 (ffffa074126f6200) 00010000:00010000:0.0:1787739082.257260:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:0.0:1787739082.257263:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306088000, rc 0) 00000100:00100000:0.0:1787739082.257323:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+41:18317:x1874579652550400:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 421us (831us total) trans 4294983507 rc 0/0 00000100:00100000:0.0:1787739082.257331:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23704 00000100:00100000:1.0:1787739082.266767:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739082.266922:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652550528 00000100:00100000:2.0:1787739082.266931:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652550528 00000100:00100000:2.0:1787739082.267001:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1424 00000100:00100000:2.0:1787739082.267021:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+41:2409:x1874579652550528:12345-192.168.203.15@tcp:103:lfs.0 00010000:00010000:2.0:1787739082.267038:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739082.267063:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1ae8f5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954e1 expref: 41 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.267084:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa07401795300) 00010000:00010000:2.0:1787739082.267089:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739082.267150:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32d00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+40:2409:x1874579652550528:12345-192.168.203.15@tcp:103:lfs.0 Request processed in 140us (392us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.267158:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1424 00000100:00100000:1.0:1787739082.270031:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.270123:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652550784 00000100:00100000:0.0:1787739082.270132:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652550784 00000100:00100000:0.0:1787739082.270190:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5081 00000100:00100000:0.0:1787739082.270200:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a31a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+40:18317:x1874579652550784:12345-192.168.203.15@tcp:35:lfs.0 00000100:00100000:0.0:1787739082.270454:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a31a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+41:18317:x1874579652550784:12345-192.168.203.15@tcp:35:lfs.0 Request processed in 255us (431us total) trans 4294983508 rc 0/0 00000100:00100000:0.0:1787739082.270462:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5081 00000100:00100000:3.0:1787739082.274982:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739082.275032:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739082.275201:0:6536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652551168 00000100:00100000:1.0:1787739082.275211:0:6536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652551168 00000100:00100000:1.0:1787739082.275294:0:6536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 0 00000100:00100000:1.0:1787739082.275303:0:6536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:18317:x1874579652551168:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:1.0:1787739082.275403:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306840200/0x7ee4453e4a1aea45 lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.275444:0:6536:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306840200/0x7ee4453e4a1aea45 lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.275455:0:6536:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306840200/0x7ee4453e4a1aea45 lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739082.275478:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a321c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665787392:0@lo:101:mdt_io00_000.0 00000100:00100000:1.0:1787739082.275523:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.275587:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.275604:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665787392 00000100:00100000:1.0:1787739082.275608:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665787392 00000100:00100000:1.0:1787739082.275637:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3635 00000100:00100000:1.0:1787739082.275643:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:6536:x1874579665787392:12345-0@lo:101:mdt_io00_000.0 00010000:00010000:1.0:1787739082.275652:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739082.275667:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1aea4c lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aea45 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.275691:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1aea4c lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aea45 expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.275704:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1aea4c lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aea45 expref: 9 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.275712:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffffa07433235700) 00010000:00010000:1.0:1787739082.275714:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.275718:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306840e00, rc 0) 00000100:00100000:1.0:1787739082.275770:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6536:x1874579665787392:12345-0@lo:101:mdt_io00_000.0 Request processed in 127us (246us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.275776:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3635 00000100:00100000:1.0:1787739082.275815:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a321c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665787392:0@lo:101:mdt_io00_000.0 00010000:00010000:1.0:1787739082.275829:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306840200/0x7ee4453e4a1aea45 lrc: 4/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aea4c expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.275845:0:6536:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306840200/0x7ee4453e4a1aea45 lrc: 4/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aea4c expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739082.275863:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000004:00080000:1.0:1787739082.275878:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:1.0:1787739082.275918:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a321c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665787520:0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787739082.275944:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.275958:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803540 going to sleep for 11 seconds 00000100:00100000:0.0:1787739082.276081:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652550912 00000100:00100000:0.0:1787739082.276086:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652550912 00000100:00100000:0.0:1787739082.276130:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1425 00000100:00100000:0.0:1787739082.276139:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba1850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+41:2408:x1874579652550912:12345-192.168.203.15@tcp:103:lfs.0 00010000:00010000:0.0:1787739082.276159:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1787739082.276178:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088000/0x7ee4453e4a1aea37 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x152b:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095551 expref: 41 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.276196:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1ae911 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x152b:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954e8 expref: 40 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.276203:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152b:0x0].0x0 (ffffa074126f6200) 00010000:00010000:0.0:1787739082.276207:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739082.276244:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba1850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+39:2408:x1874579652550912:12345-192.168.203.15@tcp:103:lfs.0 Request processed in 107us (1271us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.276250:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1425 00000100:00080000:2.0:1787739082.276644:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1787739082.277138:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665787520 00000100:00100000:3.0:1787739082.277146:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665787520 00000100:00100000:3.0:1787739082.277189:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3636 00000100:00100000:3.0:1787739082.277196:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6536:x1874579665787520:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.277300:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:6536:x1874579665787520:12345-0@lo:1000:mdt_io00_000.0 Request processed in 104us (1354us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.277308:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3636 00000100:00100000:2.0:1787739082.277383:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a321c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665787520:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.277476:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561d2c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665787648:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.277509:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.277553:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306107480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.278538:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665787648 00000100:00100000:3.0:1787739082.278546:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665787648 00000100:00100000:3.0:1787739082.278588:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3637 00000100:00100000:3.0:1787739082.278595:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6536:x1874579665787648:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.278680:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6536:x1874579665787648:12345-0@lo:1000:mdt_io00_000.0 Request processed in 86us (1170us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.278687:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3637 00000100:00100000:0.0:1787739082.278757:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561d2c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665787648:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.278817:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce1a40 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665787776:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.278844:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.278872:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.279444:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665787776 00000100:00100000:3.0:1787739082.279451:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665787776 00000100:00100000:3.0:1787739082.279478:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3638 00000100:00100000:3.0:1787739082.279485:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6536:x1874579665787776:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.279588:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6536:x1874579665787776:12345-0@lo:1000:mdt_io00_000.0 Request processed in 103us (742us total) trans 0 rc 1/1 00000100:00100000:3.0:1787739082.279595:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3638 00000100:00100000:0.0:1787739082.279953:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce1a40 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665787776:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.280027:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce1a40 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665787904:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.280058:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.280090:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.280139:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665787904 00000100:00100000:3.0:1787739082.280143:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665787904 00000100:00100000:3.0:1787739082.280176:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3639 00000100:00100000:3.0:1787739082.280182:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6536:x1874579665787904:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.280251:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+9:6536:x1874579665787904:12345-0@lo:1000:mdt_io00_000.0 Request processed in 69us (191us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.280258:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3639 00000100:00100000:0.0:1787739082.280407:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce1a40 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665787904:0@lo:1000:mdt_io00_000.0 00010000:00010000:0.0:1787739082.280486:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089a00/0x7ee4453e4a1aea53 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.280496:0:6536:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089a00/0x7ee4453e4a1aea53 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.280505:0:6536:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089a00/0x7ee4453e4a1aea53 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x1000000000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.280517:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce1a40 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665788032:0@lo:101:mdt_io00_000.0 00000100:00100000:0.0:1787739082.280553:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.280569:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3240 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.281056:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665788032 00000100:00100000:1.0:1787739082.281062:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665788032 00000100:00100000:1.0:1787739082.281100:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3640 00000100:00100000:1.0:1787739082.281108:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:6536:x1874579665788032:12345-0@lo:101:mdt_io00_000.0 00010000:00010000:1.0:1787739082.281120:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739082.281135:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841a00/0x7ee4453e4a1aea5a lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aea53 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.281158:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841a00/0x7ee4453e4a1aea5a lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aea53 expref: 10 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.281173:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841a00/0x7ee4453e4a1aea5a lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aea53 expref: 10 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.281181:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa07401795300) 00010000:00010000:1.0:1787739082.281183:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.281187:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841a00, rc 0) 00000100:00100000:1.0:1787739082.281238:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:6536:x1874579665788032:12345-0@lo:101:mdt_io00_000.0 Request processed in 130us (684us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.281247:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3640 00000100:00100000:0.0:1787739082.281458:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce1a40 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665788032:0@lo:101:mdt_io00_000.0 00010000:00010000:0.0:1787739082.281473:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089a00/0x7ee4453e4a1aea53 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aea5a expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.281483:0:6536:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089a00/0x7ee4453e4a1aea53 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aea5a expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739082.281499:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787739082.281537:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1aea61 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.281545:0:6536:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1aea61 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.281552:0:6536:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1aea61 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.281566:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce1a40 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665788160:0@lo:101:mdt_io00_000.0 00000100:00100000:0.0:1787739082.281592:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.281705:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.282689:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665788160 00000100:00100000:2.0:1787739082.282697:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665788160 00000100:00100000:2.0:1787739082.282872:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3641 00000100:00100000:2.0:1787739082.283008:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:6536:x1874579665788160:12345-0@lo:101:mdt_io00_000.0 00010000:00010000:2.0:1787739082.283026:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739082.283046:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aea68 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aea61 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.283068:0:6528:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae8cb lrc: 2/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.283087:0:6528:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aea68 lrc: 4/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aea61 expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.283105:0:6528:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae8cb lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00080000:00080000:2.0:1787739082.283133:0:6528:0:(osd_handler.c:2849:osd_commit_async()) lustre-MDT0000: async commit OSD 00010000:00010000:2.0:1787739082.283214:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aea68 lrc: 3/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aea61 expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.283232:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xf5d6a2c1 (ffffa074126f6800) 00010000:00010000:2.0:1787739082.283235:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739082.283239:0:6528:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aea68 lrc: 3/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aea61 expref: 11 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.283246:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa0200, rc 0) 00000100:00100000:2.0:1787739082.283307:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:6536:x1874579665788160:12345-0@lo:101:mdt_io00_000.0 Request processed in 427us (1713us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.283314:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3641 00000100:00100000:0.0:1787739082.283539:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce1a40 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665788160:0@lo:101:mdt_io00_000.0 00010000:00010000:0.0:1787739082.283557:0:6536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1aea61 lrc: 4/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aea68 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.283565:0:6536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1aea61 lrc: 4/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aea68 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.287233:0:6502:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089200/0x7ee4453e4a1ae8e0 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1ae8e7 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739082.287254:0:6502:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089200/0x7ee4453e4a1ae8e0 lrc: 3/0,0 mode: PW/PW res: [0x240000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1ae8e7 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.287297:0:6502:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089200/0x7ee4453e4a1ae8e0 lrc: 2/0,0 mode: --/PW res: [0x240000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae8e7 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.287304:0:6502:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739082.287343:0:6502:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089200/0x7ee4453e4a1ae8e0 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1ae8e7 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1787739082.287363:0:6502:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294983508 is committed 00000100:00100000:3.0:1787739082.287860:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1787739082.287896:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665788288:0@lo:103:jbd2/dm-0-8.0 00000100:00100000:3.0:1787739082.287965:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.288203:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665788288 00000100:00100000:1.0:1787739082.288211:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665788288 00000100:00100000:1.0:1787739082.288274:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1426 00000100:00100000:1.0:1787739082.288283:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383fc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+24:3653:x1874579665788288:12345-0@lo:103:jbd2/dm-0-8.0 00010000:00010000:1.0:1787739082.288298:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739082.288320:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1ae8e7 lrc: 0/0,0 mode: --/PW res: [0x240000401:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1ae8e0 expref: 24 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.288571:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000401:0xce:0x0].0x0 (ffffa074331c2300) 00010000:00010000:1.0:1787739082.288582:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739082.288631:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383fc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+23:3653:x1874579665788288:12345-0@lo:103:jbd2/dm-0-8.0 Request processed in 349us (666us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.288638:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1426 00000100:00100000:3.0:1787739082.288964:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665788288:0@lo:103:jbd2/dm-0-8.0 00010000:00010000:0.0:1787739082.289042:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae8cb lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.289052:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae8cb lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.289060:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae8cb lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.289067:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306088800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739082.289072:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae8cb lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.289080:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae8cb lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000020 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.289091:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xf5d6a2c1 (ffffa074126f6800) 00010000:00010000:0.0:1787739082.289094:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739082.289099:0:3637:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aea68 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aea61 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.289109:0:3637:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aea68 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1aea61 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.289118:0:3637:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aea68 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 0@lo remote: 0x7ee4453e4a1aea61 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.289153:0:3637:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aea68 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aea61 expref: 10 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.289165:0:3637:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpc_hr00_001:lustre-MDT0000_UUID:3637:1874579665788416:0@lo:105: 00000100:00100000:0.0:1787739082.289201:0:3637:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.289336:0:3637:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3240 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.289501:0:15907:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665788416 00000100:00100000:2.0:1787739082.289508:0:15907:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665788416 00000100:00100000:2.0:1787739082.289632:0:15907:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1215 00000100:00100000:2.0:1787739082.289641:0:15907:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f6580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:3637:x1874579665788416:12345-0@lo:105: 00010000:00010000:2.0:1787739082.289660:0:15907:0:(ldlm_lockd.c:2518:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1aea61 lrc: 5/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aea68 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.289670:0:15907:0:(ldlm_lockd.c:1968:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1aea61 lrc: 5/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aea68 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.289677:0:15907:0:(ldlm_lockd.c:2045:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1aea61 lrc: 5/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aea68 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.289684:0:15907:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1aea61 lrc: 5/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1aea68 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.289692:0:15907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1aea61 lrc: 6/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11080000000000 nid: local remote: 0x7ee4453e4a1aea68 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.289700:0:15907:0:(ldlm_lockd.c:2073:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1aea61 lrc: 6/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1080000000000 nid: local remote: 0x7ee4453e4a1aea68 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.289717:0:15907:0:(ldlm_lockd.c:2084:ldlm_handle_cp_callback()) ### client completion callback handler END (lock ffffa07306088600) 00000100:00100000:2.0:1787739082.289773:0:15907:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f6580 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:3637:x1874579665788416:12345-0@lo:105: Request processed in 134us (572us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.289780:0:15907:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1215 00010000:00010000:0.0:1787739082.290080:0:6536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1aea61 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aea68 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.290091:0:6536:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1aea61 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aea68 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739082.290116:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787739082.290163:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1aea6f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x152b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.290171:0:6536:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1aea6f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x152b:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.290180:0:6536:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1aea6f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x152b:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.290194:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f4780 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665788544:0@lo:101:mdt_io00_000.0 00000100:00100000:1.0:1787739082.290312:0:3637:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f70c0 pname:cluuid:pid:xid:nid:opc:job ptlrpc_hr00_001:lustre-MDT0000_UUID:3637:1874579665788416:0@lo:105: 00010000:00010000:1.0:1787739082.290820:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae8cb lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.290829:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1ae8cb lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.290845:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1ae8d2 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x152b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.290852:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1ae8d2 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x152b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.290858:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1ae8d2 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x152b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.290863:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306089c00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739082.290867:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1ae8d2 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x152b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.290875:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1ae8d2 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x152b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.290985:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.291064:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3900 going to sleep for 11 seconds 00010000:00010000:1.0:1787739082.291117:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152b:0x0].0x0 (ffffa074126f6200) 00010000:00010000:1.0:1787739082.291121:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.291125:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1ae8d2 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x152b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291131:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1ae8d2 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x152b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291146:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1ae8d9 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291153:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1ae8d9 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291159:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1ae8d9 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291164:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306088c00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739082.291166:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1ae8d9 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291171:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1ae8d9 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291179:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa074126f6600) 00010000:00010000:1.0:1787739082.291181:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.291184:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1ae8d9 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291189:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1ae8d9 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291207:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae934 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291214:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae934 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291220:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae934 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291293:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840a00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739082.291297:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae934 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291303:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae934 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291311:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0xdcc87686 (ffffa07433235000) 00010000:00010000:1.0:1787739082.291314:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.291317:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae934 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291323:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1ae934 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291350:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae95e lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291356:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae95e lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291363:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae95e lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291369:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306088e00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739082.291372:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae95e lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291378:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae95e lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291386:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0xbc0ca3a9 (ffffa074126f6500) 00010000:00010000:1.0:1787739082.291388:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.291391:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae95e lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291397:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1ae95e lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291409:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1ae988 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291448:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1ae988 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291455:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1ae988 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291460:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306089e00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739082.291464:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1ae988 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291470:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1ae988 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291477:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x391d5834 (ffffa074126f6e00) 00010000:00010000:1.0:1787739082.291480:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.291482:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1ae988 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xce:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291488:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1ae988 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xce:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291502:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1ae9b2 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291511:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1ae9b2 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291517:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1ae9b2 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291522:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306089400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739082.291524:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1ae9b2 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291597:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1ae9b2 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291607:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x18618557 (ffffa074126f6700) 00010000:00010000:1.0:1787739082.291609:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.291612:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1ae9b2 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xce:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291619:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1ae9b2 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xce:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291633:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae9dc lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291639:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae9dc lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291646:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae9dc lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291716:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1a00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739082.291780:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae9dc lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291789:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae9dc lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291798:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0xb62e0cbf (ffffa07403eeb400) 00010000:00010000:1.0:1787739082.291801:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.291805:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae9dc lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291811:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1ae9dc lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291827:0:3637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aea06 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291834:0:3637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aea06 lrc: 2/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291840:0:3637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aea06 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291845:0:3637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739082.291848:0:3637:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aea06 lrc: 3/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291855:0:3637:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aea06 lrc: 4/0,0 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291862:0:3637:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x84832a57 (ffffa07403eeb800) 00010000:00010000:1.0:1787739082.291864:0:3637:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.291867:0:3637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aea06 lrc: 2/0,0 mode: --/TXN res: [0x200000400:0xce:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.291874:0:3637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aea06 lrc: 0/0,0 mode: --/TXN res: [0x200000400:0xce:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739082.292315:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665788544 00000100:00100000:3.0:1787739082.292324:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665788544 00000100:00100000:3.0:1787739082.292432:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3642 00000100:00100000:3.0:1787739082.292443:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:6536:x1874579665788544:12345-0@lo:101:mdt_io00_000.0 00010000:00010000:3.0:1787739082.292461:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739082.292480:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aea76 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x152b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aea6f expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.292505:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aea76 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x152b:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aea6f expref: 12 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.292519:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aea76 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x152b:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aea6f expref: 12 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.292787:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152b:0x0].0x0 (ffffa074331c2000) 00010000:00010000:3.0:1787739082.292792:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 25 00010000:00010000:3.0:1787739082.292796:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab600, rc 0) 00000100:00100000:3.0:1787739082.292874:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665788544:12345-0@lo:101:mdt_io00_000.0 Request processed in 434us (1891us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.292954:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3642 00000100:00100000:0.0:1787739082.293021:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f4780 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665788544:0@lo:101:mdt_io00_000.0 00010000:00010000:0.0:1787739082.293039:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1aea6f lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x152b:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aea76 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.293049:0:6536:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1aea6f lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x152b:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aea76 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739082.293069:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000402:0x152b:0x0] 00000100:00100000:0.0:1787739082.293115:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f4780 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665788672:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.293150:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.293383:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a33c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.293495:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665788672 00000100:00100000:2.0:1787739082.293502:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665788672 00000100:00100000:2.0:1787739082.293541:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3643 00000100:00100000:2.0:1787739082.293548:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665788672:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.293617:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665788672:12345-0@lo:1000:mdt_io00_000.0 Request processed in 69us (466us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.293624:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3643 00000100:00100000:0.0:1787739082.293771:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f4780 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665788672:0@lo:1000:mdt_io00_000.0 00000004:00080000:0.0:1787739082.293793:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000402:0x152b:0x0] 00000100:00100000:0.0:1787739082.293819:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f4780 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665788800:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.293844:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.293858:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a33c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.293997:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665788800 00000100:00100000:2.0:1787739082.294002:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665788800 00000100:00100000:2.0:1787739082.294035:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3644 00000100:00100000:2.0:1787739082.294042:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665788800:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.294128:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665788800:12345-0@lo:1000:mdt_io00_000.0 Request processed in 86us (282us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.294134:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3644 00000100:00100000:0.0:1787739082.294246:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f4780 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665788800:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.294895:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f4780 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665788928:0@lo:403:mdt_io00_000.0 00000100:00100000:0.0:1787739082.294934:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.295039:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3900 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.295198:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665788928 00000100:00100000:1.0:1787739082.295203:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665788928 00000100:00100000:2.0:1787739082.295245:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3645 00000100:00100000:2.0:1787739082.295254:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665788928:12345-0@lo:403:mdt_io00_000.0 00000100:00100000:2.0:1787739082.297110:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665788928:12345-0@lo:403:mdt_io00_000.0 Request processed in 1846us (2164us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.297125:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3645 00000100:00100000:0.0:1787739082.297390:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f4780 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665788928:0@lo:403:mdt_io00_000.0 00000100:00100000:0.0:1787739082.298175:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074074f4780 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665789056:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.298219:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.298247:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a33c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.298652:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665789056 00000100:00100000:3.0:1787739082.298659:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665789056 00000100:00100000:3.0:1787739082.298697:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3646 00000100:00100000:3.0:1787739082.298706:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042452c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665789056:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.298778:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042452c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665789056:12345-0@lo:1000:mdt_io00_000.0 Request processed in 73us (558us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.298784:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3646 00000100:00100000:0.0:1787739082.298875:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074074f4780 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665789056:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.298924:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304247c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665789184:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.298948:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.298970:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a33c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.299039:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665789184 00000100:00100000:3.0:1787739082.299043:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665789184 00000100:00100000:3.0:1787739082.299061:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3647 00000100:00100000:3.0:1787739082.299067:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665789184:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.299351:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665789184:12345-0@lo:1000:mdt_io00_000.0 Request processed in 283us (401us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.299361:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3647 00000100:00100000:0.0:1787739082.299846:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304247c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665789184:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.300759:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304247c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665789312:0@lo:403:mdt_io00_000.0 00000100:00100000:0.0:1787739082.300812:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.300899:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3900 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.300990:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665789312 00000100:00100000:1.0:1787739082.300998:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665789312 00000100:00100000:1.0:1787739082.301028:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3648 00000100:00100000:1.0:1787739082.301035:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042630c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665789312:12345-0@lo:403:mdt_io00_000.0 00000100:00100000:1.0:1787739082.303709:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042630c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665789312:12345-0@lo:403:mdt_io00_000.0 Request processed in 2667us (2888us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.303722:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3648 00000100:00100000:0.0:1787739082.304384:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304247c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665789312:0@lo:403:mdt_io00_000.0 00000100:00100000:0.0:1787739082.305405:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304262940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665789440:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.305701:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.305786:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a33c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.306082:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739082.306162:0:6517:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652551296 00000100:00100000:1.0:1787739082.306169:0:6517:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652551296 00000100:00100000:1.0:1787739082.306220:0:6517:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 457 00000100:00100000:1.0:1787739082.306229:0:6517:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0740ea12450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:53427813-ebb7-48fc-94b6-ef03330d7ee5+10:2406:x1874579652551296:12345-192.168.203.15@tcp:400:kworker.0 00000100:00100000:1.0:1787739082.306286:0:6517:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0740ea12450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:53427813-ebb7-48fc-94b6-ef03330d7ee5+10:2406:x1874579652551296:12345-192.168.203.15@tcp:400:kworker.0 Request processed in 57us (206us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.306293:0:6517:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 457 00000100:00100000:3.0:1787739082.306940:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665789440 00000100:00100000:3.0:1787739082.306946:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665789440 00000100:00100000:3.0:1787739082.307016:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3649 00000100:00100000:3.0:1787739082.307024:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042612c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665789440:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.307135:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042612c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665789440:12345-0@lo:1000:mdt_io00_000.0 Request processed in 112us (1433us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.307141:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3649 00000100:00100000:0.0:1787739082.307202:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304262940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665789440:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.307263:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304262940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665789568:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.307292:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.307477:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a33c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.307659:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665789568 00000100:00100000:1.0:1787739082.307664:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665789568 00000100:00100000:1.0:1787739082.307687:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3650 00000100:00100000:1.0:1787739082.307693:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304260780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665789568:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787739082.307751:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304260780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665789568:12345-0@lo:1000:mdt_io00_000.0 Request processed in 57us (458us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.307762:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3650 00000100:00100000:0.0:1787739082.307826:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304262940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665789568:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.307857:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304262940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665789696:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.307889:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.307908:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a33c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.307942:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665789696 00000100:00100000:3.0:1787739082.307946:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665789696 00000100:00100000:3.0:1787739082.307967:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3651 00000100:00100000:3.0:1787739082.307972:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042603c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665789696:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.308032:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042603c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665789696:12345-0@lo:1000:mdt_io00_000.0 Request processed in 58us (151us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.308038:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3651 00000100:00100000:0.0:1787739082.308190:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304262940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665789696:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.308219:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304262940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665789824:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.308243:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.308262:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a33c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.308296:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665789824 00000100:00100000:2.0:1787739082.308300:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665789824 00000100:00100000:2.0:1787739082.308364:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3652 00000100:00100000:2.0:1787739082.308372:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304247c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665789824:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.308456:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304247c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665789824:12345-0@lo:1000:mdt_io00_000.0 Request processed in 84us (211us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.308465:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3652 00000100:00100000:0.0:1787739082.308520:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304262940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665789824:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.308644:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304262940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665789952:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.308668:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.308808:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a33c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.308864:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665789952 00000100:00100000:1.0:1787739082.308869:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665789952 00000100:00100000:1.0:1787739082.308901:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3653 00000100:00100000:1.0:1787739082.308907:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07420987480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665789952:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787739082.308968:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07420987480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665789952:12345-0@lo:1000:mdt_io00_000.0 Request processed in 61us (299us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.308975:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3653 00000100:00100000:0.0:1787739082.309036:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304262940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665789952:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.309667:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304262940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665790080:0@lo:403:mdt_io00_000.0 00000100:00100000:0.0:1787739082.309709:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.309729:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3900 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.309857:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665790080 00000100:00100000:1.0:1787739082.309863:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665790080 00000100:00100000:1.0:1787739082.309897:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3654 00000100:00100000:1.0:1787739082.309904:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be43c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665790080:12345-0@lo:403:mdt_io00_000.0 00000100:00100000:1.0:1787739082.311550:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be43c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665790080:12345-0@lo:403:mdt_io00_000.0 Request processed in 1640us (1834us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.311563:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3654 00000100:00100000:0.0:1787739082.311852:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304262940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665790080:0@lo:403:mdt_io00_000.0 00000100:00100000:0.0:1787739082.312695:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be7480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665790208:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.312744:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.312769:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a33c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.312865:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665790208 00000100:00100000:1.0:1787739082.312872:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665790208 00000100:00100000:1.0:1787739082.312916:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3655 00000100:00100000:1.0:1787739082.312924:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665790208:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787739082.312989:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665790208:12345-0@lo:1000:mdt_io00_000.0 Request processed in 65us (244us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.312996:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3655 00000100:00100000:0.0:1787739082.313075:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be7480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665790208:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.313208:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be7480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665790336:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.313232:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.313254:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.313302:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665790336 00000100:00100000:1.0:1787739082.313307:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665790336 00000100:00100000:1.0:1787739082.313325:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3656 00000100:00100000:1.0:1787739082.313331:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665790336:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787739082.313466:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665790336:12345-0@lo:1000:mdt_io00_000.0 Request processed in 135us (233us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.313473:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3656 00000100:00100000:0.0:1787739082.313679:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be7480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665790336:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.313714:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be7480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665790464:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.313739:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.313791:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.313899:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665790464 00000100:00100000:3.0:1787739082.313904:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665790464 00000100:00100000:1.0:1787739082.313947:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3657 00000100:00100000:1.0:1787739082.313954:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be52c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665790464:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787739082.314014:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be52c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665790464:12345-0@lo:1000:mdt_io00_000.0 Request processed in 60us (274us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.314021:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3657 00000100:00100000:0.0:1787739082.314083:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be7480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665790464:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.314186:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be7480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665790592:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.314213:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.314273:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.314311:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665790592 00000100:00100000:2.0:1787739082.314316:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665790592 00000100:00100000:2.0:1787739082.314349:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3658 00000100:00100000:2.0:1787739082.314355:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665790592:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.314519:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:6536:x1874579665790592:12345-0@lo:1000:mdt_io00_000.0 Request processed in 163us (304us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.314622:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3658 00000100:00100000:0.0:1787739082.314751:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be7480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665790592:0@lo:1000:mdt_io00_000.0 00000040:00080000:0.0:1787739082.315877:0:6536:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000400:0x3:0x0].380, 2464 off570976 00000040:00080000:0.0:1787739082.316022:0:6536:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000401:0x3:0x0].381, 2544 off625136 00000004:00080000:0.0:1787739082.316028:0:6536:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa073068a3900 with version 719 00000004:00080000:1.0:1787739082.316099:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 719 rpc_version 719 00000020:00100000:1.0:1787739082.316134:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739082.316143:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303a30780 x1874579665790848/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739082.316161:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a30780 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665790848:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739082.316217:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.316248:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241600 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.316359:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665790848 00000100:00100000:3.0:1787739082.316366:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665790848 00000100:00100000:3.0:1787739082.316430:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3659 00000100:00100000:3.0:1787739082.316439:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+12:7467:x1874579665790848:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739082.317122:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+13:7467:x1874579665790848:12345-0@lo:1000:osp_up0-1.0 Request processed in 683us (902us total) trans 4294983509 rc 0/0 00000100:00100000:3.0:1787739082.317132:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3659 00000100:00100000:1.0:1787739082.317398:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a30780 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665790848:0@lo:1000:osp_up0-1.0 00010000:00010000:0.0:1787739082.317508:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089400/0x7ee4453e4a1aea7d lrc: 3/0,1 mode: --/PW res: [0x240000402:0x186:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.317580:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089400/0x7ee4453e4a1aea7d lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x186:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.317594:0:6536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089400/0x7ee4453e4a1aea7d lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x186:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.317616:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089e00/0x7ee4453e4a1aea84 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.317624:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089e00/0x7ee4453e4a1aea84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.317631:0:6536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089e00/0x7ee4453e4a1aea84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.317661:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088e00/0x7ee4453e4a1aea8b lrc: 3/0,1 mode: --/PW res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.317672:0:6536:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088e00/0x7ee4453e4a1aea8b lrc: 3/0,1 mode: --/PW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.317682:0:6536:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088e00/0x7ee4453e4a1aea8b lrc: 3/0,1 mode: --/PW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.317697:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be7480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665790976:0@lo:101:mdt_io00_000.0 00000100:00100000:0.0:1787739082.317729:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.317753:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3840 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.317806:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665790976 00000100:00100000:1.0:1787739082.317811:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665790976 00000100:00100000:1.0:1787739082.317847:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3660 00000100:00100000:1.0:1787739082.317854:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304262940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+13:6536:x1874579665790976:12345-0@lo:101:mdt_io00_000.0 00010000:00010000:1.0:1787739082.317869:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739082.317893:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aea92 lrc: 2/0,0 mode: --/PW res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aea8b expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.317913:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aea92 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aea8b expref: 14 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.317925:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aea92 lrc: 3/0,0 mode: PW/PW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aea8b expref: 14 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.317933:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa0730713a800) 00010000:00010000:1.0:1787739082.317936:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.317939:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306840a00, rc 0) 00000100:00100000:1.0:1787739082.317978:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304262940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+14:6536:x1874579665790976:12345-0@lo:101:mdt_io00_000.0 Request processed in 124us (247us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.317985:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3660 00000100:00100000:0.0:1787739082.318051:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be7480 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0001-mdtlov_UUID:6536:1874579665790976:0@lo:101:mdt_io00_000.0 00010000:00010000:0.0:1787739082.318064:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088e00/0x7ee4453e4a1aea8b lrc: 4/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aea92 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.318079:0:6536:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088e00/0x7ee4453e4a1aea8b lrc: 4/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aea92 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739082.318165:0:6536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000aa4261d reply state 00000000a0951399 transno 4294968487 ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089e00/0x7ee4453e4a1aea84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.318178:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089e00/0x7ee4453e4a1aea84 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.318184:0:6536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa0741590d600) 00010000:00010000:0.0:1787739082.318187:0:6536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739082.318189:0:6536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739082.318195:0:6536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088e00/0x7ee4453e4a1aea8b lrc: 4/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aea92 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.318202:0:6536:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088e00/0x7ee4453e4a1aea8b lrc: 3/0,0 mode: PW/PW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aea92 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739082.318211:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000400:0xce:0x0] 00000004:00010000:0.0:1787739082.318220:0:6536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000aa4261d reply state 00000000a0951399 transno 4294968487 ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089400/0x7ee4453e4a1aea7d lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x186:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.318228:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089400/0x7ee4453e4a1aea7d lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x186:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.318235:0:6536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x186:0x0].0x0 (ffffa0741590d200) 00010000:00010000:0.0:1787739082.318237:0:6536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739082.318239:0:6536:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00080000:0.0:1787739082.318246:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000402:0x152b:0x0] 00010000:00010000:0.0:1787739082.318252:0:6536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1aea6f lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x152b:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aea76 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.318260:0:6536:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1aea6f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x152b:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aea76 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739082.318268:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787739082.318274:0:6536:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089a00/0x7ee4453e4a1aea53 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aea5a expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.318281:0:6536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089a00/0x7ee4453e4a1aea53 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aea5a expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.318288:0:6536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089a00/0x7ee4453e4a1aea53 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aea5a expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.318314:0:6536:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739082.318319:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787739082.318324:0:6536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1aea61 lrc: 4/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aea68 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.318331:0:6536:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1aea61 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aea68 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739082.318342:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:0.0:1787739082.318347:0:6536:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306840200/0x7ee4453e4a1aea45 lrc: 4/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aea4c expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.318354:0:6536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306840200/0x7ee4453e4a1aea45 lrc: 4/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aea4c expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.318360:0:6536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306840200/0x7ee4453e4a1aea45 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aea4c expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.318372:0:6536:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1787739082.318457:0:6536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:18317:x1874579652551168:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 43154us (43422us total) trans 4294968487 rc 0/0 00000100:00100000:0.0:1787739082.318464:0:6536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 0 00010000:00010000:1.0:1787739082.320443:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1787739082.320455:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089a00/0x7ee4453e4a1aea53 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aea5a expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.320468:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306089a00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739082.320475:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089a00/0x7ee4453e4a1aea53 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aea5a expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.320584:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739082.320591:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306840200/0x7ee4453e4a1aea45 lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aea4c expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.320599:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840200 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739082.320603:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306840200/0x7ee4453e4a1aea45 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aea4c expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739082.320611:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306840200/0x7ee4453e4a1aea45 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aea4c expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739082.320788:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089a00/0x7ee4453e4a1aea53 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aea5a expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.320810:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089a00/0x7ee4453e4a1aea53 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aea5a expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.320817:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739082.320853:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089a00/0x7ee4453e4a1aea53 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aea5a expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.320865:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306089a00/0x7ee4453e4a1aea53 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aea5a expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.320879:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739082.320891:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739082.320893:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739082.320907:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a321c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665791232:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739082.320977:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1787739082.321006:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306840200/0x7ee4453e4a1aea45 lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aea4c expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.321013:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739082.321031:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306840200/0x7ee4453e4a1aea45 lrc: 1/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aea4c expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.321038:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306840200/0x7ee4453e4a1aea45 lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aea4c expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.321046:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665791232 00000100:00100000:0.0:1787739082.321052:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665791232 00010000:00010000:2.0:1787739082.321057:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739082.321058:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739082.321059:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739082.321069:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787739082.321079:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665791104:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739082.321107:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1427 00000100:00100000:0.0:1787739082.321115:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383d680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+14:3653:x1874579665791232:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739082.321127:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739082.321145:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841a00/0x7ee4453e4a1aea5a lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aea53 expref: 14 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.321161:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa07401795300) 00010000:00010000:0.0:1787739082.321165:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739082.321197:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383d680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+13:3653:x1874579665791232:12345-0@lo:103:ldlm_bl.0 Request processed in 83us (220us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.321203:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1427 00000100:00100000:3.0:1787739082.321328:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a321c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3653:1874579665791232:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739082.321358:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.321637:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665791104 00000100:00100000:1.0:1787739082.321641:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665791104 00000100:00100000:1.0:1787739082.321712:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1428 00000100:00100000:1.0:1787739082.321719:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+13:3651:x1874579665791104:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739082.321737:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739082.321750:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1aea4c lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aea45 expref: 13 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.321762:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffffa07433235700) 00010000:00010000:1.0:1787739082.321768:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739082.321799:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0001-mdtlov_UUID+12:3651:x1874579665791104:12345-0@lo:103:ldlm_bl.0 Request processed in 81us (441us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.321805:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1428 00000100:00100000:2.0:1787739082.321915:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665791104:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739082.323212:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1aea61 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aea68 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739082.323229:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1aea61 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aea68 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.323262:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1aea61 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aea68 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.323269:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739082.323300:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088600/0x7ee4453e4a1aea61 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aea68 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739082.323317:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665791360:0@lo:103:jbd2/dm-1-8.0 00010000:00010000:3.0:1787739082.323319:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1aea6f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x152b:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aea76 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739082.323325:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1aea6f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x152b:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aea76 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.323340:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1aea6f lrc: 2/0,0 mode: --/EX res: [0x200000402:0x152b:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aea76 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.323345:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739082.323357:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088000/0x7ee4453e4a1aea6f lrc: 0/0,0 mode: --/EX res: [0x200000402:0x152b:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aea76 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739082.323365:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1787739082.323369:0:7453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088e00/0x7ee4453e4a1aea8b lrc: 3/0,0 mode: PW/PW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aea92 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739082.323380:0:7453:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088e00/0x7ee4453e4a1aea8b lrc: 3/0,0 mode: PW/PW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aea92 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.323394:0:7453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088e00/0x7ee4453e4a1aea8b lrc: 2/0,0 mode: --/PW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aea92 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.323399:0:7453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787739082.323405:0:7453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: ffffa07306088e00/0x7ee4453e4a1aea8b lrc: 0/0,0 mode: --/PW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aea92 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739082.323405:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665791488:0@lo:103:jbd2/dm-1-8.0 00000001:00080000:3.0:1787739082.323439:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968487 is committed 00000001:00080000:3.0:1787739082.323448:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968486 is committed 00000100:00100000:2.0:1787739082.323449:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.323465:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000020:00080000:3.0:1787739082.323468:0:7453:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0001: 388 batchid updated 00000100:00100000:2.0:1787739082.323470:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665791616:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:2.0:1787739082.323502:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000020:00080000:2.0:1787739082.323530:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 388 00000004:00080000:2.0:1787739082.323590:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07306107900 with version 720 00000020:00080000:2.0:1787739082.323601:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 381 cancel update log [0x200000401:0x3:0x0].372: rc = 0 00000100:00100000:0.0:1787739082.323638:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665791360 00000100:00100000:0.0:1787739082.323645:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665791360 00000020:00080000:2.0:1787739082.323668:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 381 cancel update log [0x240000400:0x3:0x0].372: rc = 0 00000004:00080000:2.0:1787739082.323691:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07306107480 with version 721 00000100:00100000:0.0:1787739082.323691:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665791488 00000020:00080000:2.0:1787739082.323694:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 382 cancel update log [0x200000401:0x3:0x0].373: rc = 0 00000100:00100000:0.0:1787739082.323694:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665791488 00000100:00100000:0.0:1787739082.323709:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665791616 00000100:00100000:0.0:1787739082.323711:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665791616 00000020:00080000:2.0:1787739082.323713:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 382 cancel update log [0x240000400:0x3:0x0].373: rc = 0 00000100:00100000:0.0:1787739082.323726:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1429 00000100:00100000:0.0:1787739082.323734:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+14:3651:x1874579665791360:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:0.0:1787739082.323745:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739082.323761:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aea68 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aea61 expref: 14 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.323775:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xf5d6a2c1 (ffffa074126f6800) 00010000:00010000:0.0:1787739082.323784:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739082.323821:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+13:3651:x1874579665791360:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 86us (453us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.323828:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1429 00000100:00100000:0.0:1787739082.323844:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1430 00000100:00100000:0.0:1787739082.323849:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561fc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+12:3652:x1874579665791488:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:0.0:1787739082.323856:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739082.323867:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aea76 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x152b:0x0].0x0 bits 0x25/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aea6f expref: 12 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.323876:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152b:0x0].0x0 (ffffa074331c2000) 00010000:00010000:0.0:1787739082.323891:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739082.323917:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561fc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+11:3652:x1874579665791488:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 68us (468us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.323922:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1430 00000100:00100000:0.0:1787739082.323935:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1431 00000100:00100000:0.0:1787739082.323941:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561de00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+10:3652:x1874579665791616:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:0.0:1787739082.323949:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739082.323960:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aea92 lrc: 0/0,0 mode: --/PW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aea8b expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.323973:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa0730713a800) 00010000:00010000:0.0:1787739082.323980:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739082.324000:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561de00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0001-mdtlov_UUID+9:3652:x1874579665791616:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 59us (498us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.324005:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1431 00000004:00080000:2.0:1787739082.324125:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07303803480 with version 722 00000020:00080000:2.0:1787739082.324129:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 383 cancel update log [0x200000401:0x3:0x0].374: rc = 0 00000020:00080000:2.0:1787739082.324153:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 383 cancel update log [0x240000400:0x3:0x0].374: rc = 0 00000004:00080000:2.0:1787739082.324176:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07435539240 with version 723 00000020:00080000:2.0:1787739082.324179:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 384 cancel update log [0x200000401:0x3:0x0].375: rc = 0 00000020:00080000:2.0:1787739082.324197:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 384 cancel update log [0x240000400:0x3:0x0].375: rc = 0 00000004:00080000:2.0:1787739082.324215:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa07435539540 with version 724 00000020:00080000:2.0:1787739082.324217:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 385 cancel update log [0x200000401:0x3:0x0].376: rc = 0 00000020:00080000:2.0:1787739082.324305:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 385 cancel update log [0x240000400:0x3:0x0].376: rc = 0 00000004:00080000:2.0:1787739082.324328:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa074118e0300 with version 725 00000020:00080000:2.0:1787739082.324330:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 386 cancel update log [0x200000401:0x3:0x0].377: rc = 0 00000020:00080000:2.0:1787739082.324347:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 386 cancel update log [0x240000400:0x3:0x0].377: rc = 0 00000004:00080000:2.0:1787739082.324365:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa074052af000 with version 726 00000020:00080000:2.0:1787739082.324367:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 387 cancel update log [0x200000401:0x3:0x0].378: rc = 0 00000020:00080000:2.0:1787739082.324383:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 387 cancel update log [0x240000400:0x3:0x0].378: rc = 0 00000004:00080000:2.0:1787739082.324398:0:7475:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth ffffa074052afd80 with version 727 00000020:00080000:2.0:1787739082.324400:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 388 cancel update log [0x200000401:0x3:0x0].379: rc = 0 00000020:00080000:2.0:1787739082.324445:0:7475:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 388 cancel update log [0x240000400:0x3:0x0].379: rc = 0 00000020:00080000:2.0:1787739082.324452:0:7475:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 388 00010000:00010000:1.0:1787739082.324456:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089400/0x7ee4453e4a1aea7d lrc: 3/0,1 mode: TXN/TXN res: [0x240000402:0x186:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.324465:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089400/0x7ee4453e4a1aea7d lrc: 2/0,0 mode: TXN/TXN res: [0x240000402:0x186:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.324471:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089400/0x7ee4453e4a1aea7d lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x186:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.324481:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306089400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739082.324485:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089400/0x7ee4453e4a1aea7d lrc: 3/0,0 mode: TXN/TXN res: [0x240000402:0x186:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.324492:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089400/0x7ee4453e4a1aea7d lrc: 4/0,0 mode: TXN/TXN res: [0x240000402:0x186:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.324502:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x186:0x0].0x0 (ffffa0741590d200) 00010000:00010000:1.0:1787739082.324504:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.324508:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089400/0x7ee4453e4a1aea7d lrc: 2/0,0 mode: --/TXN res: [0x240000402:0x186:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.324514:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089400/0x7ee4453e4a1aea7d lrc: 0/0,0 mode: --/TXN res: [0x240000402:0x186:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739082.324537:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383ed00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665791616:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:2.0:1787739082.324551:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383d2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665791488:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:2.0:1787739082.324582:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383cb40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3651:1874579665791360:0@lo:103:jbd2/dm-1-8.0 00010000:00010000:1.0:1787739082.324592:0:3636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089e00/0x7ee4453e4a1aea84 lrc: 3/0,1 mode: TXN/TXN res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739082.324596:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 720 rpc_version 720 00010000:00010000:1.0:1787739082.324599:0:3636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089e00/0x7ee4453e4a1aea84 lrc: 2/0,0 mode: TXN/TXN res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.324604:0:3636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089e00/0x7ee4453e4a1aea84 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.324610:0:3636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306089e00 already unused, calling callback (ffffffffc18da4f0) 00000020:00100000:2.0:1787739082.324613:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00010000:00010000:1.0:1787739082.324613:0:3636:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089e00/0x7ee4453e4a1aea84 lrc: 3/0,0 mode: TXN/TXN res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.324618:0:3636:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089e00/0x7ee4453e4a1aea84 lrc: 4/0,0 mode: TXN/TXN res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739082.324621:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0730561ed00 x1874579665791872/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787739082.324632:0:3636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa0741590d600) 00010000:00010000:1.0:1787739082.324634:0:3636:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.324636:0:3636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089e00/0x7ee4453e4a1aea84 lrc: 2/0,0 mode: --/TXN res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739082.324642:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561ed00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665791872:0@lo:1000:osp_up0-1.0 00010000:00010000:1.0:1787739082.324642:0:3636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089e00/0x7ee4453e4a1aea84 lrc: 0/0,0 mode: --/TXN res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6536 initiator: MDT1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739082.324675:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.324695:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739082.324813:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665791872 00000100:00100000:0.0:1787739082.324818:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665791872 00000100:00100000:0.0:1787739082.324856:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3661 00000100:00100000:0.0:1787739082.324864:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+9:7467:x1874579665791872:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739082.324999:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665791872:12345-0@lo:1000:osp_up0-1.0 Request processed in 135us (322us total) trans 4294983510 rc 0/0 00000100:00100000:0.0:1787739082.325006:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3661 00000100:00100000:2.0:1787739082.325159:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561ed00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665791872:0@lo:1000:osp_up0-1.0 00000004:00080000:2.0:1787739082.325169:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 721 rpc_version 721 00000020:00100000:2.0:1787739082.325185:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739082.325190:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0730561f0c0 x1874579665792128/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739082.325202:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561f0c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665792128:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739082.325226:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.325278:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.329206:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665792128 00000100:00100000:3.0:1787739082.329221:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665792128 00000100:00100000:3.0:1787739082.329324:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3662 00000100:00100000:3.0:1787739082.329335:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+10:7467:x1874579665792128:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1787739082.329780:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561cf00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+11:7467:x1874579665792128:12345-0@lo:1000:osp_up0-1.0 Request processed in 446us (4551us total) trans 4294983511 rc 0/0 00000100:00100000:3.0:1787739082.329790:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3662 00000100:00100000:2.0:1787739082.329970:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739082.330031:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561f0c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665792128:0@lo:1000:osp_up0-1.0 00000004:00080000:2.0:1787739082.330045:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 722 rpc_version 722 00000020:00100000:2.0:1787739082.330066:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739082.330072:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0730561d2c0 x1874579665792384/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739082.330168:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652551424 00000100:00100000:0.0:1787739082.330173:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652551424 00000100:00100000:0.0:1787739082.330208:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23705 00000100:00100000:0.0:1787739082.330216:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+15:18317:x1874579652551424:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739082.330242:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739082.330262:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1aea99 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095558 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.330323:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1aeaa0 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.330338:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1aeaa0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.330350:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1aeaa0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739082.330395:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 262144 00000004:00200000:0.0:1787739082.330399:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1787739082.330440:0:6524:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1aeaa0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.330463:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1aea99 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095558 expref: 17 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.330479:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089c00/0x7ee4453e4a1aeaa0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095558 expref: 16 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.330487:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa07401795300) 00010000:00010000:0.0:1787739082.330490:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1787739082.330494:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306089c00, rc 0) 00000100:00100000:0.0:1787739082.330720:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561e1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:18317:x1874579652551424:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 504us (751us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.330728:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23705 00000100:00100000:2.0:1787739082.331454:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561d2c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665792384:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739082.331511:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.334822:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.335031:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665792384 00000100:00100000:1.0:1787739082.335041:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665792384 00000100:00100000:1.0:1787739082.335106:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3663 00000100:00100000:1.0:1787739082.335115:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156eda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+11:7467:x1874579665792384:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739082.335314:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156eda40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+12:7467:x1874579665792384:12345-0@lo:1000:osp_up0-1.0 Request processed in 200us (3802us total) trans 4294983512 rc 0/0 00000100:00100000:1.0:1787739082.335322:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3663 00000100:00100000:2.0:1787739082.337070:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561d2c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665792384:0@lo:1000:osp_up0-1.0 00000004:00080000:2.0:1787739082.337096:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 723 rpc_version 723 00000020:00100000:2.0:1787739082.337123:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739082.337131:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074156ec780 x1874579665792640/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739082.337152:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ec780 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665792640:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739082.337207:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.337393:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.337597:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665792640 00000100:00100000:3.0:1787739082.337604:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665792640 00000100:00100000:2.0:1787739082.337714:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.337916:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3664 00000100:00100000:0.0:1787739082.337930:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+12:7467:x1874579665792640:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1787739082.338083:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+13:7467:x1874579665792640:12345-0@lo:1000:osp_up0-1.0 Request processed in 157us (875us total) trans 4294983513 rc 0/0 00000100:00100000:0.0:1787739082.338101:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3664 00000100:00100000:2.0:1787739082.338168:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ec780 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665792640:0@lo:1000:osp_up0-1.0 00000004:00080000:2.0:1787739082.338178:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 724 rpc_version 724 00000020:00100000:2.0:1787739082.338192:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739082.338197:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074156ed2c0 x1874579665792896/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739082.338212:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ed2c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665792896:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739082.338250:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.338679:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.338793:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665792896 00000100:00100000:1.0:1787739082.338798:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665792896 00000100:00100000:1.0:1787739082.338831:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3665 00000100:00100000:1.0:1787739082.338837:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+13:7467:x1874579665792896:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739082.338947:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+14:7467:x1874579665792896:12345-0@lo:1000:osp_up0-1.0 Request processed in 109us (696us total) trans 4294983514 rc 0/0 00000100:00100000:1.0:1787739082.338954:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3665 00000100:00100000:1.0:1787739082.338975:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652551680 00000100:00100000:1.0:1787739082.338979:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652551680 00000100:00100000:1.0:1787739082.339009:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23706 00000100:00100000:1.0:1787739082.339015:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ee940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+16:18317:x1874579652551680:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739082.339025:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739082.339045:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1aeaa7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09555f expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.339120:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aeaae lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.339134:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aeaae lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.339146:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aeaae lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.339162:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1aeab5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.339170:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1aeab5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.339177:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1aeab5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739082.339286:0:7944:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665793024:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739082.339318:0:7944:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.339335:0:7944:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062413c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.339354:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665793024 00000100:00100000:1.0:1787739082.339358:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665793024 00000100:00100000:1.0:1787739082.339376:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3666 00000100:00100000:1.0:1787739082.339381:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a321c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665793024:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1787739082.339473:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a321c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:7944:x1874579665793024:12345-0@lo:1000:mdt00_003.0 Request processed in 90us (153us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.339479:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3666 00000100:00100000:1.0:1787739082.339507:0:7944:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968486 gen 2 00000004:00080000:1.0:1787739082.339512:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968473 committed transno 4294968486 00000004:00080000:1.0:1787739082.339773:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968474 committed transno 4294968486 00000004:00080000:1.0:1787739082.339784:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968475 committed transno 4294968486 00000004:00080000:1.0:1787739082.339792:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968476 committed transno 4294968486 00000004:00080000:1.0:1787739082.339799:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968477 committed transno 4294968486 00000004:00080000:1.0:1787739082.339806:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968478 committed transno 4294968486 00000004:00080000:1.0:1787739082.339814:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968479 committed transno 4294968486 00000004:00080000:1.0:1787739082.339821:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968480 committed transno 4294968486 00000004:00080000:1.0:1787739082.339831:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968481 committed transno 4294968486 00000004:00080000:1.0:1787739082.339839:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968482 committed transno 4294968486 00000004:00080000:1.0:1787739082.339847:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968483 committed transno 4294968486 00000004:00080000:1.0:1787739082.339868:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968484 committed transno 4294968486 00000004:00080000:1.0:1787739082.339876:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968485 committed transno 4294968486 00000004:00080000:1.0:1787739082.339895:0:7944:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968486 committed transno 4294968486 00000020:00080000:1.0:1787739082.339901:0:7944:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffffa074067019c0 refcount 2 committed 0 result 0 batchid 408 00000020:00080000:1.0:1787739082.339904:0:7944:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa074035e8e00 obd lustre-MDT0001-osp-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffffa07406701180 00000020:00080000:1.0:1787739082.339907:0:7944:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x240000401:0x3:0x0].404 00000020:00080000:1.0:1787739082.339910:0:7944:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffffa074035e8200 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffffa07411bdbe00 00000020:00080000:1.0:1787739082.339912:0:7944:0:(update_trans.c:72:top_multiple_thandle_dump()) cookie [0x200000400:0x3:0x0].404 00000100:00100000:1.0:1787739082.339936:0:7944:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32d00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:7944:1874579665793024:0@lo:1000:mdt00_003.0 00010000:00010000:1.0:1787739082.339971:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841e00/0x7ee4453e4a1aeabc lrc: 3/1,0 mode: --/PR res: [0x240000402:0x186:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.339981:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841e00/0x7ee4453e4a1aeabc lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x186:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.339991:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841e00/0x7ee4453e4a1aeabc lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x186:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.340007:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aeaae lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.340014:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aeaae lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.340019:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aeaae lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.340026:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aeaae lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.340031:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739082.340034:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aeaae lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.340041:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aeaae lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.340052:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa07401795300) 00010000:00010000:1.0:1787739082.340054:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.340058:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aeaae lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.340064:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aeaae lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.340077:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1aeab5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.340094:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1aeab5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.340101:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1aeab5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.340106:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306841800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739082.340109:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1aeab5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.340115:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1aeab5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.340123:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xf5d6a2c1 (ffffa0730713af00) 00010000:00010000:1.0:1787739082.340125:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.340127:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1aeab5 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.340132:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841800/0x7ee4453e4a1aeab5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.340169:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1aeaa7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09555f expref: 18 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.340182:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841e00/0x7ee4453e4a1aeabc lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x186:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09555f expref: 17 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.340188:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x186:0x0].0x0 (ffffa0730713a200) 00010000:00010000:1.0:1787739082.340190:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739082.340192:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841e00, rc 0) 00000100:00100000:1.0:1787739082.340302:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ee940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:18317:x1874579652551680:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 1286us (2589us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.340313:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23706 00000020:00080000:1.0:1787739082.340337:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 408 committed batchid 407 00000020:00080000:1.0:1787739082.340383:0:6555:0:(update_trans.c:1590:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 407 00000100:00100000:2.0:1787739082.340599:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ed2c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665792896:0@lo:1000:osp_up0-1.0 00000004:00080000:2.0:1787739082.340610:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 725 rpc_version 725 00000020:00100000:2.0:1787739082.340626:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739082.340633:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074156ef840 x1874579665793280/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739082.340658:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665793280:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739082.340698:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.340722:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.341175:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665793280 00000100:00100000:1.0:1787739082.341181:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665793280 00000100:00100000:1.0:1787739082.341210:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3667 00000100:00100000:1.0:1787739082.341227:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156efc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+14:7467:x1874579665793280:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739082.341338:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156efc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+15:7467:x1874579665793280:12345-0@lo:1000:osp_up0-1.0 Request processed in 119us (638us total) trans 4294983515 rc 0/0 00000100:00100000:1.0:1787739082.341346:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3667 00000100:00100000:2.0:1787739082.341556:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef840 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665793280:0@lo:1000:osp_up0-1.0 00000004:00080000:2.0:1787739082.341566:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 726 rpc_version 726 00000020:00100000:2.0:1787739082.341581:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739082.341587:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa074156ef0c0 x1874579665793536/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739082.341603:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665793536:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739082.341635:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.341664:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.342069:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665793536 00000100:00100000:1.0:1787739082.342075:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665793536 00000100:00100000:1.0:1787739082.342108:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3668 00000100:00100000:1.0:1787739082.342114:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+15:7467:x1874579665793536:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739082.342202:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ec3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+16:7467:x1874579665793536:12345-0@lo:1000:osp_up0-1.0 Request processed in 87us (566us total) trans 4294983516 rc 0/0 00000100:00100000:1.0:1787739082.342209:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3668 00000100:00100000:2.0:1787739082.344817:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef0c0 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665793536:0@lo:1000:osp_up0-1.0 00000004:00080000:2.0:1787739082.344837:0:7467:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07411b97f00 version 727 rpc_version 727 00000020:00100000:2.0:1787739082.344870:0:7467:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0000-osp-MDT0001: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739082.344878:0:7467:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07305f79e00 x1874579665793792/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 488/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739082.344910:0:7467:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f79e00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665793792:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1787739082.344975:0:7467:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.345015:0:7467:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af480 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.345070:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665793792 00000100:00100000:1.0:1787739082.345077:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665793792 00000100:00100000:1.0:1787739082.345127:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3669 00000100:00100000:1.0:1787739082.345135:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+16:7467:x1874579665793792:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739082.345434:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+17:7467:x1874579665793792:12345-0@lo:1000:osp_up0-1.0 Request processed in 279us (438us total) trans 4294983517 rc 0/0 00000100:00100000:1.0:1787739082.345443:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3669 00000100:00100000:2.0:1787739082.345523:0:7467:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f79e00 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:7467:1874579665793792:0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1787739082.345978:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739082.346048:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652551936 00000100:00100000:3.0:1787739082.346053:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652551936 00000100:00100000:3.0:1787739082.346172:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23707 00000100:00100000:3.0:1787739082.346182:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+6:18317:x1874579652551936:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739082.346197:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739082.346294:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aeac3 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x186:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095566 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739082.346479:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x186:0x0]: returning size 4096 00010000:00010000:3.0:1787739082.346737:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aeac3 lrc: 1/0,0 mode: --/CR res: [0x240000402:0x186:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095566 expref: 8 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.346752:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x186:0x0].0x0 (ffffa074331c2d00) 00010000:00010000:3.0:1787739082.346755:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739082.346760:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aeac3 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x186:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095566 expref: 8 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.346778:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abe00, rc 0) 00000100:00100000:3.0:1787739082.346940:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:18317:x1874579652551936:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 759us (963us total) trans 4294968488 rc 301/301 00000100:00100000:3.0:1787739082.346949:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23707 00000100:00100000:2.0:1787739082.351676:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.351762:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652552064 00000100:00100000:0.0:1787739082.351772:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652552064 00000100:00100000:0.0:1787739082.351827:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23708 00000100:00100000:0.0:1787739082.351838:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:18317:x1874579652552064:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739082.351866:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739082.351898:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088c00/0x7ee4453e4a1aead1 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09556d expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.351968:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089200/0x7ee4453e4a1aead8 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.351982:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089200/0x7ee4453e4a1aead8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.351993:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089200/0x7ee4453e4a1aead8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739082.352040:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xcc:0x0]: returning size 4096 00000004:00200000:0.0:1787739082.352045:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 3 MDT count 2 00010000:00010000:0.0:1787739082.352077:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306088c00/0x7ee4453e4a1aead1 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09556d expref: 9 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.352166:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089200/0x7ee4453e4a1aead8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09556d expref: 8 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.352176:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa0741590db00) 00010000:00010000:0.0:1787739082.352179:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739082.352182:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306089200, rc 0) 00000100:00100000:0.0:1787739082.352250:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:18317:x1874579652552064:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 413us (583us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.352257:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23708 00000100:00100000:1.0:1787739082.355703:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739082.358404:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652552192 00000100:00100000:2.0:1787739082.358429:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652552192 00000100:00100000:2.0:1787739082.358510:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23709 00000100:00100000:2.0:1787739082.358520:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f7c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+17:18317:x1874579652552192:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739082.358540:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739082.358562:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aeadf lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095574 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.358622:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1aeae6 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.358635:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1aeae6 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.358647:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1aeae6 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739082.358688:0:7944:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xce:0x0]: returning size 4096 00000004:00200000:2.0:1787739082.358693:0:7944:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 3 MDT count 2 00010000:00010000:2.0:1787739082.358728:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aeadf lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095574 expref: 19 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.358748:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1aeae6 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095574 expref: 18 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.358757:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:2.0:1787739082.358761:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739082.358764:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa1200, rc 0) 00000100:00100000:2.0:1787739082.358905:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f7c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:18317:x1874579652552192:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 386us (3212us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.358913:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23709 00000100:00100000:2.0:1787739082.365481:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739082.365637:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652552320 00000100:00100000:3.0:1787739082.365647:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652552320 00000100:00100000:3.0:1787739082.365703:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23710 00000100:00100000:3.0:1787739082.365717:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f79680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:18317:x1874579652552320:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739082.365739:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739082.365762:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aeaed lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09557b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.365827:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1aeaf4 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x186:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.365841:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1aeaf4 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x186:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.365853:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1aeaf4 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x186:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739082.365948:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x186:0x0]: returning size 4096 00000004:00200000:3.0:1787739082.365953:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 3 MDT count 2 00000004:00010000:3.0:1787739082.365972:0:6525:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1aeaf4 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x186:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.365996:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aeaed lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09557b expref: 10 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.366016:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1aeaf4 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x186:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09557b expref: 9 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.366025:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x186:0x0].0x0 (ffffa074331c2e00) 00010000:00010000:3.0:1787739082.366028:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1787739082.366031:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739082.366203:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f79680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:18317:x1874579652552320:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 490us (730us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.366211:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23710 00000100:00100000:1.0:1787739082.374659:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739082.374715:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652552448 00000100:00100000:3.0:1787739082.374732:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652552448 00000100:00100000:2.0:1787739082.374864:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5082 00000100:00100000:2.0:1787739082.374875:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:18317:x1874579652552448:12345-192.168.203.15@tcp:37:lfs.0 00000100:00100000:2.0:1787739082.379485:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:18317:x1874579652552448:12345-192.168.203.15@tcp:37:lfs.0 Request processed in 4601us (5085us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.379498:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5082 00000100:00100000:1.0:1787739082.389686:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739082.389876:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652552576 00000100:00100000:2.0:1787739082.389896:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652552576 00000100:00100000:2.0:1787739082.389953:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5083 00000100:00100000:2.0:1787739082.389963:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074332aa050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:18317:x1874579652552576:12345-192.168.203.15@tcp:37:lfs.0 00000100:00100000:0.0:1787739082.397139:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074332aa050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:18317:x1874579652552576:12345-192.168.203.15@tcp:37:lfs.0 Request processed in 7167us (7452us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.397157:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5083 00000100:00100000:1.0:1787739082.409971:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.410067:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652552704 00000100:00100000:0.0:1787739082.410077:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652552704 00000100:00100000:0.0:1787739082.410138:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23711 00000100:00100000:0.0:1787739082.410149:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:18317:x1874579652552704:12345-192.168.203.15@tcp:34:lfs.0 00010000:00010000:0.0:1787739082.410229:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1aeafb lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410246:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1aeafb lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410258:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1aeafb lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410274:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aeb02 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410281:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aeb02 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410288:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aeb02 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410343:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1aeb09 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x152d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410351:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1aeb09 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410358:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1aeb09 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739082.410450:0:6523:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1aeb09 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410465:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1aeafb lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410471:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1aeafb lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410477:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1aeafb lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410483:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1aeafb lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410489:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306088c00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739082.410494:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1aeafb lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410501:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1aeafb lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410513:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:0.0:1787739082.410516:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739082.410520:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1aeafb lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410807:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1aeafb lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410825:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aeb02 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410831:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aeb02 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410838:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aeb02 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410843:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306088a00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739082.410846:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aeb02 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410852:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aeb02 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410860:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0xbc0ca3a9 (ffffa0741590de00) 00010000:00010000:0.0:1787739082.410863:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739082.410865:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aeb02 lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410871:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aeb02 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410896:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1aeb09 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410902:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1aeb09 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x152d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410908:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1aeb09 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x152d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410913:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306089e00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739082.410916:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1aeb09 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x152d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410922:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1aeb09 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x152d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410929:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152d:0x0].0x0 (ffffa074126f6000) 00010000:00010000:0.0:1787739082.410932:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739082.410935:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1aeb09 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x152d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.410941:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1aeb09 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x152d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.411113:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:18317:x1874579652552704:12345-192.168.203.15@tcp:34:lfs.0 Request processed in 964us (1148us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.411121:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23711 00000100:00100000:1.0:1787739082.421090:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739082.421169:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652552960 00000100:00100000:3.0:1787739082.421179:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652552960 00000100:00100000:3.0:1787739082.421337:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23712 00000100:00100000:3.0:1787739082.421349:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+18:18317:x1874579652552960:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739082.421369:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739082.421390:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aeb10 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x152d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095582 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.421541:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aeb17 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x152d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.421557:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aeb17 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x152d:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.421568:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aeb17 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x152d:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739082.421581:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x152d:0x0]: returning size 2 00010000:00010000:3.0:1787739082.421913:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aeb10 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x152d:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095582 expref: 21 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.421932:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aeb17 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x152d:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095582 expref: 20 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.421942:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152d:0x0].0x0 (ffffa074126f6000) 00010000:00010000:3.0:1787739082.421946:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:3.0:1787739082.421949:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa000, rc 0) 00000100:00100000:3.0:1787739082.422000:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+20:18317:x1874579652552960:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 654us (920us total) trans 4294983518 rc 0/0 00000100:00100000:3.0:1787739082.422008:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23712 00000100:00100000:0.0:1787739082.427500:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739082.427626:0:14678:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652553088 00000100:00100000:1.0:1787739082.427635:0:14678:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652553088 00000100:00100000:1.0:1787739082.427698:0:14678:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5195 00000100:00100000:1.0:1787739082.427708:0:14678:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305656d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2409:x1874579652553088:12345-192.168.203.15@tcp:1:lfs.0 00010000:00010000:1.0:1787739082.427760:0:14678:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e8beb733/0x7ee4453e4a1aeb25 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14678 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.427778:0:14678:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004c07339b/0x7ee4453e4a1ae973 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09550b expref: 10 pid: 14680 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.427786:0:14678:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x9c8:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1787739082.427790:0:14678:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x9c8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787739082.427794:0:14678:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e8beb733/0x7ee4453e4a1aeb25 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14678 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.427825:0:14678:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004c07339b/0x7ee4453e4a1ae973 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09550b expref: 10 pid: 14680 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.427840:0:14678:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004c07339b/0x7ee4453e4a1ae973 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09550b expref: 10 pid: 14680 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739082.427853:0:14678:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb4780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:14678:1874579665793920:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739082.427954:0:14678:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073062413c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.434571:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739082.434715:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652553216 00000100:00100000:2.0:1787739082.434734:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652553216 00002000:00100000:2.0:1787739082.434787:0:8442:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x9c8:0x0] (0->1) req@ffffa07409fb6580 x1874579652553216/t0(0) o4->e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2@192.168.203.15@tcp:263/0 lens 496/0 e 0 to 0 dl 1787739093 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787739082.434798:0:8442:0:(ofd_dev.c:2572:ofd_prolong_extent_locks()) Prolong locks for req ffffa07409fb6580 with x1874579652553216 ext(0->1) 00010000:00010000:2.0:1787739082.434807:0:8442:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000004c07339b/0x7ee4453e4a1ae973 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09550b expref: 11 pid: 14680 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.434824:0:8442:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000004c07339b/0x7ee4453e4a1ae973 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09550b expref: 11 pid: 14680 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787739082.434833:0:8442:0:(ofd_dev.c:2721:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffffa07409fb6580 00000100:00100000:2.0:1787739082.434869:0:8442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 0 00000100:00100000:2.0:1787739082.434889:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb6580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2409:x1874579652553216:12345-192.168.203.15@tcp:4:bash.0 00000100:00100000:2.0:1787739082.435537:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb6580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:2409:x1874579652553216:12345-192.168.203.15@tcp:4:bash.0 Request processed in 646us (961us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.435547:0:8442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 0 00002000:00100000:2.0:1787739082.435554:0:8442:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x9c8:0x0] (0->1) req@ffffa07409fb6580 x1874579652553216/t0(0) o4->e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2@192.168.203.15@tcp:263/0 lens 496/448 e 0 to 0 dl 1787739093 ref 1 fl Complete:H/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787739082.435562:0:8442:0:(ofd_dev.c:2572:ofd_prolong_extent_locks()) Prolong locks for req ffffa07409fb6580 with x1874579652553216 ext(0->1) 00010000:00010000:2.0:1787739082.435569:0:8442:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000004c07339b/0x7ee4453e4a1ae973 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09550b expref: 12 pid: 14680 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.435581:0:8442:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000004c07339b/0x7ee4453e4a1ae973 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09550b expref: 12 pid: 14680 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787739082.435588:0:8442:0:(ofd_dev.c:2721:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffffa07409fb6580 00000100:00100000:0.0:1787739082.436366:0:14678:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb4780 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0001_UUID:14678:1874579665793920:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739082.436393:0:14678:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e8beb733/0x7ee4453e4a1aeb25 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14678 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739082.445472:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.445612:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652553344 00000100:00100000:0.0:1787739082.445621:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652553344 00010000:00010000:0.0:1787739082.445670:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004c07339b/0x7ee4453e4a1ae973 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09550b expref: 12 pid: 14680 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.445716:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1598 00000100:00100000:0.0:1787739082.445733:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba2050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:2406:x1874579652553344:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787739082.445749:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1787739082.445766:0:10340:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x2c0000401:0x9c8:0x0] updating lvb mtime from disk: 0 -> 1787739082 00002000:00010000:0.0:1787739082.445769:0:10340:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x2c0000401:0x9c8:0x0] updating lvb ctime from disk: 0 -> 1787739082 00010000:00010000:0.0:1787739082.445774:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004c07339b/0x7ee4453e4a1ae973 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09550b expref: 12 pid: 14680 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.445789:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004c07339b/0x7ee4453e4a1ae973 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09550b expref: 12 pid: 14680 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.445813:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004c07339b/0x7ee4453e4a1ae973 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09550b expref: 12 pid: 14680 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.445829:0:10340:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x9c8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787739082.445833:0:10340:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x9c8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787739082.445840:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e8beb733/0x7ee4453e4a1aeb25 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 14678 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.445850:0:10340:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e8beb733/0x7ee4453e4a1aeb25 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 14678 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.445891:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739082.445940:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba2050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2406:x1874579652553344:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 209us (476us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.445948:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1598 00010000:00010000:2.0:1787739082.446108:0:14678:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e8beb733/0x7ee4453e4a1aeb25 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14678 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.446118:0:14678:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e8beb733/0x7ee4453e4a1aeb25 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14678 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.446138:0:14678:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e8beb733/0x7ee4453e4a1aeb25 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14678 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.446147:0:14678:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e8beb733/0x7ee4453e4a1aeb25 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14678 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.446155:0:14678:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e8beb733/0x7ee4453e4a1aeb25 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14678 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.446161:0:14678:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840400 already unused, calling callback (ffffffffc0cce440) 00010000:00010000:2.0:1787739082.446165:0:14678:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e8beb733/0x7ee4453e4a1aeb25 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14678 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.446174:0:14678:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e8beb733/0x7ee4453e4a1aeb25 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14678 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.446187:0:14678:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e8beb733/0x7ee4453e4a1aeb25 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14678 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.446196:0:14678:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e8beb733/0x7ee4453e4a1aeb25 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14678 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739082.446277:0:14678:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305656d00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2409:x1874579652553088:12345-192.168.203.15@tcp:1:lfs.0 Request processed in 18570us (18786us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.446286:0:14678:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5195 00000100:00100000:2.0:1787739082.461102:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739082.461195:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652553472 00000100:00100000:1.0:1787739082.461204:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652553472 00000100:00100000:1.0:1787739082.461282:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1432 00000100:00100000:1.0:1787739082.461292:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f78780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:2406:x1874579652553472:12345-192.168.203.15@tcp:103:lfs.0 00010000:00010000:1.0:1787739082.461309:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739082.461334:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1aeaf4 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x186:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09557b expref: 9 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.461353:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x186:0x0].0x0 (ffffa074331c2e00) 00010000:00010000:1.0:1787739082.461363:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739082.461467:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f78780 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2406:x1874579652553472:12345-192.168.203.15@tcp:103:lfs.0 Request processed in 177us (374us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.461475:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1432 00000100:00100000:0.0:1787739082.467670:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.467705:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.467760:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739082.467849:0:6538:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652553984 00000100:00100000:3.0:1787739082.467859:0:6538:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652553984 00000100:00100000:3.0:1787739082.467930:0:6538:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1 00000100:00100000:3.0:1787739082.467941:0:6538:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+20:18317:x1874579652553984:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:3.0:1787739082.467992:0:6538:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739082.468018:0:6538:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088200/0x7ee4453e4a1ae965 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x152d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095504 expref: 20 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.468037:0:6538:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152d:0x0].0x0 (ffffa074126f6000) 00010000:00010000:3.0:1787739082.468041:0:6538:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739082.468062:0:6538:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aeb2c lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.468075:0:6538:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aeb2c lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.468085:0:6538:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aeb2c lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739082.468106:0:6538:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000402:0x186:0x0] 00000100:00100000:3.0:1787739082.468155:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383f840 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665794048:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787739082.468208:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739082.468242:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.468306:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652553600 00000100:00100000:1.0:1787739082.468311:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652553600 00000100:00100000:1.0:1787739082.468347:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652553728 00000100:00100000:1.0:1787739082.468350:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652553728 00000100:00100000:1.0:1787739082.468373:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1433 00000100:00100000:1.0:1787739082.468380:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+20:2406:x1874579652553600:12345-192.168.203.15@tcp:103:lfs.0 00010000:00010000:1.0:1787739082.468390:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739082.468402:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1aeae6 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095574 expref: 20 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.468447:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:1.0:1787739082.468456:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739082.468494:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561c3c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+19:2406:x1874579652553600:12345-192.168.203.15@tcp:103:lfs.0 Request processed in 114us (834us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.468502:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1433 00000100:00100000:1.0:1787739082.468518:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1434 00000100:00100000:1.0:1787739082.468524:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:2406:x1874579652553728:12345-192.168.203.15@tcp:103:lfs.0 00010000:00010000:1.0:1787739082.468532:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739082.468544:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306089200/0x7ee4453e4a1aead8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09556d expref: 8 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.468556:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa0741590db00) 00010000:00010000:1.0:1787739082.468564:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739082.468580:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561cb40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:2406:x1874579652553728:12345-192.168.203.15@tcp:103:lfs.0 Request processed in 56us (875us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.468587:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1434 00000100:00100000:1.0:1787739082.468606:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665794048 00000100:00100000:1.0:1787739082.468610:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665794048 00000100:00100000:1.0:1787739082.468630:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3670 00000100:00100000:1.0:1787739082.468635:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:6538:x1874579665794048:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.468714:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:6538:x1874579665794048:12345-0@lo:1000:mdt_io00_002.0 Request processed in 78us (506us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.468721:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3670 00000100:00100000:1.0:1787739082.468751:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383f840 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665794048:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.469332:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb52c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665794176:0@lo:403:mdt_io00_002.0 00000100:00100000:1.0:1787739082.469376:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.469396:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241000 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.469445:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665794176 00000100:00100000:1.0:1787739082.469450:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665794176 00000100:00100000:1.0:1787739082.469473:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3671 00000100:00100000:1.0:1787739082.469479:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:6538:x1874579665794176:12345-0@lo:403:mdt_io00_002.0 00000100:00100000:1.0:1787739082.471123:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb4780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+9:6538:x1874579665794176:12345-0@lo:403:mdt_io00_002.0 Request processed in 1636us (1739us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.471136:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3671 00000100:00100000:1.0:1787739082.471180:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb52c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665794176:0@lo:403:mdt_io00_002.0 00000100:00100000:1.0:1787739082.474067:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb52c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665794304:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.474133:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.474214:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.474379:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665794304 00000100:00100000:3.0:1787739082.474386:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665794304 00000100:00100000:2.0:1787739082.477753:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3672 00000100:00100000:2.0:1787739082.477776:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:6538:x1874579665794304:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787739082.477916:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb5680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+9:6538:x1874579665794304:12345-0@lo:1000:mdt_io00_002.0 Request processed in 151us (3782us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.477924:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3672 00000100:00100000:0.0:1787739082.478007:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb52c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665794304:0@lo:1000:mdt_io00_002.0 00010000:00010000:0.0:1787739082.478096:0:6538:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1aeb33 lrc: 3/0,1 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.478108:0:6538:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1aeb33 lrc: 3/0,1 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.478118:0:6538:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1aeb33 lrc: 3/0,1 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.478138:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561de00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665794432:0@lo:101:mdt_io00_002.0 00000100:00100000:0.0:1787739082.478176:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.478447:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3600 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.478677:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665794432 00000100:00100000:1.0:1787739082.478684:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665794432 00000100:00100000:1.0:1787739082.478716:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3673 00000100:00100000:1.0:1787739082.478722:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+9:6538:x1874579665794432:12345-0@lo:101:mdt_io00_002.0 00010000:00010000:1.0:1787739082.478733:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739082.478752:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841600/0x7ee4453e4a1aeb3a lrc: 2/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aeb33 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.478778:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841600/0x7ee4453e4a1aeb3a lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aeb33 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.478792:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841600/0x7ee4453e4a1aeb3a lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aeb33 expref: 10 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.478800:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa0730713af00) 00010000:00010000:1.0:1787739082.478803:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.478807:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841600, rc 0) 00000100:00100000:1.0:1787739082.478843:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:6538:x1874579665794432:12345-0@lo:101:mdt_io00_002.0 Request processed in 121us (666us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.478850:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3673 00000100:00100000:0.0:1787739082.479000:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561de00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665794432:0@lo:101:mdt_io00_002.0 00010000:00010000:0.0:1787739082.479014:0:6538:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1aeb33 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aeb3a expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.479024:0:6538:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1aeb33 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aeb3a expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739082.479042:0:6538:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0xcc:0x0] 00010000:00010000:0.0:1787739082.479066:0:6538:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088000/0x7ee4453e4a1aeb41 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xcc:0x0].0xbc0ca3a9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.479072:0:6538:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088000/0x7ee4453e4a1aeb41 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xcc:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.479080:0:6538:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088000/0x7ee4453e4a1aeb41 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xcc:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.479097:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a31a40 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665794560:0@lo:101:mdt_io00_002.0 00000100:00100000:0.0:1787739082.479122:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.479146:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3600 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.479194:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665794560 00000100:00100000:1.0:1787739082.479198:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665794560 00000100:00100000:1.0:1787739082.479231:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3674 00000100:00100000:1.0:1787739082.479238:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+10:6538:x1874579665794560:12345-0@lo:101:mdt_io00_002.0 00010000:00010000:1.0:1787739082.479246:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739082.479257:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840400/0x7ee4453e4a1aeb48 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xcc:0x0].0xbc0ca3a9 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aeb41 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.479273:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840400/0x7ee4453e4a1aeb48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aeb41 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.479283:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840400/0x7ee4453e4a1aeb48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aeb41 expref: 11 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.479290:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0xbc0ca3a9 (ffffa0730713ae00) 00010000:00010000:1.0:1787739082.479293:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.479295:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306840400, rc 0) 00000100:00100000:1.0:1787739082.479331:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+11:6538:x1874579665794560:12345-0@lo:101:mdt_io00_002.0 Request processed in 94us (208us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.479338:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3674 00000100:00100000:0.0:1787739082.479519:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a31a40 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665794560:0@lo:101:mdt_io00_002.0 00010000:00010000:0.0:1787739082.479541:0:6538:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088000/0x7ee4453e4a1aeb41 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0xcc:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aeb48 expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.479550:0:6538:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088000/0x7ee4453e4a1aeb41 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0xcc:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aeb48 expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739082.479562:0:6538:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0xcc:0x0] 00010000:00010000:0.0:1787739082.479577:0:6538:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1aeb4f lrc: 3/0,1 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.479586:0:6538:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1aeb4f lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.479594:0:6538:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1aeb4f lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.479605:0:6538:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1aeb56 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.479612:0:6538:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1aeb56 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.479617:0:6538:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1aeb56 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.479670:0:6538:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1aeb5d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x152d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.479682:0:6538:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aeb17 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x152d:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095582 expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.479688:0:6538:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1aeb5d lrc: 4/0,1 mode: --/EX res: [0x200000402:0x152d:0x0].0x0 bits 0x25/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.479709:0:6538:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aeb17 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x152d:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095582 expref: 18 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.479723:0:6538:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aeb17 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x152d:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095582 expref: 18 pid: 6524 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.479733:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a31a40 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6538:1874579665794688:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739082.479796:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3600 going to sleep for 11 seconds 00010000:00010000:1.0:1787739082.483180:0:6538:0:(ldlm_lockd.c:741:ldlm_handle_ast_error()) ### client (nid 192.168.203.15@tcp) returned -22 from blocking AST (req@00000000b985013b x1874579665794688) - normal race ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aeb17 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x152d:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095582 expref: 18 pid: 6524 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.483204:0:6538:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aeb17 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x152d:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095582 expref: 18 pid: 6524 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.483484:0:6538:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aeb17 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x152d:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095582 expref: 18 pid: 6524 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739082.483501:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a31a40 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6538:1874579665794688:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739082.483523:0:6538:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152d:0x0].0x0 (ffffa074126f6000) 00010000:00010000:1.0:1787739082.483538:0:6538:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1aeb5d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x152d:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.483546:0:6538:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1aeb5d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x152d:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x50290000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.483554:0:6538:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1aeb5d lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x152d:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x50290000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.483576:0:6538:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1aeb5d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x152d:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.483584:0:6538:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1aeb5d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x152d:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.483590:0:6538:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1aeb5d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x152d:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739082.483731:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07409fb52c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665794816:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.483784:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.483819:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739082.483940:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665794816 00000100:00100000:0.0:1787739082.483947:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665794816 00000100:00100000:0.0:1787739082.483985:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3675 00000100:00100000:0.0:1787739082.483994:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042470c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+11:6538:x1874579665794816:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787739082.484103:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042470c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+11:6538:x1874579665794816:12345-0@lo:1000:mdt_io00_002.0 Request processed in 109us (318us total) trans 0 rc -2/-2 00000100:00100000:0.0:1787739082.484110:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3675 00000100:00100000:1.0:1787739082.484759:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07409fb52c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665794816:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.484812:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304245e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665794944:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.484844:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.484869:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.484894:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665794944 00000100:00100000:1.0:1787739082.484898:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665794944 00000100:00100000:1.0:1787739082.484926:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3676 00000100:00100000:1.0:1787739082.484932:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07409fb52c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+11:6538:x1874579665794944:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.484985:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07409fb52c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+11:6538:x1874579665794944:12345-0@lo:1000:mdt_io00_002.0 Request processed in 53us (141us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.484992:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3676 00000100:00100000:1.0:1787739082.485018:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304245e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665794944:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.485109:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304245e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665795072:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.485132:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.485147:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.485159:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665795072 00000100:00100000:1.0:1787739082.485162:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665795072 00000100:00100000:1.0:1787739082.485179:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3677 00000100:00100000:1.0:1787739082.485185:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+11:6538:x1874579665795072:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.485315:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+11:6538:x1874579665795072:12345-0@lo:1000:mdt_io00_002.0 Request processed in 130us (181us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.485323:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3677 00000100:00100000:1.0:1787739082.485347:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304245e00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665795072:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.485379:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce12c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665795200:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.485402:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.485435:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.485446:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665795200 00000100:00100000:1.0:1787739082.485449:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665795200 00000100:00100000:1.0:1787739082.485466:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3678 00000100:00100000:1.0:1787739082.485472:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+11:6538:x1874579665795200:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.485518:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+11:6538:x1874579665795200:12345-0@lo:1000:mdt_io00_002.0 Request processed in 45us (114us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.485593:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3678 00000100:00100000:1.0:1787739082.485620:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce12c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665795200:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.485752:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce12c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665795328:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.485776:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.485789:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241300 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.485801:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665795328 00000100:00100000:1.0:1787739082.485804:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665795328 00000100:00100000:1.0:1787739082.485821:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3679 00000100:00100000:1.0:1787739082.485827:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+11:6538:x1874579665795328:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.485872:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+11:6538:x1874579665795328:12345-0@lo:1000:mdt_io00_002.0 Request processed in 44us (95us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.485889:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3679 00000100:00100000:1.0:1787739082.485913:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce12c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665795328:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.485939:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce12c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665795456:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.485960:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.485973:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0741042a780 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.485983:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665795456 00000100:00100000:1.0:1787739082.485986:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665795456 00000100:00100000:1.0:1787739082.486001:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3680 00000100:00100000:1.0:1787739082.486006:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+11:6538:x1874579665795456:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.486051:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+11:6538:x1874579665795456:12345-0@lo:1000:mdt_io00_002.0 Request processed in 44us (90us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.486058:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3680 00000100:00100000:1.0:1787739082.486079:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce12c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665795456:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.486139:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce12c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665795584:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.486165:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.486182:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0741042a780 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.486193:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665795584 00000100:00100000:1.0:1787739082.486196:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665795584 00000100:00100000:1.0:1787739082.486213:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3681 00000100:00100000:1.0:1787739082.486218:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304245e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6538:x1874579665795584:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.486695:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304245e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:6538:x1874579665795584:12345-0@lo:1000:mdt_io00_002.0 Request processed in 475us (528us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.486703:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3681 00000100:00100000:2.0:1787739082.486943:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce12c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665795584:0@lo:1000:mdt_io00_002.0 00000040:00080000:2.0:1787739082.487777:0:6538:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].405, 1264 off607048 00000040:00080000:2.0:1787739082.487831:0:6538:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].405, 1344 off658632 00000100:00100000:2.0:1787739082.487864:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f78f00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665795712:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787739082.487922:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.488017:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af600 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.488162:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665795712 00000100:00100000:3.0:1787739082.488169:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665795712 00000100:00100000:3.0:1787739082.488363:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3682 00000100:00100000:3.0:1787739082.488374:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f792c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:6538:x1874579665795712:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787739082.488520:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f792c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6538:x1874579665795712:12345-0@lo:1000:mdt_io00_002.0 Request processed in 148us (597us total) trans 4294969879 rc 0/0 00000100:00100000:3.0:1787739082.488538:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3682 00000100:00100000:2.0:1787739082.488611:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f78f00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665795712:0@lo:1000:mdt_io00_002.0 00000004:00080000:2.0:1787739082.488620:0:6538:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306241e40 with version 773 00000004:00080000:0.0:1787739082.488782:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 773 rpc_version 773 00000020:00100000:0.0:1787739082.488807:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:0.0:1787739082.488815:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07303a31a40 x1874579665795968/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 3344/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787739082.488832:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a31a40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665795968:0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739082.488889:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.488973:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.489044:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665795968 00000100:00100000:1.0:1787739082.489049:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665795968 00000100:00100000:1.0:1787739082.489073:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3683 00000100:00100000:1.0:1787739082.489079:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410b0a450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+11:7740:x1874579665795968:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739082.489440:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410b0a450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+12:7740:x1874579665795968:12345-0@lo:1000:osp_up1-0.0 Request processed in 359us (559us total) trans 4294968489 rc 0/0 00000100:00100000:1.0:1787739082.489448:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3683 00000100:00100000:0.0:1787739082.489644:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a31a40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665795968:0@lo:1000:osp_up1-0.0 00000004:00010000:3.0:1787739082.490096:0:6538:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006d146248 reply state 00000000f2ee00f5 transno 4294983519 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1aeb5d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x152d:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.490120:0:6538:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089a00/0x7ee4453e4a1aeb5d lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x152d:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.490128:0:6538:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152d:0x0].0x0 (ffffa074126f6000) 00010000:00010000:3.0:1787739082.490131:0:6538:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 25 00000100:00100000:3.0:1787739082.490135:0:6538:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787739082.490142:0:6538:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1aeb4f lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.490149:0:6538:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1aeb4f lrc: 2/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.490156:0:6538:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1aeb4f lrc: 3/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.490162:0:6538:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306088600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739082.490166:0:6538:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1aeb4f lrc: 3/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.490173:0:6538:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1aeb4f lrc: 4/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.490183:0:6538:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa0741590db00) 00010000:00010000:3.0:1787739082.490185:0:6538:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739082.490188:0:6538:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1aeb4f lrc: 2/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.490193:0:6538:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1aeb4f lrc: 0/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739082.490212:0:6538:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006d146248 reply state 00000000f2ee00f5 transno 4294983519 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1aeb56 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.490320:0:6538:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088800/0x7ee4453e4a1aeb56 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xce:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.490327:0:6538:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0xbc0ca3a9 (ffffa0741590d600) 00010000:00010000:3.0:1787739082.490329:0:6538:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739082.490332:0:6538:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00080000:3.0:1787739082.490338:0:6538:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0xcc:0x0] 00010000:00010000:3.0:1787739082.490346:0:6538:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1aeb33 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aeb3a expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.490355:0:6538:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1aeb33 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aeb3a expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.490362:0:6538:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1aeb33 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aeb3a expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.490438:0:6538:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739082.490444:0:6538:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0xcc:0x0] 00010000:00010000:3.0:1787739082.490450:0:6538:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088000/0x7ee4453e4a1aeb41 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0xcc:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aeb48 expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.490459:0:6538:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088000/0x7ee4453e4a1aeb41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0xbc0ca3a9 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aeb48 expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739082.490471:0:6538:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000402:0x186:0x0] 00010000:00010000:3.0:1787739082.490478:0:6538:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aeb2c lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.490484:0:6538:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aeb2c lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.490491:0:6538:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aeb2c lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.490496:0:6538:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739082.490498:0:6538:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aeb2c lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.490504:0:6538:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aeb2c lrc: 4/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.490512:0:6538:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffffa074331c2d00) 00010000:00010000:3.0:1787739082.490514:0:6538:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739082.490517:0:6538:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aeb2c lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.490523:0:6538:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aeb2c lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739082.490606:0:6538:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561fc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+19:18317:x1874579652553984:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 22665us (22843us total) trans 4294983519 rc 0/0 00000100:00100000:3.0:1787739082.490615:0:6538:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1 00010000:00010000:1.0:1787739082.495712:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739082.495727:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1aeb33 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aeb3a expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.495739:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306088e00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739082.495746:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1aeb33 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aeb3a expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739082.495758:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1aeb33 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aeb3a expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.495799:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1aeb33 lrc: 2/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aeb3a expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.495805:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739082.495840:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1aeb33 lrc: 1/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aeb3a expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.495847:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1aeb33 lrc: 0/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aeb3a expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.495866:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739082.495868:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739082.495870:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787739082.495913:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787739082.495931:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042452c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665796096:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739082.495999:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.496095:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665796096 00000100:00100000:1.0:1787739082.496102:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665796096 00000100:00100000:1.0:1787739082.496154:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1435 00000100:00100000:1.0:1787739082.496162:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+12:3651:x1874579665796096:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739082.496175:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739082.496192:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841600/0x7ee4453e4a1aeb3a lrc: 0/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aeb33 expref: 12 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.496205:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa0730713af00) 00010000:00010000:1.0:1787739082.496213:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739082.496568:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+11:3651:x1874579665796096:12345-0@lo:103:ldlm_bl.0 Request processed in 405us (567us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.496576:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1435 00000100:00100000:0.0:1787739082.496826:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739082.496903:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652554112 00000100:00100000:2.0:1787739082.496909:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652554112 00000100:00100000:2.0:1787739082.496950:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23713 00000100:00100000:2.0:1787739082.496958:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+19:18317:x1874579652554112:12345-192.168.203.15@tcp:34:lfs.0 00010000:00010000:2.0:1787739082.497025:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1aeb64 lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497038:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1aeb64 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497047:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1aeb64 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497059:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1aeb6b lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497065:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1aeb6b lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497071:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1aeb6b lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497125:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1aeb72 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1530:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497134:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1aeb72 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1530:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497141:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1aeb72 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1530:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739082.497189:0:6525:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1aeb72 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1530:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497202:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1aeb64 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497211:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1aeb64 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497217:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1aeb64 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497238:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1aeb64 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497244:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0e00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739082.497248:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1aeb64 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497255:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1aeb64 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497264:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6fd00) 00010000:00010000:2.0:1787739082.497267:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739082.497271:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1aeb64 lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497277:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1aeb64 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497293:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1aeb6b lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497299:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1aeb6b lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497305:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1aeb6b lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497310:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1e00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739082.497312:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1aeb6b lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497318:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1aeb6b lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497809:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0xb62e0cbf (ffffa07403a6f100) 00010000:00010000:2.0:1787739082.497814:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739082.497818:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1aeb6b lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497825:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1e00/0x7ee4453e4a1aeb6b lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497840:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1aeb72 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1530:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497846:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1aeb72 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1530:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497852:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1aeb72 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1530:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497857:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0c00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739082.497860:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1aeb72 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1530:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497867:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1aeb72 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1530:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497888:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1530:0x0].0x0 (ffffa07403eeb600) 00010000:00010000:2.0:1787739082.497890:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739082.497893:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1aeb72 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x1530:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.497899:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1aeb72 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1530:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739082.497966:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a31e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+19:18317:x1874579652554112:12345-192.168.203.15@tcp:34:lfs.0 Request processed in 1008us (1140us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.497974:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23713 00000100:00100000:3.0:1787739082.502744:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042452c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665796096:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739082.502925:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.503012:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652554368 00000100:00100000:0.0:1787739082.503020:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652554368 00000100:00100000:0.0:1787739082.503100:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23714 00000100:00100000:0.0:1787739082.503111:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+19:18317:x1874579652554368:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739082.503128:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739082.503148:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1aeb79 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1530:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095589 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.503266:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aeb80 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1530:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.503282:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aeb80 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1530:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.503294:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aeb80 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1530:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739082.503307:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1530:0x0]: returning size 2 00010000:00010000:0.0:1787739082.503406:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088c00/0x7ee4453e4a1aeb79 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1530:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095589 expref: 22 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.503791:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aeb80 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1530:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095589 expref: 21 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.503800:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1530:0x0].0x0 (ffffa07403eeb600) 00010000:00010000:0.0:1787739082.503803:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1787739082.503806:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306088a00, rc 0) 00000100:00100000:0.0:1787739082.503866:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7a1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:18317:x1874579652554368:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 756us (942us total) trans 4294983520 rc 0/0 00000100:00100000:0.0:1787739082.503873:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23714 00000100:00100000:3.0:1787739082.508728:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739082.508808:0:14683:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652554496 00000100:00100000:1.0:1787739082.508817:0:14683:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652554496 00000100:00100000:1.0:1787739082.508872:0:14683:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5196 00000100:00100000:1.0:1787739082.508891:0:14683:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:2409:x1874579652554496:12345-192.168.203.15@tcp:1:lfs.0 00010000:00010000:1.0:1787739082.508933:0:14683:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8f30e46/0x7ee4453e4a1aeb8e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14683 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.508952:0:14683:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003e81f861/0x7ee4453e4a1ae9f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095535 expref: 9 pid: 14683 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.508961:0:14683:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x9ca:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787739082.508966:0:14683:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9ca:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787739082.508971:0:14683:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8f30e46/0x7ee4453e4a1aeb8e lrc: 4/0,1 mode: --/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14683 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.509003:0:14683:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003e81f861/0x7ee4453e4a1ae9f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095535 expref: 9 pid: 14683 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.509017:0:14683:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003e81f861/0x7ee4453e4a1ae9f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095535 expref: 9 pid: 14683 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739082.509031:0:14683:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304247c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_012:lustre-OST0000_UUID:14683:1874579665796224:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739082.509109:0:14683:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0741042a900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.512942:0:14683:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304247c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_012:lustre-OST0000_UUID:14683:1874579665796224:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739082.512971:0:14683:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b8f30e46/0x7ee4453e4a1aeb8e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14683 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739082.515232:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739082.515322:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652554624 00000100:00100000:2.0:1787739082.515334:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652554624 00002000:00100000:2.0:1787739082.515383:0:8444:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x9ca:0x0] (0->1) req@ffffa0730383c780 x1874579652554624/t0(0) o4->e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2@192.168.203.15@tcp:263/0 lens 496/0 e 0 to 0 dl 1787739093 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787739082.515394:0:8444:0:(ofd_dev.c:2572:ofd_prolong_extent_locks()) Prolong locks for req ffffa0730383c780 with x1874579652554624 ext(0->1) 00010000:00010000:2.0:1787739082.515402:0:8444:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000003e81f861/0x7ee4453e4a1ae9f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095535 expref: 10 pid: 14683 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.515453:0:8444:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000003e81f861/0x7ee4453e4a1ae9f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095535 expref: 10 pid: 14683 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787739082.515462:0:8444:0:(ofd_dev.c:2721:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffffa0730383c780 00000100:00100000:2.0:1787739082.515493:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1 00000100:00100000:2.0:1787739082.515503:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2409:x1874579652554624:12345-192.168.203.15@tcp:4:bash.0 00000100:00100000:2.0:1787739082.515922:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383c780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2409:x1874579652554624:12345-192.168.203.15@tcp:4:bash.0 Request processed in 419us (708us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.515931:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1 00002000:00100000:2.0:1787739082.515938:0:8444:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x9ca:0x0] (0->1) req@ffffa0730383c780 x1874579652554624/t0(0) o4->e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2@192.168.203.15@tcp:263/0 lens 496/448 e 0 to 0 dl 1787739093 ref 1 fl Complete:H/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787739082.515945:0:8444:0:(ofd_dev.c:2572:ofd_prolong_extent_locks()) Prolong locks for req ffffa0730383c780 with x1874579652554624 ext(0->1) 00010000:00010000:2.0:1787739082.515951:0:8444:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000003e81f861/0x7ee4453e4a1ae9f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095535 expref: 11 pid: 14683 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.515963:0:8444:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000003e81f861/0x7ee4453e4a1ae9f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095535 expref: 11 pid: 14683 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787739082.515970:0:8444:0:(ofd_dev.c:2721:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffffa0730383c780 00000100:00100000:1.0:1787739082.520927:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739082.521069:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652554752 00000100:00100000:2.0:1787739082.521078:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652554752 00010000:00010000:2.0:1787739082.521119:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003e81f861/0x7ee4453e4a1ae9f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095535 expref: 11 pid: 14683 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739082.521163:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1599 00000100:00100000:2.0:1787739082.521172:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042470c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2409:x1874579652554752:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739082.521190:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1787739082.521208:0:9525:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x9ca:0x0] updating lvb mtime from disk: 0 -> 1787739082 00002000:00010000:2.0:1787739082.521211:0:9525:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x9ca:0x0] updating lvb ctime from disk: 0 -> 1787739082 00010000:00010000:2.0:1787739082.521380:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003e81f861/0x7ee4453e4a1ae9f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095535 expref: 11 pid: 14683 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.521404:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003e81f861/0x7ee4453e4a1ae9f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095535 expref: 11 pid: 14683 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.521463:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e81f861/0x7ee4453e4a1ae9f1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095535 expref: 11 pid: 14683 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.521481:0:9525:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9ca:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787739082.521485:0:9525:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9ca:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787739082.521490:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b8f30e46/0x7ee4453e4a1aeb8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 14683 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.521500:0:9525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8f30e46/0x7ee4453e4a1aeb8e lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 14683 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.521625:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739082.521729:0:14683:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b8f30e46/0x7ee4453e4a1aeb8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14683 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.521740:0:14683:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8f30e46/0x7ee4453e4a1aeb8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14683 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.521760:0:14683:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8f30e46/0x7ee4453e4a1aeb8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14683 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.521769:0:14683:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8f30e46/0x7ee4453e4a1aeb8e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14683 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.521777:0:14683:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8f30e46/0x7ee4453e4a1aeb8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14683 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.521784:0:14683:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306841600 already unused, calling callback (ffffffffc0cce440) 00010000:00010000:3.0:1787739082.521788:0:14683:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8f30e46/0x7ee4453e4a1aeb8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14683 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.521796:0:14683:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8f30e46/0x7ee4453e4a1aeb8e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14683 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.521808:0:14683:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8f30e46/0x7ee4453e4a1aeb8e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14683 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.521821:0:14683:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8f30e46/0x7ee4453e4a1aeb8e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x9ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14683 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739082.521928:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042470c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2409:x1874579652554752:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 755us (1009us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.521936:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1599 00000100:00100000:2.0:1787739082.524803:0:14683:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383da40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:2409:x1874579652554496:12345-192.168.203.15@tcp:1:lfs.0 Request processed in 15913us (16074us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.524818:0:14683:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5196 00000100:00100000:1.0:1787739082.528917:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.529633:0:6536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652555008 00000100:00100000:0.0:1787739082.529642:0:6536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652555008 00000100:00100000:0.0:1787739082.529766:0:6536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 2 00000100:00100000:0.0:1787739082.529776:0:6536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304245e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:18317:x1874579652555008:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:0.0:1787739082.529829:0:6536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739082.529855:0:6536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1400/0x7ee4453e4a1ae9e3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1530:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09552e expref: 21 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.529875:0:6536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1530:0x0].0x0 (ffffa07403eeb600) 00010000:00010000:0.0:1787739082.529890:0:6536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739082.529915:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1aeb95 lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.529928:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1aeb95 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.529938:0:6536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1aeb95 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739082.529956:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000402:0x186:0x0] 00000100:00100000:0.0:1787739082.530004:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a330c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665796352:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.530058:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.530091:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3780 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.531028:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665796352 00000100:00100000:1.0:1787739082.531036:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665796352 00000100:00100000:1.0:1787739082.531080:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3684 00000100:00100000:1.0:1787739082.531087:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+11:6536:x1874579665796352:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787739082.531188:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+11:6536:x1874579665796352:12345-0@lo:1000:mdt_io00_000.0 Request processed in 102us (1130us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.531196:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3684 00000100:00100000:0.0:1787739082.531450:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a330c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665796352:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.533738:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a330c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665796480:0@lo:403:mdt_io00_000.0 00000100:00100000:0.0:1787739082.533810:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.533912:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.534729:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665796480 00000100:00100000:2.0:1787739082.534739:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665796480 00000100:00100000:2.0:1787739082.534801:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3685 00000100:00100000:2.0:1787739082.534811:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+11:6536:x1874579665796480:12345-0@lo:403:mdt_io00_000.0 00000100:00100000:2.0:1787739082.539120:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+11:6536:x1874579665796480:12345-0@lo:403:mdt_io00_000.0 Request processed in 4302us (5301us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.539481:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a330c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665796480:0@lo:403:mdt_io00_000.0 00010000:00010000:0.0:1787739082.540153:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089e00/0x7ee4453e4a1aeb9c lrc: 3/0,1 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.540164:0:6536:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089e00/0x7ee4453e4a1aeb9c lrc: 3/0,1 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.540173:0:6536:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089e00/0x7ee4453e4a1aeb9c lrc: 3/0,1 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.540193:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a330c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665796608:0@lo:101:mdt_io00_000.0 00000100:00100000:0.0:1787739082.540233:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.540267:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.540306:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665796608 00000100:00100000:3.0:1787739082.540313:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665796608 00000100:00100000:3.0:1787739082.540365:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3686 00000100:00100000:3.0:1787739082.540375:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+12:6536:x1874579665796608:12345-0@lo:101:mdt_io00_000.0 00010000:00010000:3.0:1787739082.540390:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739082.540407:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aeba3 lrc: 2/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aeb9c expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.540459:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aeba3 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aeb9c expref: 13 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.540473:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aeba3 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aeb9c expref: 13 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.540481:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa074331c2d00) 00010000:00010000:3.0:1787739082.540484:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739082.540488:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069abe00, rc 0) 00000100:00100000:3.0:1787739082.540717:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+13:6536:x1874579665796608:12345-0@lo:101:mdt_io00_000.0 Request processed in 344us (483us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.540724:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3686 00000100:00100000:2.0:1787739082.542537:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3685 00000100:00100000:0.0:1787739082.542745:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a330c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665796608:0@lo:101:mdt_io00_000.0 00010000:00010000:0.0:1787739082.542769:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089e00/0x7ee4453e4a1aeb9c lrc: 4/0,1 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aeba3 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.542785:0:6536:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089e00/0x7ee4453e4a1aeb9c lrc: 4/0,1 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aeba3 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739082.542805:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0xcc:0x0] 00010000:00010000:0.0:1787739082.542844:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088c00/0x7ee4453e4a1aebaa lrc: 3/0,1 mode: --/PW res: [0x240000400:0xcc:0x0].0xb62e0cbf bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.542853:0:6536:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088c00/0x7ee4453e4a1aebaa lrc: 3/0,1 mode: --/PW res: [0x240000400:0xcc:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.542861:0:6536:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088c00/0x7ee4453e4a1aebaa lrc: 3/0,1 mode: --/PW res: [0x240000400:0xcc:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.542890:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a330c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665796736:0@lo:101:mdt_io00_000.0 00000100:00100000:0.0:1787739082.542982:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.543013:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.543158:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665796736 00000100:00100000:1.0:1787739082.543165:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665796736 00000100:00100000:1.0:1787739082.543212:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3687 00000100:00100000:1.0:1787739082.543221:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a321c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+12:6536:x1874579665796736:12345-0@lo:101:mdt_io00_000.0 00010000:00010000:1.0:1787739082.543236:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739082.543255:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841c00/0x7ee4453e4a1aebb1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xcc:0x0].0xb62e0cbf bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aebaa expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.543275:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841c00/0x7ee4453e4a1aebb1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aebaa expref: 13 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.543287:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841c00/0x7ee4453e4a1aebb1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aebaa expref: 13 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.543294:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0xb62e0cbf (ffffa0730713af00) 00010000:00010000:1.0:1787739082.543297:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.543300:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841c00, rc 0) 00000100:00100000:1.0:1787739082.543356:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a321c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+13:6536:x1874579665796736:12345-0@lo:101:mdt_io00_000.0 Request processed in 136us (374us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.543361:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3687 00000100:00100000:0.0:1787739082.543588:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a330c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665796736:0@lo:101:mdt_io00_000.0 00010000:00010000:0.0:1787739082.543600:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088c00/0x7ee4453e4a1aebaa lrc: 4/0,1 mode: PW/PW res: [0x240000400:0xcc:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aebb1 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.543609:0:6536:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088c00/0x7ee4453e4a1aebaa lrc: 4/0,1 mode: PW/PW res: [0x240000400:0xcc:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aebb1 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739082.543623:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0xcc:0x0] 00010000:00010000:0.0:1787739082.543639:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1aebb8 lrc: 3/0,1 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.543647:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1aebb8 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.543655:0:6536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1aebb8 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.543666:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1aebbf lrc: 3/0,1 mode: --/PW res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.543677:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1aebbf lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.543683:0:6536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1aebbf lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.543722:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088200/0x7ee4453e4a1aebc6 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1530:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.543733:0:6536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aeb80 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1530:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095589 expref: 20 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.543741:0:6536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088200/0x7ee4453e4a1aebc6 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x1530:0x0].0x0 bits 0x25/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.543762:0:6536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aeb80 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1530:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095589 expref: 20 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.543776:0:6536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aeb80 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1530:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095589 expref: 20 pid: 6523 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.543786:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a330c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000_UUID:6536:1874579665796864:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739082.543852:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3f00 going to sleep for 11 seconds 00010000:00010000:2.0:1787739082.549116:0:6536:0:(ldlm_lockd.c:741:ldlm_handle_ast_error()) ### client (nid 192.168.203.15@tcp) returned -22 from blocking AST (req@0000000055f421d3 x1874579665796864) - normal race ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aeb80 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1530:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095589 expref: 20 pid: 6523 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.549143:0:6536:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aeb80 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1530:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095589 expref: 20 pid: 6523 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.549169:0:6536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aeb80 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1530:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095589 expref: 20 pid: 6523 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739082.549187:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a330c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000_UUID:6536:1874579665796864:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739082.549211:0:6536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1530:0x0].0x0 (ffffa07403eeb600) 00010000:00010000:2.0:1787739082.549334:0:6536:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088200/0x7ee4453e4a1aebc6 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1530:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.549347:0:6536:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088200/0x7ee4453e4a1aebc6 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1530:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x50290000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.549356:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088200/0x7ee4453e4a1aebc6 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1530:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x50290000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.549378:0:6536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088200/0x7ee4453e4a1aebc6 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1530:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.549387:0:6536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088200/0x7ee4453e4a1aebc6 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1530:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.549397:0:6536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088200/0x7ee4453e4a1aebc6 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1530:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739082.549675:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665796992:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.549723:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.549756:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.550018:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665796992 00000100:00100000:3.0:1787739082.550025:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665796992 00000100:00100000:3.0:1787739082.550079:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3688 00000100:00100000:3.0:1787739082.550089:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f79a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+13:6536:x1874579665796992:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.550210:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f79a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+13:6536:x1874579665796992:12345-0@lo:1000:mdt_io00_000.0 Request processed in 122us (485us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739082.550229:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3688 00000100:00100000:2.0:1787739082.550796:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665796992:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.550848:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665797120:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.550877:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.550914:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.551034:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665797120 00000100:00100000:3.0:1787739082.551039:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665797120 00000100:00100000:3.0:1787739082.551074:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3689 00000100:00100000:3.0:1787739082.551081:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f783c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+13:6536:x1874579665797120:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.551147:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f783c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+13:6536:x1874579665797120:12345-0@lo:1000:mdt_io00_000.0 Request processed in 66us (269us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.551154:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3689 00000100:00100000:1.0:1787739082.551231:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665797120:0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787739082.551330:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304247c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665797248:0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787739082.551357:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.551383:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0741042a300 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.551493:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665797248 00000100:00100000:3.0:1787739082.551498:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665797248 00000100:00100000:3.0:1787739082.551540:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3690 00000100:00100000:3.0:1787739082.551547:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073042452c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+13:6536:x1874579665797248:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.551603:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073042452c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+13:6536:x1874579665797248:12345-0@lo:1000:mdt_io00_000.0 Request processed in 55us (244us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.551609:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3690 00000100:00100000:1.0:1787739082.551984:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304247c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665797248:0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787739082.552018:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304247c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665797376:0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787739082.552042:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.552054:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0741042a300 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.552105:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665797376 00000100:00100000:3.0:1787739082.552109:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665797376 00000100:00100000:3.0:1787739082.552139:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3691 00000100:00100000:3.0:1787739082.552146:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305656940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+13:6536:x1874579665797376:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.552198:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305656940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+13:6536:x1874579665797376:12345-0@lo:1000:mdt_io00_000.0 Request processed in 53us (155us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.552204:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3691 00000100:00100000:1.0:1787739082.552316:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304247c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665797376:0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787739082.552395:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304247c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665797504:0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787739082.552444:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.552466:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa0741042af00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.552672:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665797504 00000100:00100000:2.0:1787739082.552677:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665797504 00000100:00100000:2.0:1787739082.552704:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3692 00000100:00100000:2.0:1787739082.552710:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f743c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6536:x1874579665797504:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.552768:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f743c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6536:x1874579665797504:12345-0@lo:1000:mdt_io00_000.0 Request processed in 58us (324us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.552775:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3692 00000100:00100000:1.0:1787739082.552905:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304247c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665797504:0@lo:1000:mdt_io00_000.0 00000040:00080000:1.0:1787739082.553278:0:6536:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].406, 1264 off608312 00000040:00080000:1.0:1787739082.553326:0:6536:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].406, 1344 off659976 00000100:00100000:1.0:1787739082.553351:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304247c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665797632:0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787739082.553380:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.553405:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803780 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.553689:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665797632 00000100:00100000:2.0:1787739082.553694:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665797632 00000100:00100000:2.0:1787739082.553726:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3693 00000100:00100000:2.0:1787739082.553733:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c417840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:6536:x1874579665797632:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.553824:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c417840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6536:x1874579665797632:12345-0@lo:1000:mdt_io00_000.0 Request processed in 91us (443us total) trans 4294969881 rc 0/0 00000100:00100000:2.0:1787739082.553831:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3693 00000100:00100000:1.0:1787739082.553998:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304247c00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665797632:0@lo:1000:mdt_io00_000.0 00000004:00080000:1.0:1787739082.554007:0:6536:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa0741042a6c0 with version 774 00000004:00080000:2.0:1787739082.554169:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 774 rpc_version 774 00000020:00100000:2.0:1787739082.554190:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739082.554198:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07305f7a580 x1874579665797888/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 3088/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739082.554311:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7a580 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665797888:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739082.554345:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.554368:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.554435:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665797888 00000100:00100000:3.0:1787739082.554441:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665797888 00000100:00100000:3.0:1787739082.554468:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3694 00000100:00100000:3.0:1787739082.554475:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f78b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+13:7740:x1874579665797888:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739082.555107:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f78b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+14:7740:x1874579665797888:12345-0@lo:1000:osp_up1-0.0 Request processed in 631us (759us total) trans 4294968490 rc 0/0 00000100:00100000:3.0:1787739082.555116:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3694 00000100:00100000:2.0:1787739082.555303:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7a580 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665797888:0@lo:1000:osp_up1-0.0 00000004:00010000:0.0:1787739082.555450:0:6536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000301dbfab reply state 00000000dea58f05 transno 4294983521 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088200/0x7ee4453e4a1aebc6 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1530:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.555470:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088200/0x7ee4453e4a1aebc6 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1530:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.555484:0:6536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1530:0x0].0x0 (ffffa07403eeb600) 00010000:00010000:0.0:1787739082.555487:0:6536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 25 00000100:00100000:0.0:1787739082.555491:0:6536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1787739082.555498:0:6536:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1aebb8 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.555506:0:6536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1aebb8 lrc: 4/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.555511:0:6536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1aebb8 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.555518:0:6536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1aebb8 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.555629:0:6536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306088e00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739082.555635:0:6536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1aebb8 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.555644:0:6536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1aebb8 lrc: 5/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.555716:0:6536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa0741590dc00) 00010000:00010000:0.0:1787739082.555719:0:6536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739082.555724:0:6536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1aebb8 lrc: 3/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.555730:0:6536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088e00/0x7ee4453e4a1aebb8 lrc: 0/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739082.555750:0:6536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000301dbfab reply state 00000000dea58f05 transno 4294983521 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1aebbf lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.555759:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088600/0x7ee4453e4a1aebbf lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xce:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.555765:0:6536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0xb62e0cbf (ffffa0741590de00) 00010000:00010000:0.0:1787739082.555768:0:6536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1787739082.555770:0:6536:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00080000:0.0:1787739082.555775:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0xcc:0x0] 00010000:00010000:0.0:1787739082.555782:0:6536:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089e00/0x7ee4453e4a1aeb9c lrc: 4/0,1 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aeba3 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.555791:0:6536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089e00/0x7ee4453e4a1aeb9c lrc: 4/0,1 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aeba3 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.555798:0:6536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089e00/0x7ee4453e4a1aeb9c lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aeba3 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.555818:0:6536:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:0.0:1787739082.555822:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0xcc:0x0] 00010000:00010000:0.0:1787739082.555827:0:6536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088c00/0x7ee4453e4a1aebaa lrc: 4/0,1 mode: PW/PW res: [0x240000400:0xcc:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aebb1 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.555834:0:6536:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088c00/0x7ee4453e4a1aebaa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0xb62e0cbf bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aebb1 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739082.555846:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000402:0x186:0x0] 00010000:00010000:0.0:1787739082.555853:0:6536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1aeb95 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.555859:0:6536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1aeb95 lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.555865:0:6536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1aeb95 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.555870:0:6536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306089400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739082.555873:0:6536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1aeb95 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.556011:0:6536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1aeb95 lrc: 4/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.556021:0:6536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffffa0741590d200) 00010000:00010000:0.0:1787739082.556023:0:6536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739082.556026:0:6536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1aeb95 lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.556031:0:6536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1aeb95 lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.556174:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739082.556180:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089e00/0x7ee4453e4a1aeb9c lrc: 2/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aeba3 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.556187:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306089e00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739082.556192:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089e00/0x7ee4453e4a1aeb9c lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aeba3 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739082.556199:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089e00/0x7ee4453e4a1aeb9c lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aeba3 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.556833:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089e00/0x7ee4453e4a1aeb9c lrc: 2/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aeba3 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.556843:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739082.556864:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089e00/0x7ee4453e4a1aeb9c lrc: 1/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aeba3 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.556871:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306089e00/0x7ee4453e4a1aeb9c lrc: 0/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aeba3 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.556975:0:6536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304245e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:18317:x1874579652555008:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 27197us (28063us total) trans 4294983521 rc 0/0 00000100:00100000:0.0:1787739082.556984:0:6536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 2 00010000:00010000:2.0:1787739082.560410:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739082.560426:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739082.560428:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739082.560446:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787739082.560467:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665798016:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739082.561174:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739082.563089:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665798016 00000100:00100000:3.0:1787739082.563103:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665798016 00000100:00100000:3.0:1787739082.563176:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1436 00000100:00100000:3.0:1787739082.563185:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7a940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+14:3650:x1874579665798016:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739082.563201:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739082.563236:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aeba3 lrc: 0/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aeb9c expref: 14 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.563253:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa074331c2d00) 00010000:00010000:3.0:1787739082.563263:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1787739082.563341:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7a940 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+13:3650:x1874579665798016:12345-0@lo:103:ldlm_bl.0 Request processed in 157us (2168us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.563349:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1436 00000100:00100000:2.0:1787739082.564918:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665798016:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739082.566112:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739082.567008:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652555136 00000100:00100000:3.0:1787739082.567018:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652555136 00000100:00100000:3.0:1787739082.567073:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23715 00000100:00100000:3.0:1787739082.567082:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c417c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:18317:x1874579652555136:12345-192.168.203.15@tcp:34:lfs.0 00010000:00010000:3.0:1787739082.567152:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aebcd lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567168:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aebcd lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567183:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aebcd lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567196:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aebd4 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0x391d5834 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567203:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aebd4 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567210:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aebd4 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567277:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aebdb lrc: 3/1,0 mode: --/PR res: [0x200000402:0x152e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567286:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aebdb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567293:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aebdb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739082.567340:0:6524:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aebdb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567353:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aebcd lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567360:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aebcd lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567366:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aebcd lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567377:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aebcd lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567383:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aaa00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739082.567387:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aebcd lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567395:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aebcd lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567406:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa074331c2800) 00010000:00010000:3.0:1787739082.567410:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739082.567440:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aebcd lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567447:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aaa00/0x7ee4453e4a1aebcd lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567463:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aebd4 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567470:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aebd4 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567475:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aebd4 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567481:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739082.567484:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aebd4 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567494:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aebd4 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567502:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x391d5834 (ffffa074331c2700) 00010000:00010000:3.0:1787739082.567504:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739082.567507:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aebd4 lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567512:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa200/0x7ee4453e4a1aebd4 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567523:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aebdb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567757:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aebdb lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x152e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567763:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aebdb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x152e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567768:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739082.567772:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aebdb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x152e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567777:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aebdb lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x152e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567785:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152e:0x0].0x0 (ffffa074126f6300) 00010000:00010000:3.0:1787739082.567788:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739082.567791:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aebdb lrc: 2/0,0 mode: --/PR res: [0x200000402:0x152e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.567797:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aebdb lrc: 0/0,0 mode: --/PR res: [0x200000402:0x152e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739082.568005:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c417c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:18317:x1874579652555136:12345-192.168.203.15@tcp:34:lfs.0 Request processed in 923us (1896us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.568014:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23715 00000100:00100000:3.0:1787739082.573832:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739082.574358:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652555392 00000100:00100000:1.0:1787739082.574368:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652555392 00000100:00100000:3.0:1787739082.574770:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23716 00000100:00100000:3.0:1787739082.574782:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+21:18317:x1874579652555392:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:3.0:1787739082.574802:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739082.574822:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aebe2 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x152e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095590 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.574939:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aebe9 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x152e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.574953:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aebe9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x152e:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.574964:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aebe9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x152e:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739082.574976:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x152e:0x0]: returning size 2 00010000:00010000:3.0:1787739082.575072:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aebe2 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x152e:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095590 expref: 24 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.575089:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aebe9 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x152e:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095590 expref: 23 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.575102:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152e:0x0].0x0 (ffffa074126f6300) 00010000:00010000:3.0:1787739082.575104:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:3.0:1787739082.575107:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aac00, rc 0) 00000100:00100000:3.0:1787739082.575349:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383de00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+23:18317:x1874579652555392:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 569us (1525us total) trans 4294983522 rc 0/0 00000100:00100000:3.0:1787739082.575358:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23716 00000100:00100000:1.0:1787739082.580976:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.581112:0:8438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652555520 00000100:00100000:0.0:1787739082.581123:0:8438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652555520 00000100:00100000:0.0:1787739082.581179:0:8438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5197 00000100:00100000:0.0:1787739082.581189:0:8438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c414000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:2406:x1874579652555520:12345-192.168.203.15@tcp:1:lfs.0 00010000:00010000:0.0:1787739082.581245:0:8438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c03d130/0x7ee4453e4a1aebf7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.581265:0:8438:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000cf6e65f9/0x7ee4453e4a1ae99d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095519 expref: 9 pid: 14660 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.581275:0:8438:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x9c9:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787739082.581279:0:8438:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787739082.581283:0:8438:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c03d130/0x7ee4453e4a1aebf7 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.581317:0:8438:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000cf6e65f9/0x7ee4453e4a1ae99d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095519 expref: 9 pid: 14660 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.581329:0:8438:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000cf6e65f9/0x7ee4453e4a1ae99d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095519 expref: 9 pid: 14660 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.581344:0:8438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a30f00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8438:1874579665798144:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739082.581454:0:8438:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3540 going to sleep for 11 seconds 00000100:00100000:0.0:1787739082.586493:0:8438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a30f00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8438:1874579665798144:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739082.586525:0:8438:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009c03d130/0x7ee4453e4a1aebf7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739082.589316:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739082.589862:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652555648 00000100:00100000:2.0:1787739082.589872:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652555648 00002000:00100000:2.0:1787739082.589926:0:8442:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x9c9:0x0] (0->1) req@ffffa0743c4143c0 x1874579652555648/t0(0) o4->e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2@192.168.203.15@tcp:263/0 lens 496/0 e 0 to 0 dl 1787739093 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787739082.589935:0:8442:0:(ofd_dev.c:2572:ofd_prolong_extent_locks()) Prolong locks for req ffffa0743c4143c0 with x1874579652555648 ext(0->1) 00010000:00010000:2.0:1787739082.589943:0:8442:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000cf6e65f9/0x7ee4453e4a1ae99d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095519 expref: 10 pid: 14660 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.589960:0:8442:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000cf6e65f9/0x7ee4453e4a1ae99d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095519 expref: 10 pid: 14660 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787739082.589968:0:8442:0:(ofd_dev.c:2721:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffffa0743c4143c0 00000100:00100000:2.0:1787739082.590002:0:8442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 2 00000100:00100000:2.0:1787739082.590011:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c4143c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652555648:12345-192.168.203.15@tcp:4:bash.0 00000100:00100000:2.0:1787739082.590331:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c4143c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2408:x1874579652555648:12345-192.168.203.15@tcp:4:bash.0 Request processed in 320us (1021us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.590340:0:8442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 2 00002000:00100000:2.0:1787739082.590347:0:8442:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x9c9:0x0] (0->1) req@ffffa0743c4143c0 x1874579652555648/t0(0) o4->e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2@192.168.203.15@tcp:263/0 lens 496/448 e 0 to 0 dl 1787739093 ref 1 fl Complete:H/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1787739082.590355:0:8442:0:(ofd_dev.c:2572:ofd_prolong_extent_locks()) Prolong locks for req ffffa0743c4143c0 with x1874579652555648 ext(0->1) 00010000:00010000:2.0:1787739082.590360:0:8442:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000cf6e65f9/0x7ee4453e4a1ae99d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095519 expref: 11 pid: 14660 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.590371:0:8442:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000cf6e65f9/0x7ee4453e4a1ae99d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095519 expref: 11 pid: 14660 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1787739082.590378:0:8442:0:(ofd_dev.c:2721:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffffa0743c4143c0 00000100:00100000:2.0:1787739082.596345:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739082.596845:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652555776 00000100:00100000:1.0:1787739082.596855:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652555776 00010000:00010000:1.0:1787739082.596904:0:6508:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000cf6e65f9/0x7ee4453e4a1ae99d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095519 expref: 11 pid: 14660 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739082.596946:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1600 00000100:00100000:1.0:1787739082.596956:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba3050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2409:x1874579652555776:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739082.596971:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1787739082.596987:0:6508:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x9c9:0x0] updating lvb mtime from disk: 0 -> 1787739082 00002000:00010000:1.0:1787739082.596990:0:6508:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x9c9:0x0] updating lvb ctime from disk: 0 -> 1787739082 00010000:00010000:1.0:1787739082.597007:0:6508:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000cf6e65f9/0x7ee4453e4a1ae99d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095519 expref: 11 pid: 14660 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.597022:0:6508:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000cf6e65f9/0x7ee4453e4a1ae99d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095519 expref: 11 pid: 14660 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.597045:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf6e65f9/0x7ee4453e4a1ae99d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095519 expref: 11 pid: 14660 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.597062:0:6508:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787739082.597066:0:6508:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787739082.597073:0:6508:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009c03d130/0x7ee4453e4a1aebf7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.597083:0:6508:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c03d130/0x7ee4453e4a1aebf7 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.597124:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739082.597182:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba3050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2409:x1874579652555776:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 227us (857us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.597190:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1600 00010000:00010000:2.0:1787739082.598785:0:8438:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009c03d130/0x7ee4453e4a1aebf7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.598802:0:8438:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c03d130/0x7ee4453e4a1aebf7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.598829:0:8438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c03d130/0x7ee4453e4a1aebf7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.598837:0:8438:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c03d130/0x7ee4453e4a1aebf7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.598847:0:8438:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c03d130/0x7ee4453e4a1aebf7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.598859:0:8438:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306089400 already unused, calling callback (ffffffffc0cce440) 00010000:00010000:2.0:1787739082.598862:0:8438:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c03d130/0x7ee4453e4a1aebf7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.598871:0:8438:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c03d130/0x7ee4453e4a1aebf7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.598895:0:8438:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c03d130/0x7ee4453e4a1aebf7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.598903:0:8438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c03d130/0x7ee4453e4a1aebf7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x9c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739082.598990:0:8438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c414000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:2406:x1874579652555520:12345-192.168.203.15@tcp:1:lfs.0 Request processed in 17800us (18021us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.599000:0:8438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5197 00000100:00100000:2.0:1787739082.607307:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739082.607457:0:6538:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652556032 00000100:00100000:3.0:1787739082.607467:0:6538:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652556032 00000100:00100000:3.0:1787739082.607522:0:6538:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 3 00000100:00100000:3.0:1787739082.607545:0:6538:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+23:18317:x1874579652556032:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:3.0:1787739082.607595:0:6538:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1787739082.607620:0:6538:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1ae98f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x152e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095512 expref: 23 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.607639:0:6538:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152e:0x0].0x0 (ffffa074126f6300) 00010000:00010000:3.0:1787739082.607644:0:6538:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1787739082.607678:0:6538:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aebfe lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.607691:0:6538:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aebfe lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.607701:0:6538:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aebfe lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739082.607718:0:6538:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000402:0x186:0x0] 00000100:00100000:3.0:1787739082.607771:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665798272:0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787739082.607822:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739082.607859:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.608226:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665798272 00000100:00100000:1.0:1787739082.608231:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665798272 00000100:00100000:1.0:1787739082.608276:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3695 00000100:00100000:1.0:1787739082.608284:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+13:6538:x1874579665798272:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.608391:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383e580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+13:6538:x1874579665798272:12345-0@lo:1000:mdt_io00_002.0 Request processed in 107us (567us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.608398:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3695 00000100:00100000:2.0:1787739082.608680:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383c000 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665798272:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787739082.609276:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7b0c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665798400:0@lo:403:mdt_io00_002.0 00000100:00100000:2.0:1787739082.609322:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.609358:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af240 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.609952:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665798400 00000100:00100000:1.0:1787739082.609958:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665798400 00000100:00100000:1.0:1787739082.610004:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3696 00000100:00100000:1.0:1787739082.610012:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f78780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+13:6538:x1874579665798400:12345-0@lo:403:mdt_io00_002.0 00000100:00100000:1.0:1787739082.615306:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f78780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+13:6538:x1874579665798400:12345-0@lo:403:mdt_io00_002.0 Request processed in 5286us (5974us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.615320:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3696 00000100:00100000:2.0:1787739082.615979:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7b0c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665798400:0@lo:403:mdt_io00_002.0 00010000:00010000:2.0:1787739082.617927:0:6538:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1aec05 lrc: 3/0,1 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.617943:0:6538:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1aec05 lrc: 3/0,1 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.617954:0:6538:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1aec05 lrc: 3/0,1 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739082.617976:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7b0c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665798528:0@lo:101:mdt_io00_002.0 00000100:00100000:2.0:1787739082.618035:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.618100:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af240 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.618150:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665798528 00000100:00100000:3.0:1787739082.618158:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665798528 00000100:00100000:3.0:1787739082.618303:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3697 00000100:00100000:3.0:1787739082.618313:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+13:6538:x1874579665798528:12345-0@lo:101:mdt_io00_002.0 00010000:00010000:3.0:1787739082.618340:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739082.618358:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aec0c lrc: 2/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aec05 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.618383:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aec0c lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aec05 expref: 14 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.618395:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aec0c lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aec05 expref: 14 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.618404:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa074331c2800) 00010000:00010000:3.0:1787739082.618407:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739082.618437:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab400, rc 0) 00000100:00100000:3.0:1787739082.618500:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+14:6538:x1874579665798528:12345-0@lo:101:mdt_io00_002.0 Request processed in 187us (463us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.618507:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3697 00000100:00100000:0.0:1787739082.618586:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7b0c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665798528:0@lo:101:mdt_io00_002.0 00010000:00010000:0.0:1787739082.618602:0:6538:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1aec05 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aec0c expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.618613:0:6538:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1aec05 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aec0c expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739082.618637:0:6538:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0xcc:0x0] 00010000:00010000:0.0:1787739082.618908:0:6538:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1aec13 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xcc:0x0].0x391d5834 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.618916:0:6538:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1aec13 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xcc:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.618923:0:6538:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1aec13 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xcc:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.618935:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a30f00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665798656:0@lo:101:mdt_io00_002.0 00000100:00100000:0.0:1787739082.618964:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.619841:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3540 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.619948:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665798656 00000100:00100000:2.0:1787739082.619956:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665798656 00000100:00100000:2.0:1787739082.619995:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3698 00000100:00100000:2.0:1787739082.620003:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a330c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+14:6538:x1874579665798656:12345-0@lo:101:mdt_io00_002.0 00010000:00010000:2.0:1787739082.620018:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739082.620036:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1aec1a lrc: 2/0,0 mode: --/PW res: [0x240000400:0xcc:0x0].0x391d5834 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aec13 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.620059:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1aec1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aec13 expref: 15 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.620072:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0e00/0x7ee4453e4a1aec1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aec13 expref: 15 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.620079:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x391d5834 (ffffa07403a6fd00) 00010000:00010000:2.0:1787739082.620082:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739082.620085:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa0e00, rc 0) 00000100:00100000:2.0:1787739082.620136:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a330c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+15:6538:x1874579665798656:12345-0@lo:101:mdt_io00_002.0 Request processed in 134us (1171us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.620143:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3698 00000100:00100000:0.0:1787739082.620900:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a30f00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665798656:0@lo:101:mdt_io00_002.0 00010000:00010000:0.0:1787739082.620921:0:6538:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1aec13 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0xcc:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aec1a expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.620936:0:6538:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1aec13 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0xcc:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aec1a expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739082.620955:0:6538:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0xcc:0x0] 00010000:00010000:0.0:1787739082.620976:0:6538:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089200/0x7ee4453e4a1aec21 lrc: 3/0,1 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.620984:0:6538:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089200/0x7ee4453e4a1aec21 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.620993:0:6538:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089200/0x7ee4453e4a1aec21 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.621010:0:6538:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1aec28 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xce:0x0].0x391d5834 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.621017:0:6538:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1aec28 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.621023:0:6538:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1aec28 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.621064:0:6538:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1aec2f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x152e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.621076:0:6538:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aebe9 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x152e:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095590 expref: 22 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.621084:0:6538:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1aec2f lrc: 4/0,1 mode: --/EX res: [0x200000402:0x152e:0x0].0x0 bits 0x25/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.621113:0:6538:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aebe9 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x152e:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095590 expref: 22 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.621127:0:6538:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aebe9 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x152e:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095590 expref: 22 pid: 13793 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.621140:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a30f00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6538:1874579665798784:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739082.621333:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3540 going to sleep for 11 seconds 00010000:00010000:2.0:1787739082.624780:0:6538:0:(ldlm_lockd.c:741:ldlm_handle_ast_error()) ### client (nid 192.168.203.15@tcp) returned -22 from blocking AST (req@00000000ab1723f4 x1874579665798784) - normal race ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aebe9 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x152e:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095590 expref: 22 pid: 13793 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.624807:0:6538:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aebe9 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x152e:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095590 expref: 22 pid: 13793 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.624833:0:6538:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aebe9 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x152e:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095590 expref: 22 pid: 13793 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739082.624849:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a30f00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6538:1874579665798784:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739082.624868:0:6538:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152e:0x0].0x0 (ffffa074126f6300) 00010000:00010000:2.0:1787739082.624873:0:6538:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1aec2f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x152e:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.624892:0:6538:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1aec2f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x152e:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x50290000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.624900:0:6538:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1aec2f lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x152e:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x50290000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.624923:0:6538:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1aec2f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x152e:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.624932:0:6538:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1aec2f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x152e:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.624938:0:6538:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1aec2f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x152e:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739082.625071:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665798912:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787739082.625119:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.625191:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074052af240 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.625348:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665798912 00000100:00100000:1.0:1787739082.625355:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665798912 00000100:00100000:3.0:1787739082.625434:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3699 00000100:00100000:3.0:1787739082.625445:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+15:6538:x1874579665798912:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787739082.625835:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+15:6538:x1874579665798912:12345-0@lo:1000:mdt_io00_002.0 Request processed in 392us (714us total) trans 0 rc -2/-2 00000100:00100000:3.0:1787739082.625843:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3699 00000100:00100000:2.0:1787739082.625922:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665798912:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787739082.625966:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665799040:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787739082.625991:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.626053:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.626151:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665799040 00000100:00100000:3.0:1787739082.626155:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665799040 00000100:00100000:3.0:1787739082.626178:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3700 00000100:00100000:3.0:1787739082.626184:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f792c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+15:6538:x1874579665799040:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787739082.626365:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f792c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+15:6538:x1874579665799040:12345-0@lo:1000:mdt_io00_002.0 Request processed in 180us (371us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.626373:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3700 00000100:00100000:2.0:1787739082.626446:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665799040:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787739082.626546:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665799168:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787739082.626571:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.626592:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.626785:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665799168 00000100:00100000:3.0:1787739082.626790:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665799168 00000100:00100000:3.0:1787739082.626809:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3701 00000100:00100000:3.0:1787739082.626814:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+15:6538:x1874579665799168:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787739082.626874:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+15:6538:x1874579665799168:12345-0@lo:1000:mdt_io00_002.0 Request processed in 59us (302us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.626890:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3701 00000100:00100000:2.0:1787739082.626946:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665799168:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787739082.626978:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665799296:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787739082.627001:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.627020:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.627063:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665799296 00000100:00100000:3.0:1787739082.627078:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665799296 00000100:00100000:3.0:1787739082.627095:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3702 00000100:00100000:3.0:1787739082.627101:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+15:6538:x1874579665799296:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787739082.627150:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+15:6538:x1874579665799296:12345-0@lo:1000:mdt_io00_002.0 Request processed in 49us (148us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.627157:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3702 00000100:00100000:2.0:1787739082.627212:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665799296:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787739082.627492:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce30c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665799424:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787739082.627519:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.627799:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887840 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.627943:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665799424 00000100:00100000:1.0:1787739082.627948:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665799424 00000100:00100000:1.0:1787739082.627975:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3703 00000100:00100000:1.0:1787739082.627982:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6538:x1874579665799424:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.628057:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6538:x1874579665799424:12345-0@lo:1000:mdt_io00_002.0 Request processed in 75us (536us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.628065:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3703 00000100:00100000:2.0:1787739082.628131:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce30c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665799424:0@lo:1000:mdt_io00_002.0 00000040:00080000:2.0:1787739082.628490:0:6538:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].407, 1264 off609576 00000040:00080000:2.0:1787739082.628554:0:6538:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].407, 1344 off661320 00000100:00100000:2.0:1787739082.628579:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce30c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665799552:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787739082.628608:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.628622:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.628961:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665799552 00000100:00100000:1.0:1787739082.628967:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665799552 00000100:00100000:1.0:1787739082.628994:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3704 00000100:00100000:1.0:1787739082.629000:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:6538:x1874579665799552:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.629102:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6538:x1874579665799552:12345-0@lo:1000:mdt_io00_002.0 Request processed in 101us (493us total) trans 4294969882 rc 0/0 00000100:00100000:1.0:1787739082.629109:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3704 00000100:00100000:2.0:1787739082.629179:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce30c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665799552:0@lo:1000:mdt_io00_002.0 00000004:00080000:2.0:1787739082.629188:0:6538:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306887a80 with version 775 00000004:00080000:1.0:1787739082.629404:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 775 rpc_version 775 00000020:00100000:1.0:1787739082.629797:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:1.0:1787739082.629805:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0743c415a40 x1874579665799808/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 3088/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787739082.629823:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c415a40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665799808:0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739082.629864:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.629897:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241300 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.629949:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665799808 00000100:00100000:3.0:1787739082.629954:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665799808 00000100:00100000:3.0:1787739082.629991:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3705 00000100:00100000:3.0:1787739082.629998:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c4170c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+15:7740:x1874579665799808:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739082.630494:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c4170c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+16:7740:x1874579665799808:12345-0@lo:1000:osp_up1-0.0 Request processed in 494us (627us total) trans 4294968491 rc 0/0 00000100:00100000:3.0:1787739082.630506:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3705 00000100:00100000:1.0:1787739082.630784:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c415a40 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665799808:0@lo:1000:osp_up1-0.0 00000004:00010000:3.0:1787739082.631012:0:6538:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000443f75c0 reply state 000000003df0a2a2 transno 4294983523 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1aec2f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x152e:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.631031:0:6538:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089400/0x7ee4453e4a1aec2f lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x152e:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.631039:0:6538:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152e:0x0].0x0 (ffffa074126f6300) 00010000:00010000:3.0:1787739082.631042:0:6538:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 25 00000100:00100000:3.0:1787739082.631046:0:6538:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1787739082.631053:0:6538:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089200/0x7ee4453e4a1aec21 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.631060:0:6538:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089200/0x7ee4453e4a1aec21 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.631068:0:6538:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089200/0x7ee4453e4a1aec21 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.631074:0:6538:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306089200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739082.631079:0:6538:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089200/0x7ee4453e4a1aec21 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.631087:0:6538:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089200/0x7ee4453e4a1aec21 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.631097:0:6538:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa0741590dc00) 00010000:00010000:3.0:1787739082.631099:0:6538:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739082.631103:0:6538:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089200/0x7ee4453e4a1aec21 lrc: 2/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.631109:0:6538:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089200/0x7ee4453e4a1aec21 lrc: 0/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1787739082.631130:0:6538:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000443f75c0 reply state 000000003df0a2a2 transno 4294983523 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1aec28 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.631139:0:6538:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089000/0x7ee4453e4a1aec28 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.631144:0:6538:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x391d5834 (ffffa0741590db00) 00010000:00010000:3.0:1787739082.631146:0:6538:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1787739082.631148:0:6538:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00080000:3.0:1787739082.631154:0:6538:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0xcc:0x0] 00010000:00010000:3.0:1787739082.631161:0:6538:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1aec05 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aec0c expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.631169:0:6538:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1aec05 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aec0c expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.631174:0:6538:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1aec05 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aec0c expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.631197:0:6538:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:3.0:1787739082.631202:0:6538:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0xcc:0x0] 00010000:00010000:3.0:1787739082.631206:0:6538:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1aec13 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0xcc:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aec1a expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.631214:0:6538:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306088e00/0x7ee4453e4a1aec13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0x391d5834 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aec1a expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739082.631486:0:6538:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000402:0x186:0x0] 00010000:00010000:1.0:1787739082.631746:0:10339:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739082.631753:0:10339:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1aec05 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aec0c expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.631761:0:10339:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1e00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739082.631766:0:10339:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1aec05 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aec0c expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739082.631774:0:10339:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1aec05 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aec0c expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.631808:0:10339:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1aec05 lrc: 2/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aec0c expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.631815:0:10339:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739082.631845:0:10339:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1aec05 lrc: 1/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aec0c expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.631852:0:10339:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306aa1e00/0x7ee4453e4a1aec05 lrc: 0/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aec0c expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.631866:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739082.631868:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739082.631870:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739082.631888:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1787739082.631902:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c415680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665799936:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739082.631954:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.631992:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665799936 00000100:00100000:1.0:1787739082.631998:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665799936 00000100:00100000:1.0:1787739082.632042:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1437 00000100:00100000:1.0:1787739082.632051:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c417480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+16:3653:x1874579665799936:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739082.632062:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739082.632079:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aec0c lrc: 0/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aec05 expref: 16 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.632091:0:9525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa074331c2800) 00010000:00010000:1.0:1787739082.632098:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739082.632128:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c417480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000-mdtlov_UUID+15:3653:x1874579665799936:12345-0@lo:103:ldlm_bl.0 Request processed in 79us (175us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.632136:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1437 00000100:00100000:1.0:1787739082.632172:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c415680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665799936:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739082.632944:0:6538:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aebfe lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.632953:0:6538:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aebfe lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.632959:0:6538:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aebfe lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.632964:0:6538:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739082.632968:0:6538:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aebfe lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.632975:0:6538:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aebfe lrc: 4/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.632985:0:6538:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffffa074331c2700) 00010000:00010000:2.0:1787739082.632988:0:6538:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739082.632991:0:6538:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aebfe lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.632996:0:6538:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aebfe lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739082.633124:0:6538:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+23:18317:x1874579652556032:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 25579us (25823us total) trans 4294983523 rc 0/0 00000100:00100000:2.0:1787739082.633134:0:6538:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 3 00000100:00100000:1.0:1787739082.641048:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.641161:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652556160 00000100:00100000:0.0:1787739082.641171:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652556160 00000100:00100000:0.0:1787739082.641634:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23717 00000100:00100000:0.0:1787739082.641715:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c415680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+23:18317:x1874579652556160:12345-192.168.203.15@tcp:34:lfs.0 00010000:00010000:0.0:1787739082.641791:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1aec36 lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.641809:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1aec36 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.641821:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1aec36 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.641834:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aec3d lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0x18618557 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.641841:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aec3d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.641847:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aec3d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.641910:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1aec44 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x152f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.641919:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1aec44 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.641926:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1aec44 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739082.641968:0:6523:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1aec44 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.641982:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1aec36 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.641990:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1aec36 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.641996:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1aec36 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.642004:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1aec36 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.642010:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306089e00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739082.642015:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1aec36 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.642023:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1aec36 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.642035:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa0741590da00) 00010000:00010000:0.0:1787739082.642038:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739082.642041:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1aec36 lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.642048:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089e00/0x7ee4453e4a1aec36 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.642067:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aec3d lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.642074:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aec3d lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.642081:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aec3d lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.642087:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306088a00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739082.642090:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aec3d lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.642096:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aec3d lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.642103:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x18618557 (ffffa073068a9b00) 00010000:00010000:0.0:1787739082.642105:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739082.642107:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aec3d lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.642113:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306088a00/0x7ee4453e4a1aec3d lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.642123:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1aec44 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.642129:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1aec44 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x152f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.642134:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1aec44 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x152f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.642139:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fd000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739082.642142:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1aec44 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x152f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.642148:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1aec44 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x152f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.642159:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152f:0x0].0x0 (ffffa074126f6100) 00010000:00010000:0.0:1787739082.642162:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739082.642164:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1aec44 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x152f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.642170:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1aec44 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x152f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.642543:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c415680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+23:18317:x1874579652556160:12345-192.168.203.15@tcp:34:lfs.0 Request processed in 895us (1500us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.642553:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23717 00000100:00100000:1.0:1787739082.648322:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739082.648739:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652556416 00000100:00100000:2.0:1787739082.648749:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652556416 00000100:00100000:2.0:1787739082.648838:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23718 00000100:00100000:2.0:1787739082.648849:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c4161c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+23:18317:x1874579652556416:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:2.0:1787739082.648868:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739082.648897:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1aec4b lrc: 2/0,0 mode: --/CR res: [0x200000402:0x152f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095597 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.648996:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1400/0x7ee4453e4a1aec52 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x152f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.649010:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1400/0x7ee4453e4a1aec52 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x152f:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.649022:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1400/0x7ee4453e4a1aec52 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x152f:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739082.649033:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x152f:0x0]: returning size 2 00010000:00010000:2.0:1787739082.649124:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1aec4b lrc: 0/0,0 mode: --/CR res: [0x200000402:0x152f:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095597 expref: 26 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.649142:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1400/0x7ee4453e4a1aec52 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x152f:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095597 expref: 25 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.649151:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152f:0x0].0x0 (ffffa074126f6100) 00010000:00010000:2.0:1787739082.649154:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1787739082.649157:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa1400, rc 0) 00000100:00100000:2.0:1787739082.649229:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c4161c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+25:18317:x1874579652556416:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 373us (1014us total) trans 4294983524 rc 0/0 00000100:00100000:2.0:1787739082.649237:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23718 00000100:00100000:0.0:1787739082.657291:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739082.657483:0:14663:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652556544 00000100:00100000:3.0:1787739082.657493:0:14663:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652556544 00000100:00100000:3.0:1787739082.657562:0:14663:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5198 00000100:00100000:3.0:1787739082.657573:0:14663:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652556544:12345-192.168.203.15@tcp:1:lfs.0 00010000:00010000:3.0:1787739082.657616:0:14663:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000a92a1b2/0x7ee4453e4a1aec60 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14663 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.657636:0:14663:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002bbfd57a/0x7ee4453e4a1ae9c7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095527 expref: 10 pid: 14659 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.657645:0:14663:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x9c9:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1787739082.657660:0:14663:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x9c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1787739082.657665:0:14663:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000a92a1b2/0x7ee4453e4a1aec60 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14663 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.657699:0:14663:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002bbfd57a/0x7ee4453e4a1ae9c7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095527 expref: 10 pid: 14659 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.657714:0:14663:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002bbfd57a/0x7ee4453e4a1ae9c7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095527 expref: 10 pid: 14659 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739082.657729:0:14663:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383f840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0001_UUID:14663:1874579665800064:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739082.657808:0:14663:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.663589:0:14663:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383f840 pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0001_UUID:14663:1874579665800064:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739082.663619:0:14663:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000000a92a1b2/0x7ee4453e4a1aec60 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14663 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.665191:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739082.665282:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652556672 00000100:00100000:3.0:1787739082.665291:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652556672 00002000:00100000:3.0:1787739082.665335:0:8443:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x9c9:0x0] (0->1) req@ffffa07306be43c0 x1874579652556672/t0(0) o4->e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2@192.168.203.15@tcp:263/0 lens 496/0 e 0 to 0 dl 1787739093 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787739082.665351:0:8443:0:(ofd_dev.c:2572:ofd_prolong_extent_locks()) Prolong locks for req ffffa07306be43c0 with x1874579652556672 ext(0->1) 00010000:00010000:3.0:1787739082.665359:0:8443:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000002bbfd57a/0x7ee4453e4a1ae9c7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095527 expref: 11 pid: 14659 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.665374:0:8443:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000002bbfd57a/0x7ee4453e4a1ae9c7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095527 expref: 11 pid: 14659 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787739082.665382:0:8443:0:(ofd_dev.c:2721:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffffa07306be43c0 00000100:00100000:3.0:1787739082.665443:0:8443:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 3 00000100:00100000:3.0:1787739082.665454:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be43c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2408:x1874579652556672:12345-192.168.203.15@tcp:4:bash.0 00000100:00100000:3.0:1787739082.665824:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be43c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:2408:x1874579652556672:12345-192.168.203.15@tcp:4:bash.0 Request processed in 370us (637us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.665832:0:8443:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 3 00002000:00100000:3.0:1787739082.665839:0:8443:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x9c9:0x0] (0->1) req@ffffa07306be43c0 x1874579652556672/t0(0) o4->e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2@192.168.203.15@tcp:263/0 lens 496/448 e 0 to 0 dl 1787739093 ref 1 fl Complete:H/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787739082.665847:0:8443:0:(ofd_dev.c:2572:ofd_prolong_extent_locks()) Prolong locks for req ffffa07306be43c0 with x1874579652556672 ext(0->1) 00010000:00010000:3.0:1787739082.665854:0:8443:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000002bbfd57a/0x7ee4453e4a1ae9c7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095527 expref: 12 pid: 14659 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.665865:0:8443:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000002bbfd57a/0x7ee4453e4a1ae9c7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095527 expref: 12 pid: 14659 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787739082.665872:0:8443:0:(ofd_dev.c:2721:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffffa07306be43c0 00000100:00100000:3.0:1787739082.671630:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739082.672894:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652556800 00000100:00100000:1.0:1787739082.672907:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652556800 00010000:00010000:2.0:1787739082.673032:0:10340:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002bbfd57a/0x7ee4453e4a1ae9c7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095527 expref: 12 pid: 14659 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739082.673111:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1601 00000100:00100000:2.0:1787739082.673121:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383fc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:2408:x1874579652556800:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739082.673137:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1787739082.673153:0:10340:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x2c0000401:0x9c9:0x0] updating lvb mtime from disk: 0 -> 1787739082 00002000:00010000:2.0:1787739082.673156:0:10340:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x2c0000401:0x9c9:0x0] updating lvb ctime from disk: 0 -> 1787739082 00010000:00010000:2.0:1787739082.673162:0:10340:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002bbfd57a/0x7ee4453e4a1ae9c7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095527 expref: 12 pid: 14659 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.673176:0:10340:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002bbfd57a/0x7ee4453e4a1ae9c7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095527 expref: 12 pid: 14659 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.673200:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002bbfd57a/0x7ee4453e4a1ae9c7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095527 expref: 12 pid: 14659 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.673230:0:10340:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x9c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787739082.673235:0:10340:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x9c9:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787739082.673240:0:10340:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000a92a1b2/0x7ee4453e4a1aec60 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 14663 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.673249:0:10340:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000a92a1b2/0x7ee4453e4a1aec60 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 14663 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.673282:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739082.673334:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383fc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2408:x1874579652556800:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 215us (1712us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.673343:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1601 00010000:00010000:3.0:1787739082.674243:0:14663:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000000a92a1b2/0x7ee4453e4a1aec60 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14663 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.674257:0:14663:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000a92a1b2/0x7ee4453e4a1aec60 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14663 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.674281:0:14663:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000a92a1b2/0x7ee4453e4a1aec60 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14663 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.674290:0:14663:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000000a92a1b2/0x7ee4453e4a1aec60 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14663 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.674299:0:14663:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000000a92a1b2/0x7ee4453e4a1aec60 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14663 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.674306:0:14663:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa200 already unused, calling callback (ffffffffc0cce440) 00010000:00010000:3.0:1787739082.674310:0:14663:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000000a92a1b2/0x7ee4453e4a1aec60 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14663 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.674320:0:14663:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000000a92a1b2/0x7ee4453e4a1aec60 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14663 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.674344:0:14663:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000000a92a1b2/0x7ee4453e4a1aec60 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14663 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.674350:0:14663:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000a92a1b2/0x7ee4453e4a1aec60 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x9c9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14663 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739082.674457:0:14663:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a30f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652556544:12345-192.168.203.15@tcp:1:lfs.0 Request processed in 16886us (17173us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.674466:0:14663:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5198 00000100:00100000:3.0:1787739082.680125:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.680179:0:6536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652557056 00000100:00100000:0.0:1787739082.680187:0:6536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652557056 00000100:00100000:0.0:1787739082.680245:0:6536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 4 00000100:00100000:0.0:1787739082.680254:0:6536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+25:18317:x1874579652557056:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:0.0:1787739082.680306:0:6536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739082.680333:0:6536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306089800/0x7ee4453e4a1ae9b9 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x152f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095520 expref: 25 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.680353:0:6536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152f:0x0].0x0 (ffffa074126f6100) 00010000:00010000:0.0:1787739082.680357:0:6536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1787739082.680392:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc400/0x7ee4453e4a1aec67 lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.680405:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc400/0x7ee4453e4a1aec67 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.680444:0:6536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc400/0x7ee4453e4a1aec67 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1787739082.680462:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000402:0x186:0x0] 00000100:00100000:0.0:1787739082.680532:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be61c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665800192:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.680579:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.680611:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3c00 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.680644:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665800192 00000100:00100000:3.0:1787739082.680657:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665800192 00000100:00100000:3.0:1787739082.680692:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3706 00000100:00100000:3.0:1787739082.680699:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+15:6536:x1874579665800192:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.680793:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+15:6536:x1874579665800192:12345-0@lo:1000:mdt_io00_000.0 Request processed in 94us (212us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.680801:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3706 00000100:00100000:0.0:1787739082.681074:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be61c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665800192:0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.681613:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be61c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665800320:0@lo:403:mdt_io00_000.0 00000100:00100000:0.0:1787739082.681648:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.681670:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3540 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.682103:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665800320 00000100:00100000:2.0:1787739082.682110:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665800320 00000100:00100000:2.0:1787739082.682157:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3707 00000100:00100000:2.0:1787739082.682165:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be52c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+15:6536:x1874579665800320:12345-0@lo:403:mdt_io00_000.0 00000100:00100000:0.0:1787739082.685708:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be52c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+15:6536:x1874579665800320:12345-0@lo:403:mdt_io00_000.0 Request processed in 3536us (4051us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.685722:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3707 00000100:00100000:1.0:1787739082.685859:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be61c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665800320:0@lo:403:mdt_io00_000.0 00010000:00010000:1.0:1787739082.687091:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840e00/0x7ee4453e4a1aec6e lrc: 3/0,1 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.687791:0:6536:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840e00/0x7ee4453e4a1aec6e lrc: 3/0,1 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.687806:0:6536:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840e00/0x7ee4453e4a1aec6e lrc: 3/0,1 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739082.687831:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c416580 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665800448:0@lo:101:mdt_io00_000.0 00000100:00100000:1.0:1787739082.687896:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.687971:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803780 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.688172:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665800448 00000100:00100000:2.0:1787739082.688179:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665800448 00000100:00100000:2.0:1787739082.688289:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3708 00000100:00100000:2.0:1787739082.688300:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c415e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+15:6536:x1874579665800448:12345-0@lo:101:mdt_io00_000.0 00010000:00010000:2.0:1787739082.688315:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739082.688332:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1aec75 lrc: 2/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aec6e expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.688358:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1aec75 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aec6e expref: 16 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.688371:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1aec75 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aec6e expref: 16 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.688379:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:2.0:1787739082.688382:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739082.688385:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa0c00, rc 0) 00000100:00100000:2.0:1787739082.688451:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c415e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+16:6536:x1874579665800448:12345-0@lo:101:mdt_io00_000.0 Request processed in 153us (555us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.688461:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3708 00000100:00100000:1.0:1787739082.688614:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c416580 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665800448:0@lo:101:mdt_io00_000.0 00010000:00010000:1.0:1787739082.688631:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840e00/0x7ee4453e4a1aec6e lrc: 4/0,1 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aec75 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.688662:0:6536:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840e00/0x7ee4453e4a1aec6e lrc: 4/0,1 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aec75 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739082.688683:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0xcc:0x0] 00010000:00010000:1.0:1787739082.688707:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841800/0x7ee4453e4a1aec7c lrc: 3/0,1 mode: --/PW res: [0x240000400:0xcc:0x0].0x18618557 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.688714:0:6536:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841800/0x7ee4453e4a1aec7c lrc: 3/0,1 mode: --/PW res: [0x240000400:0xcc:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.688722:0:6536:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841800/0x7ee4453e4a1aec7c lrc: 3/0,1 mode: --/PW res: [0x240000400:0xcc:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739082.688735:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c416580 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665800576:0@lo:101:mdt_io00_000.0 00000100:00100000:1.0:1787739082.688762:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.694886:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803780 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.698202:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665800576 00000100:00100000:3.0:1787739082.698217:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665800576 00000100:00100000:3.0:1787739082.698338:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3709 00000100:00100000:3.0:1787739082.698349:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c416d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+16:6536:x1874579665800576:12345-0@lo:101:mdt_io00_000.0 00010000:00010000:3.0:1787739082.698370:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739082.698393:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1aec83 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xcc:0x0].0x18618557 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aec7c expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.698445:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1aec83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aec7c expref: 17 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.698460:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa200/0x7ee4453e4a1aec83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aec7c expref: 17 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.698475:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x18618557 (ffffa074331c2600) 00010000:00010000:3.0:1787739082.698478:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739082.698482:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aa200, rc 0) 00000100:00100000:3.0:1787739082.698575:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c416d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+17:6536:x1874579665800576:12345-0@lo:101:mdt_io00_000.0 Request processed in 228us (9811us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.698583:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3709 00000100:00100000:1.0:1787739082.698732:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c416580 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665800576:0@lo:101:mdt_io00_000.0 00010000:00010000:1.0:1787739082.698752:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841800/0x7ee4453e4a1aec7c lrc: 4/0,1 mode: PW/PW res: [0x240000400:0xcc:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aec83 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.698762:0:6536:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841800/0x7ee4453e4a1aec7c lrc: 4/0,1 mode: PW/PW res: [0x240000400:0xcc:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aec83 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739082.698784:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0xcc:0x0] 00010000:00010000:1.0:1787739082.698804:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aec8a lrc: 3/0,1 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.698814:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aec8a lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.698823:0:6536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aec8a lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.698834:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841a00/0x7ee4453e4a1aec91 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xce:0x0].0x18618557 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.698841:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841a00/0x7ee4453e4a1aec91 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.698848:0:6536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841a00/0x7ee4453e4a1aec91 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.698919:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840200/0x7ee4453e4a1aec98 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x152f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.698941:0:6536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1400/0x7ee4453e4a1aec52 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x152f:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095597 expref: 24 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.698949:0:6536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840200/0x7ee4453e4a1aec98 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x152f:0x0].0x0 bits 0x25/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.698979:0:6536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1400/0x7ee4453e4a1aec52 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x152f:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095597 expref: 24 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.698993:0:6536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1400/0x7ee4453e4a1aec52 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x152f:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095597 expref: 24 pid: 6525 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739082.699008:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c416580 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000_UUID:6536:1874579665800704:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739082.699083:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07303803780 going to sleep for 11 seconds 00010000:00010000:1.0:1787739082.706556:0:6536:0:(ldlm_lockd.c:741:ldlm_handle_ast_error()) ### client (nid 192.168.203.15@tcp) returned -22 from blocking AST (req@00000000c5425594 x1874579665800704) - normal race ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1400/0x7ee4453e4a1aec52 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x152f:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095597 expref: 24 pid: 6525 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.706583:0:6536:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1400/0x7ee4453e4a1aec52 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x152f:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095597 expref: 24 pid: 6525 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.706609:0:6536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1400/0x7ee4453e4a1aec52 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x152f:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095597 expref: 24 pid: 6525 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739082.706638:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c416580 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000_UUID:6536:1874579665800704:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739082.706659:0:6536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152f:0x0].0x0 (ffffa074126f6100) 00010000:00010000:1.0:1787739082.706665:0:6536:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840200/0x7ee4453e4a1aec98 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x152f:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.706674:0:6536:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840200/0x7ee4453e4a1aec98 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x152f:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x50290000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.706690:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840200/0x7ee4453e4a1aec98 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x152f:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x50290000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.706711:0:6536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840200/0x7ee4453e4a1aec98 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x152f:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.706720:0:6536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840200/0x7ee4453e4a1aec98 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x152f:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.706726:0:6536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840200/0x7ee4453e4a1aec98 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x152f:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739082.706863:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c416580 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665800832:0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787739082.706953:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.706983:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306241300 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.707401:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665800832 00000100:00100000:2.0:1787739082.707408:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665800832 00000100:00100000:2.0:1787739082.707478:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3710 00000100:00100000:2.0:1787739082.707488:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c416940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+17:6536:x1874579665800832:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.708006:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c416940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+17:6536:x1874579665800832:12345-0@lo:1000:mdt_io00_000.0 Request processed in 519us (1052us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739082.708015:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3710 00000100:00100000:3.0:1787739082.711266:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c416580 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665800832:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.719803:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce1a40 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665800960:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.720004:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.720042:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.720089:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665800960 00000100:00100000:3.0:1787739082.720098:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665800960 00000100:00100000:3.0:1787739082.720147:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3711 00000100:00100000:3.0:1787739082.720158:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+17:6536:x1874579665800960:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.720242:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+17:6536:x1874579665800960:12345-0@lo:1000:mdt_io00_000.0 Request processed in 87us (240us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.720249:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3711 00000100:00100000:2.0:1787739082.720388:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce1a40 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665800960:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.720526:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce1a40 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665801088:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.720552:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.720576:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887900 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.720768:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665801088 00000100:00100000:1.0:1787739082.720773:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665801088 00000100:00100000:1.0:1787739082.720810:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3712 00000100:00100000:1.0:1787739082.720818:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+17:6536:x1874579665801088:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787739082.720909:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+17:6536:x1874579665801088:12345-0@lo:1000:mdt_io00_000.0 Request processed in 84us (347us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.720917:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3712 00000100:00100000:2.0:1787739082.720921:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce1a40 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665801088:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.721025:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce1a40 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665801216:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.721051:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.721116:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887900 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.721252:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665801216 00000100:00100000:3.0:1787739082.721256:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665801216 00000100:00100000:3.0:1787739082.721311:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3713 00000100:00100000:3.0:1787739082.721328:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+17:6536:x1874579665801216:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.721387:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce1680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+17:6536:x1874579665801216:12345-0@lo:1000:mdt_io00_000.0 Request processed in 68us (334us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.721393:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3713 00000100:00100000:1.0:1787739082.721484:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce1a40 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665801216:0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787739082.721579:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c414b40 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665801344:0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787739082.721612:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.721627:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306107a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787739082.721700:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665801344 00000100:00100000:0.0:1787739082.721705:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665801344 00000100:00100000:0.0:1787739082.721742:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3714 00000100:00100000:0.0:1787739082.721750:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c4152c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6536:x1874579665801344:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.721827:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c4152c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:6536:x1874579665801344:12345-0@lo:1000:mdt_io00_000.0 Request processed in 77us (213us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.721833:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3714 00000100:00100000:1.0:1787739082.722092:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c414b40 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665801344:0@lo:1000:mdt_io00_000.0 00000040:00080000:1.0:1787739082.722441:0:6536:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].408, 1264 off610840 00000040:00080000:1.0:1787739082.722495:0:6536:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].408, 1344 off662664 00000100:00100000:1.0:1787739082.722520:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743c414b40 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665801472:0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787739082.722554:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.722567:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306107c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.722616:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665801472 00000100:00100000:2.0:1787739082.722621:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665801472 00000100:00100000:2.0:1787739082.722655:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3715 00000100:00100000:2.0:1787739082.722662:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c416580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:6536:x1874579665801472:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.722767:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c416580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6536:x1874579665801472:12345-0@lo:1000:mdt_io00_000.0 Request processed in 105us (213us total) trans 4294969880 rc 0/0 00000100:00100000:2.0:1787739082.722775:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3715 00000100:00100000:1.0:1787739082.722809:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743c414b40 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665801472:0@lo:1000:mdt_io00_000.0 00000004:00080000:1.0:1787739082.722818:0:6536:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306107cc0 with version 776 00000004:00080000:2.0:1787739082.722876:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 776 rpc_version 776 00000020:00100000:2.0:1787739082.723032:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739082.723041:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07304ce0780 x1874579665801728/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 3088/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739082.723056:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce0780 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665801728:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739082.723087:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.723111:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887480 going to sleep for 11 seconds 00000100:00100000:0.0:1787739082.723160:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665801728 00000100:00100000:0.0:1787739082.723165:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665801728 00000100:00100000:0.0:1787739082.723200:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3716 00000100:00100000:0.0:1787739082.723207:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+17:7740:x1874579665801728:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739082.723517:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+18:7740:x1874579665801728:12345-0@lo:1000:osp_up1-0.0 Request processed in 310us (428us total) trans 4294968492 rc 0/0 00000100:00100000:0.0:1787739082.723525:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3716 00000100:00100000:1.0:1787739082.723675:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce0780 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665801728:0@lo:1000:osp_up1-0.0 00000004:00010000:2.0:1787739082.723741:0:6536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009da80427 reply state 0000000015d03b46 transno 4294983525 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840200/0x7ee4453e4a1aec98 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x152f:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.723760:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840200/0x7ee4453e4a1aec98 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x152f:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.723769:0:6536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152f:0x0].0x0 (ffffa074126f6100) 00010000:00010000:2.0:1787739082.723771:0:6536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 25 00000100:00100000:2.0:1787739082.723775:0:6536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1787739082.723783:0:6536:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aec8a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.723790:0:6536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aec8a lrc: 4/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.723796:0:6536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aec8a lrc: 3/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.723802:0:6536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aec8a lrc: 4/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.723809:0:6536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739082.723814:0:6536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aec8a lrc: 4/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.723823:0:6536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aec8a lrc: 5/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.723833:0:6536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa0730713a900) 00010000:00010000:2.0:1787739082.723835:0:6536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739082.723839:0:6536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aec8a lrc: 3/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.723845:0:6536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aec8a lrc: 0/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739082.723864:0:6536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009da80427 reply state 0000000015d03b46 transno 4294983525 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841a00/0x7ee4453e4a1aec91 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.723873:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841a00/0x7ee4453e4a1aec91 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.723889:0:6536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x18618557 (ffffa0730713a300) 00010000:00010000:2.0:1787739082.723892:0:6536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1787739082.723893:0:6536:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00080000:2.0:1787739082.723898:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0xcc:0x0] 00010000:00010000:2.0:1787739082.723913:0:6536:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840e00/0x7ee4453e4a1aec6e lrc: 4/0,1 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aec75 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.723922:0:6536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840e00/0x7ee4453e4a1aec6e lrc: 4/0,1 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aec75 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.724043:0:6536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840e00/0x7ee4453e4a1aec6e lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aec75 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.724071:0:6536:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:2.0:1787739082.724077:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0xcc:0x0] 00010000:00010000:2.0:1787739082.724083:0:6536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841800/0x7ee4453e4a1aec7c lrc: 4/0,1 mode: PW/PW res: [0x240000400:0xcc:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aec83 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.724090:0:6536:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841800/0x7ee4453e4a1aec7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0x18618557 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aec83 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739082.724102:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000402:0x186:0x0] 00010000:00010000:2.0:1787739082.724110:0:6536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc400/0x7ee4453e4a1aec67 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.724116:0:6536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc400/0x7ee4453e4a1aec67 lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.724121:0:6536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc400/0x7ee4453e4a1aec67 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.724126:0:6536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fc400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739082.724129:0:6536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc400/0x7ee4453e4a1aec67 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.724135:0:6536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc400/0x7ee4453e4a1aec67 lrc: 4/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.724144:0:6536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffffa073068a9b00) 00010000:00010000:2.0:1787739082.724147:0:6536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739082.724150:0:6536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc400/0x7ee4453e4a1aec67 lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.724160:0:6536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc400/0x7ee4453e4a1aec67 lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739082.724228:0:6536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383d680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+25:18317:x1874579652557056:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 43973us (44110us total) trans 4294983525 rc 0/0 00000100:00100000:2.0:1787739082.724236:0:6536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 4 00010000:00010000:1.0:1787739082.724507:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787739082.724514:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840e00/0x7ee4453e4a1aec6e lrc: 2/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aec75 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.724525:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840e00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:1.0:1787739082.724529:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840e00/0x7ee4453e4a1aec6e lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aec75 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739082.724537:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840e00/0x7ee4453e4a1aec6e lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aec75 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.724564:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840e00/0x7ee4453e4a1aec6e lrc: 2/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aec75 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.724571:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787739082.724590:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840e00/0x7ee4453e4a1aec6e lrc: 1/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aec75 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.724598:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306840e00/0x7ee4453e4a1aec6e lrc: 0/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aec75 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.724613:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739082.724615:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787739082.724616:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739082.724625:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787739082.724636:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073042470c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665801856:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739082.724675:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.724722:0:6510:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665801856 00000100:00100000:1.0:1787739082.724727:0:6510:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665801856 00000100:00100000:1.0:1787739082.724764:0:6510:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1438 00000100:00100000:1.0:1787739082.724771:0:6510:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c3840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+18:3652:x1874579665801856:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1787739082.724785:0:6510:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1787739082.724802:0:6510:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0c00/0x7ee4453e4a1aec75 lrc: 0/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aec6e expref: 18 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.724814:0:6510:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:1.0:1787739082.724821:0:6510:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739082.724850:0:6510:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c3840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000-mdtlov_UUID+17:3652:x1874579665801856:12345-0@lo:103:ldlm_bl.0 Request processed in 80us (175us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.724857:0:6510:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1438 00000100:00100000:1.0:1787739082.726056:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073042470c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3652:1874579665801856:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739082.736132:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.736299:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652557184 00000100:00100000:0.0:1787739082.736309:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652557184 00000100:00100000:0.0:1787739082.736377:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23719 00000100:00100000:0.0:1787739082.736388:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+25:18317:x1874579652557184:12345-192.168.203.15@tcp:34:lfs.0 00010000:00010000:0.0:1787739082.736502:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fda00/0x7ee4453e4a1aec9f lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.736519:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fda00/0x7ee4453e4a1aec9f lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.736531:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fda00/0x7ee4453e4a1aec9f lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.736545:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1aeca6 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0x84832a57 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.736552:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1aeca6 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.736559:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1aeca6 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.736618:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1aecad lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1531:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.736626:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1aecad lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1531:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.736632:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1aecad lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1531:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739082.736688:0:13793:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1aecad lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1531:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.736702:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fda00/0x7ee4453e4a1aec9f lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.736710:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fda00/0x7ee4453e4a1aec9f lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.736717:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fda00/0x7ee4453e4a1aec9f lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.736723:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fda00/0x7ee4453e4a1aec9f lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.736730:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fda00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739082.736734:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fda00/0x7ee4453e4a1aec9f lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.736742:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fda00/0x7ee4453e4a1aec9f lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.736754:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa073068a9c00) 00010000:00010000:0.0:1787739082.736757:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739082.736762:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fda00/0x7ee4453e4a1aec9f lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.736768:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fda00/0x7ee4453e4a1aec9f lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.736786:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1aeca6 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.736792:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1aeca6 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.736799:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1aeca6 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.736804:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fd400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739082.736807:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1aeca6 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.736813:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1aeca6 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.737067:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x84832a57 (ffffa073068a9700) 00010000:00010000:0.0:1787739082.737071:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739082.737075:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1aeca6 lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.737081:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd400/0x7ee4453e4a1aeca6 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.737093:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1aecad lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1531:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.737099:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1aecad lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1531:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.737105:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1aecad lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1531:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.737110:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fc800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739082.737114:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1aecad lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1531:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.737120:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1aecad lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1531:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.737127:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1531:0x0].0x0 (ffffa07403eeb100) 00010000:00010000:0.0:1787739082.737130:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739082.737132:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1aecad lrc: 2/0,0 mode: --/PR res: [0x200000402:0x1531:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.737138:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1aecad lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1531:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.737218:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383cb40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+25:18317:x1874579652557184:12345-192.168.203.15@tcp:34:lfs.0 Request processed in 830us (1095us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.737226:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23719 00000100:00100000:0.0:1787739082.743696:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739082.746078:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652557440 00000100:00100000:1.0:1787739082.746093:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652557440 00000100:00100000:1.0:1787739082.746166:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23720 00000100:00100000:1.0:1787739082.746177:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+25:18317:x1874579652557440:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739082.746196:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739082.746217:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841600/0x7ee4453e4a1aecb4 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1531:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09559e expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.746327:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841400/0x7ee4453e4a1aecbb lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1531:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.746341:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841400/0x7ee4453e4a1aecbb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1531:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.746352:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841400/0x7ee4453e4a1aecbb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1531:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739082.746365:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1531:0x0]: returning size 3 00010000:00010000:1.0:1787739082.746501:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841600/0x7ee4453e4a1aecb4 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1531:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09559e expref: 28 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.746516:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841400/0x7ee4453e4a1aecbb lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1531:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09559e expref: 27 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.746523:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1531:0x0].0x0 (ffffa07403eeb100) 00010000:00010000:1.0:1787739082.746526:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:1.0:1787739082.746528:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841400, rc 0) 00000100:00100000:1.0:1787739082.746690:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be70c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+27:18317:x1874579652557440:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 514us (3001us total) trans 4294983526 rc 0/0 00000100:00100000:1.0:1787739082.746698:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23720 00000100:00100000:1.0:1787739082.763019:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.764197:0:8885:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652557568 00000100:00100000:0.0:1787739082.764213:0:8885:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652557568 00000100:00100000:0.0:1787739082.764285:0:8885:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5199 00000100:00100000:0.0:1787739082.764296:0:8885:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c1680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2409:x1874579652557568:12345-192.168.203.15@tcp:1:lfs.0 00010000:00010000:0.0:1787739082.764342:0:8885:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b4cfedb0/0x7ee4453e4a1aecc9 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8885 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.764364:0:8885:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e6a9f713/0x7ee4453e4a1aea1b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095543 expref: 10 pid: 8439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.764373:0:8885:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x2c0000401:0x9ca:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1787739082.764379:0:8885:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x9ca:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1787739082.764384:0:8885:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b4cfedb0/0x7ee4453e4a1aecc9 lrc: 4/0,1 mode: --/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8885 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.764473:0:8885:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e6a9f713/0x7ee4453e4a1aea1b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095543 expref: 10 pid: 8439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.764487:0:8885:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e6a9f713/0x7ee4453e4a1aea1b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095543 expref: 10 pid: 8439 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.764501:0:8885:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07306be7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8885:1874579665801984:192.168.203.15@tcp:104: 00000100:00100000:0.0:1787739082.764596:0:8885:0:(client.c:2715:ptlrpc_set_wait()) set ffffa073068a3780 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.769774:0:8885:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07306be7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8885:1874579665801984:192.168.203.15@tcp:104: 00010000:00010000:2.0:1787739082.769806:0:8885:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b4cfedb0/0x7ee4453e4a1aecc9 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8885 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739082.771264:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.771503:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652557696 00000100:00100000:0.0:1787739082.771512:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652557696 00002000:00100000:0.0:1787739082.771572:0:8444:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x9ca:0x0] (0->2) req@ffffa073043c0b40 x1874579652557696/t0(0) o4->e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2@192.168.203.15@tcp:263/0 lens 496/0 e 0 to 0 dl 1787739093 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787739082.771583:0:8444:0:(ofd_dev.c:2572:ofd_prolong_extent_locks()) Prolong locks for req ffffa073043c0b40 with x1874579652557696 ext(0->2) 00010000:00010000:0.0:1787739082.771591:0:8444:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000e6a9f713/0x7ee4453e4a1aea1b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095543 expref: 11 pid: 8439 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.771607:0:8444:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000e6a9f713/0x7ee4453e4a1aea1b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095543 expref: 11 pid: 8439 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787739082.771615:0:8444:0:(ofd_dev.c:2721:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffffa073043c0b40 00000100:00100000:0.0:1787739082.771685:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 4 00000100:00100000:0.0:1787739082.771694:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c0b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2408:x1874579652557696:12345-192.168.203.15@tcp:4:bash.0 00000100:00100000:0.0:1787739082.772090:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c0b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:2408:x1874579652557696:12345-192.168.203.15@tcp:4:bash.0 Request processed in 395us (832us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.772098:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 4 00002000:00100000:0.0:1787739082.772106:0:8444:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x9ca:0x0] (0->2) req@ffffa073043c0b40 x1874579652557696/t0(0) o4->e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2@192.168.203.15@tcp:263/0 lens 496/448 e 0 to 0 dl 1787739093 ref 1 fl Complete:H/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787739082.772113:0:8444:0:(ofd_dev.c:2572:ofd_prolong_extent_locks()) Prolong locks for req ffffa073043c0b40 with x1874579652557696 ext(0->2) 00010000:00010000:0.0:1787739082.772119:0:8444:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000e6a9f713/0x7ee4453e4a1aea1b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095543 expref: 12 pid: 8439 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.772130:0:8444:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000e6a9f713/0x7ee4453e4a1aea1b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095543 expref: 12 pid: 8439 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1787739082.772150:0:8444:0:(ofd_dev.c:2721:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffffa073043c0b40 00000100:00100000:0.0:1787739082.780352:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739082.780454:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652557824 00000100:00100000:1.0:1787739082.780464:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652557824 00010000:00010000:1.0:1787739082.780511:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e6a9f713/0x7ee4453e4a1aea1b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095543 expref: 12 pid: 8439 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739082.780629:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1602 00000100:00100000:1.0:1787739082.780640:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:2407:x1874579652557824:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739082.780655:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1787739082.780672:0:6509:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x2c0000401:0x9ca:0x0] updating lvb mtime from disk: 0 -> 1787739082 00002000:00010000:1.0:1787739082.780675:0:6509:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x2c0000401:0x9ca:0x0] updating lvb ctime from disk: 0 -> 1787739082 00010000:00010000:1.0:1787739082.780681:0:6509:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e6a9f713/0x7ee4453e4a1aea1b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095543 expref: 12 pid: 8439 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.780698:0:6509:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e6a9f713/0x7ee4453e4a1aea1b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095543 expref: 12 pid: 8439 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.780721:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e6a9f713/0x7ee4453e4a1aea1b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095543 expref: 12 pid: 8439 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.780739:0:6509:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x9ca:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787739082.780743:0:6509:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x2c0000401:0x9ca:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787739082.780748:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b4cfedb0/0x7ee4453e4a1aecc9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8885 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.780757:0:6509:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b4cfedb0/0x7ee4453e4a1aecc9 lrc: 4/0,1 mode: PW/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8885 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.780784:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739082.780844:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2407:x1874579652557824:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 206us (501us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.780852:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1602 00010000:00010000:0.0:1787739082.780973:0:8885:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b4cfedb0/0x7ee4453e4a1aecc9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8885 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.780983:0:8885:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b4cfedb0/0x7ee4453e4a1aecc9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8885 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.781004:0:8885:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b4cfedb0/0x7ee4453e4a1aecc9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8885 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.781012:0:8885:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b4cfedb0/0x7ee4453e4a1aecc9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8885 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.781026:0:8885:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b4cfedb0/0x7ee4453e4a1aecc9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8885 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.781033:0:8885:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fd400 already unused, calling callback (ffffffffc0cce440) 00010000:00010000:0.0:1787739082.781037:0:8885:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b4cfedb0/0x7ee4453e4a1aecc9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8885 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.781046:0:8885:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b4cfedb0/0x7ee4453e4a1aecc9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8885 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.781057:0:8885:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b4cfedb0/0x7ee4453e4a1aecc9 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8885 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.781064:0:8885:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b4cfedb0/0x7ee4453e4a1aecc9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x9ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8885 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.781109:0:8885:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c1680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2409:x1874579652557568:12345-192.168.203.15@tcp:1:lfs.0 Request processed in 16816us (18099us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.781117:0:8885:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5199 00000100:00100000:1.0:1787739082.791163:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739082.791253:0:6538:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652558080 00000100:00100000:2.0:1787739082.791262:0:6538:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652558080 00000100:00100000:2.0:1787739082.791321:0:6538:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5 00000100:00100000:2.0:1787739082.791329:0:6538:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+27:18317:x1874579652558080:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:2.0:1787739082.791379:0:6538:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739082.791403:0:6538:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0800/0x7ee4453e4a1aea0d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1531:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09553c expref: 27 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.791446:0:6538:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1531:0x0].0x0 (ffffa07403eeb100) 00010000:00010000:2.0:1787739082.791451:0:6538:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739082.791484:0:6538:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1aecd0 lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.791500:0:6538:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1aecd0 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.791510:0:6538:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1aecd0 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739082.791702:0:6538:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000402:0x186:0x0] 00000100:00100000:2.0:1787739082.791755:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07304ce1a40 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665802112:0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787739082.791808:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.791845:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887240 going to sleep for 11 seconds 00000100:00100000:0.0:1787739082.791925:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665802112 00000100:00100000:0.0:1787739082.791931:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665802112 00000100:00100000:0.0:1787739082.791973:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3717 00000100:00100000:0.0:1787739082.791990:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+17:6538:x1874579665802112:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787739082.792096:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+17:6538:x1874579665802112:12345-0@lo:1000:mdt_io00_002.0 Request processed in 116us (286us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.792104:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3717 00000100:00100000:1.0:1787739082.792464:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07304ce1a40 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665802112:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.793057:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665802240:0@lo:403:mdt_io00_002.0 00000100:00100000:1.0:1787739082.793111:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.793143:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435449000 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.793349:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665802240 00000100:00100000:3.0:1787739082.793357:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665802240 00000100:00100000:3.0:1787739082.793437:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3718 00000100:00100000:3.0:1787739082.793447:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+17:6538:x1874579665802240:12345-0@lo:403:mdt_io00_002.0 00000100:00100000:3.0:1787739082.797489:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+17:6538:x1874579665802240:12345-0@lo:403:mdt_io00_002.0 Request processed in 4032us (4367us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.797514:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3718 00000100:00100000:1.0:1787739082.797659:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665802240:0@lo:403:mdt_io00_002.0 00010000:00010000:1.0:1787739082.798334:0:6538:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1aecd7 lrc: 3/0,1 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.798346:0:6538:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1aecd7 lrc: 3/0,1 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.798357:0:6538:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1aecd7 lrc: 3/0,1 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739082.798376:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665802368:0@lo:101:mdt_io00_002.0 00000100:00100000:1.0:1787739082.798441:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.798469:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435449000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.798839:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665802368 00000100:00100000:2.0:1787739082.798847:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665802368 00000100:00100000:2.0:1787739082.798901:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3719 00000100:00100000:2.0:1787739082.798911:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+17:6538:x1874579665802368:12345-0@lo:101:mdt_io00_002.0 00010000:00010000:2.0:1787739082.798928:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739082.798944:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aecde lrc: 2/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aecd7 expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.798970:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aecde lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aecd7 expref: 18 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.798985:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aecde lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aecd7 expref: 18 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.798992:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f000) 00010000:00010000:2.0:1787739082.798995:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739082.798999:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa1800, rc 0) 00000100:00100000:2.0:1787739082.799045:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+18:6538:x1874579665802368:12345-0@lo:101:mdt_io00_002.0 Request processed in 136us (604us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.799052:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3719 00000100:00100000:1.0:1787739082.799625:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665802368:0@lo:101:mdt_io00_002.0 00010000:00010000:1.0:1787739082.799641:0:6538:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1aecd7 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aecde expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.799653:0:6538:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1aecd7 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aecde expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739082.799670:0:6538:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0xcc:0x0] 00010000:00010000:1.0:1787739082.799693:0:6538:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841200/0x7ee4453e4a1aece5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xcc:0x0].0x84832a57 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.799700:0:6538:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841200/0x7ee4453e4a1aece5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xcc:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.799708:0:6538:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841200/0x7ee4453e4a1aece5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xcc:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739082.799721:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665802496:0@lo:101:mdt_io00_002.0 00000100:00100000:1.0:1787739082.799750:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.799776:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435449000 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.800745:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665802496 00000100:00100000:3.0:1787739082.800754:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665802496 00000100:00100000:3.0:1787739082.800812:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3720 00000100:00100000:3.0:1787739082.800820:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+18:6538:x1874579665802496:12345-0@lo:101:mdt_io00_002.0 00010000:00010000:3.0:1787739082.800834:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739082.800851:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1aecec lrc: 2/0,0 mode: --/PW res: [0x240000400:0xcc:0x0].0x84832a57 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aece5 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.800887:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1aecec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aece5 expref: 19 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.800901:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab800/0x7ee4453e4a1aecec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aece5 expref: 19 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.800910:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x84832a57 (ffffa074331c2900) 00010000:00010000:3.0:1787739082.800914:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739082.800917:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab800, rc 0) 00000100:00100000:3.0:1787739082.801036:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+19:6538:x1874579665802496:12345-0@lo:101:mdt_io00_002.0 Request processed in 216us (1283us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.801042:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3720 00000100:00100000:1.0:1787739082.802153:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665802496:0@lo:101:mdt_io00_002.0 00010000:00010000:1.0:1787739082.802176:0:6538:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841200/0x7ee4453e4a1aece5 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0xcc:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aecec expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.802187:0:6538:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841200/0x7ee4453e4a1aece5 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0xcc:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aecec expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739082.802207:0:6538:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0xcc:0x0] 00010000:00010000:1.0:1787739082.802231:0:6538:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1aecf3 lrc: 3/0,1 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.802240:0:6538:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1aecf3 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.802249:0:6538:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1aecf3 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.802260:0:6538:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841600/0x7ee4453e4a1aecfa lrc: 3/0,1 mode: --/PW res: [0x200000400:0xce:0x0].0x84832a57 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.802266:0:6538:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841600/0x7ee4453e4a1aecfa lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.802273:0:6538:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841600/0x7ee4453e4a1aecfa lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.802319:0:6538:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1aed01 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1531:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.802334:0:6538:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841400/0x7ee4453e4a1aecbb lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1531:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09559e expref: 26 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.802342:0:6538:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1aed01 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x1531:0x0].0x0 bits 0x25/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.802371:0:6538:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841400/0x7ee4453e4a1aecbb lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1531:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09559e expref: 26 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.802760:0:6538:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841400/0x7ee4453e4a1aecbb lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1531:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09559e expref: 26 pid: 6523 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739082.802775:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6538:1874579665802624:192.168.203.15@tcp:104: 00000100:00100000:1.0:1787739082.802862:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435449000 going to sleep for 11 seconds 00010000:00010000:1.0:1787739082.806013:0:6538:0:(ldlm_lockd.c:741:ldlm_handle_ast_error()) ### client (nid 192.168.203.15@tcp) returned -22 from blocking AST (req@000000006790524f x1874579665802624) - normal race ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841400/0x7ee4453e4a1aecbb lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1531:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09559e expref: 26 pid: 6523 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.806039:0:6538:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841400/0x7ee4453e4a1aecbb lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1531:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09559e expref: 26 pid: 6523 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.806063:0:6538:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841400/0x7ee4453e4a1aecbb lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1531:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09559e expref: 26 pid: 6523 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739082.806081:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000_UUID:6538:1874579665802624:192.168.203.15@tcp:104: 00010000:00010000:1.0:1787739082.806101:0:6538:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1531:0x0].0x0 (ffffa07403eeb100) 00010000:00010000:1.0:1787739082.806106:0:6538:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1aed01 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1531:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.806117:0:6538:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1aed01 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1531:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x50290000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.806125:0:6538:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1aed01 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1531:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x50290000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.806144:0:6538:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1aed01 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1531:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.806153:0:6538:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1aed01 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1531:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.806159:0:6538:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1aed01 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1531:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739082.806299:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665802752:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.806351:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.806398:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435449840 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.806572:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665802752 00000100:00100000:2.0:1787739082.806579:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665802752 00000100:00100000:2.0:1787739082.806637:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3721 00000100:00100000:2.0:1787739082.806659:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+19:6538:x1874579665802752:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787739082.806780:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+19:6538:x1874579665802752:12345-0@lo:1000:mdt_io00_002.0 Request processed in 133us (428us total) trans 0 rc -2/-2 00000100:00100000:2.0:1787739082.806787:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3721 00000100:00100000:1.0:1787739082.807550:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665802752:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.807610:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665802880:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.807657:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.807683:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435449840 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.807846:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665802880 00000100:00100000:3.0:1787739082.807852:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665802880 00000100:00100000:3.0:1787739082.807907:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3722 00000100:00100000:3.0:1787739082.807916:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+19:6538:x1874579665802880:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787739082.807991:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c12c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+19:6538:x1874579665802880:12345-0@lo:1000:mdt_io00_002.0 Request processed in 76us (344us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.807998:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3722 00000100:00100000:1.0:1787739082.808148:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665802880:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.808245:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665803008:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.808270:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.808297:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435449840 going to sleep for 11 seconds 00000100:00100000:0.0:1787739082.808349:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665803008 00000100:00100000:0.0:1787739082.808354:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665803008 00000100:00100000:0.0:1787739082.808399:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3723 00000100:00100000:0.0:1787739082.808407:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+19:6538:x1874579665803008:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:0.0:1787739082.808500:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+19:6538:x1874579665803008:12345-0@lo:1000:mdt_io00_002.0 Request processed in 94us (229us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.808507:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3723 00000100:00100000:1.0:1787739082.808795:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665803008:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.808829:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665803136:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.808854:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.808887:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435449840 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.808966:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665803136 00000100:00100000:2.0:1787739082.808970:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665803136 00000100:00100000:2.0:1787739082.809006:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3724 00000100:00100000:2.0:1787739082.809013:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+19:6538:x1874579665803136:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:2.0:1787739082.809075:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+19:6538:x1874579665803136:12345-0@lo:1000:mdt_io00_002.0 Request processed in 63us (219us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.809082:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3724 00000100:00100000:1.0:1787739082.809146:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665803136:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.809234:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665803264:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.809263:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.809288:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07435449480 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.809342:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665803264 00000100:00100000:3.0:1787739082.809347:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665803264 00000100:00100000:3.0:1787739082.809370:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3725 00000100:00100000:3.0:1787739082.809393:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6538:x1874579665803264:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787739082.809494:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6538:x1874579665803264:12345-0@lo:1000:mdt_io00_002.0 Request processed in 102us (230us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.809501:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3725 00000100:00100000:1.0:1787739082.809700:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665803264:0@lo:1000:mdt_io00_002.0 00000040:00080000:1.0:1787739082.810031:0:6538:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].409, 1264 off612104 00000040:00080000:1.0:1787739082.810081:0:6538:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].409, 1344 off664008 00000100:00100000:1.0:1787739082.810104:0:6538:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665803392:0@lo:1000:mdt_io00_002.0 00000100:00100000:1.0:1787739082.810135:0:6538:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739082.810152:0:6538:0:(client.c:2715:ptlrpc_set_wait()) set ffffa074354490c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787739082.810280:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665803392 00000100:00100000:3.0:1787739082.810285:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665803392 00000100:00100000:3.0:1787739082.810309:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3726 00000100:00100000:3.0:1787739082.810316:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:6538:x1874579665803392:12345-0@lo:1000:mdt_io00_002.0 00000100:00100000:3.0:1787739082.810609:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:6538:x1874579665803392:12345-0@lo:1000:mdt_io00_002.0 Request processed in 292us (472us total) trans 4294969881 rc 0/0 00000100:00100000:3.0:1787739082.810616:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3726 00000100:00100000:1.0:1787739082.810874:0:6538:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c21c0 pname:cluuid:pid:xid:nid:opc:job mdt_io00_002:lustre-MDT0000-mdtlov_UUID:6538:1874579665803392:0@lo:1000:mdt_io00_002.0 00000004:00080000:1.0:1787739082.810896:0:6538:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07435449840 with version 777 00000004:00080000:3.0:1787739082.811075:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 777 rpc_version 777 00000020:00100000:3.0:1787739082.811099:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:3.0:1787739082.811107:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa0730383d2c0 x1874579665803648/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 3088/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787739082.811122:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730383d2c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665803648:0@lo:1000:osp_up1-0.0 00000100:00100000:3.0:1787739082.811160:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739082.811195:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739082.811245:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665803648 00000100:00100000:0.0:1787739082.811250:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665803648 00000100:00100000:0.0:1787739082.811286:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3727 00000100:00100000:0.0:1787739082.811294:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410b0bc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+19:7740:x1874579665803648:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:0.0:1787739082.812058:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410b0bc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+20:7740:x1874579665803648:12345-0@lo:1000:osp_up1-0.0 Request processed in 763us (894us total) trans 4294968493 rc 0/0 00000100:00100000:0.0:1787739082.812066:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3727 00000100:00100000:3.0:1787739082.812348:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730383d2c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665803648:0@lo:1000:osp_up1-0.0 00000004:00010000:1.0:1787739082.812492:0:6538:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ccdf9fe6 reply state 000000001ee55f7f transno 4294983527 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1aed01 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1531:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.812515:0:6538:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840e00/0x7ee4453e4a1aed01 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x1531:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.812534:0:6538:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1531:0x0].0x0 (ffffa07403eeb100) 00010000:00010000:1.0:1787739082.812538:0:6538:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 25 00000100:00100000:1.0:1787739082.812543:0:6538:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787739082.812550:0:6538:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1aecf3 lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.812562:0:6538:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1aecf3 lrc: 2/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.812569:0:6538:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1aecf3 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.812574:0:6538:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840c00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739082.812578:0:6538:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1aecf3 lrc: 3/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.812586:0:6538:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1aecf3 lrc: 4/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.812595:0:6538:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa0730713a100) 00010000:00010000:1.0:1787739082.812598:0:6538:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.812601:0:6538:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1aecf3 lrc: 2/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.812607:0:6538:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1aecf3 lrc: 0/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739082.812627:0:6538:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ccdf9fe6 reply state 000000001ee55f7f transno 4294983527 ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841600/0x7ee4453e4a1aecfa lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.812635:0:6538:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841600/0x7ee4453e4a1aecfa lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xce:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.812641:0:6538:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x84832a57 (ffffa0730713ac00) 00010000:00010000:1.0:1787739082.812643:0:6538:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739082.812645:0:6538:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00080000:1.0:1787739082.812772:0:6538:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0xcc:0x0] 00010000:00010000:1.0:1787739082.812780:0:6538:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1aecd7 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aecde expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.812788:0:6538:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1aecd7 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aecde expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.812794:0:6538:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1aecd7 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aecde expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.812893:0:6538:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739082.812899:0:6538:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0xcc:0x0] 00010000:00010000:1.0:1787739082.812904:0:6538:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841200/0x7ee4453e4a1aece5 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0xcc:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aecec expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.812911:0:6538:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841200/0x7ee4453e4a1aece5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0x84832a57 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aecec expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739082.812923:0:6538:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000402:0x186:0x0] 00010000:00010000:1.0:1787739082.812930:0:6538:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1aecd0 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.812935:0:6538:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1aecd0 lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.812941:0:6538:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1aecd0 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.812946:0:6538:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739082.812950:0:6538:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1aecd0 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.812960:0:6538:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1aecd0 lrc: 4/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.812968:0:6538:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffffa07403a6f300) 00010000:00010000:1.0:1787739082.812970:0:6538:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.812974:0:6538:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1aecd0 lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.812979:0:6538:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1aecd0 lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6538 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739082.813067:0:6538:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c0780 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+27:18317:x1874579652558080:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 21736us (21909us total) trans 4294983527 rc 0/0 00000100:00100000:1.0:1787739082.813075:0:6538:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5 00010000:00010000:2.0:1787739082.815150:0:6512:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739082.815163:0:6512:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1aecd7 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aecde expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.815174:0:6512:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306841000 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739082.815181:0:6512:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1aecd7 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aecde expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739082.815191:0:6512:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1aecd7 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aecde expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.815236:0:6512:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1aecd7 lrc: 2/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aecde expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.815243:0:6512:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:3.0:1787739082.815319:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f79a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665803776:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1787739082.815392:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739082.815656:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665803776 00000100:00100000:0.0:1787739082.815664:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665803776 00000100:00100000:0.0:1787739082.815722:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1439 00000100:00100000:0.0:1787739082.815731:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383ed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+20:3650:x1874579665803776:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739082.815744:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1787739082.815762:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aecde lrc: 0/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aecd7 expref: 20 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.815778:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f000) 00010000:00010000:0.0:1787739082.815788:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739082.815902:0:6512:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1aecd7 lrc: 1/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aecde expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.815915:0:6512:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa07306841000/0x7ee4453e4a1aecd7 lrc: 0/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aecde expref: -99 pid: 6538 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.815932:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739082.815934:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739082.815936:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739082.819310:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383ed00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:lustre-MDT0000-mdtlov_UUID+19:3650:x1874579665803776:12345-0@lo:103:ldlm_bl.0 Request processed in 3570us (3908us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.819339:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1439 00000100:00100000:3.0:1787739082.820829:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f79a40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665803776:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1787739082.821037:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739082.821226:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652558208 00000100:00100000:2.0:1787739082.821233:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652558208 00000100:00100000:2.0:1787739082.821285:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23721 00000100:00100000:2.0:1787739082.821295:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be4f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+27:18317:x1874579652558208:12345-192.168.203.15@tcp:34:lfs.0 00010000:00010000:2.0:1787739082.821386:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0800/0x7ee4453e4a1aed08 lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.821403:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0800/0x7ee4453e4a1aed08 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.821461:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0800/0x7ee4453e4a1aed08 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.821477:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aed0f lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.821484:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aed0f lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.821492:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aed0f lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.821868:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aed16 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x152c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.821887:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aed16 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.821896:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aed16 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739082.821947:0:6525:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aed16 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.821961:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0800/0x7ee4453e4a1aed08 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.821968:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0800/0x7ee4453e4a1aed08 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.821973:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0800/0x7ee4453e4a1aed08 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.821979:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0800/0x7ee4453e4a1aed08 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.821985:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739082.821988:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0800/0x7ee4453e4a1aed08 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.821995:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0800/0x7ee4453e4a1aed08 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.822006:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:2.0:1787739082.822009:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739082.822013:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0800/0x7ee4453e4a1aed08 lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.822019:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0800/0x7ee4453e4a1aed08 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.822037:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aed0f lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.822044:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aed0f lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.822049:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aed0f lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.822054:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739082.822056:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aed0f lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.822061:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aed0f lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.822068:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0xdcc87686 (ffffa07403a6f100) 00010000:00010000:2.0:1787739082.822070:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739082.822073:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aed0f lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.822078:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aed0f lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.822087:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aed16 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x152c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.822093:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aed16 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x152c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.822099:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aed16 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x152c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.822104:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739082.822107:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aed16 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x152c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.822112:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aed16 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x152c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.822118:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152c:0x0].0x0 (ffffa07433235100) 00010000:00010000:2.0:1787739082.822121:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739082.822123:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aed16 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x152c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.822129:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aed16 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x152c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739082.822199:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be4f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+27:18317:x1874579652558208:12345-192.168.203.15@tcp:34:lfs.0 Request processed in 905us (1163us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.822207:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23721 00000100:00100000:3.0:1787739082.825739:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.826100:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652558464 00000100:00100000:0.0:1787739082.826111:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652558464 00000100:00100000:0.0:1787739082.826173:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23722 00000100:00100000:0.0:1787739082.826185:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+27:18317:x1874579652558464:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739082.826205:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739082.826225:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fda00/0x7ee4453e4a1aed1d lrc: 2/0,0 mode: --/CR res: [0x200000402:0x152c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0955a5 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.826347:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1aed24 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x152c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.826363:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1aed24 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x152c:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.826376:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1aed24 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x152c:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739082.826388:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x152c:0x0]: returning size 2 00010000:00010000:0.0:1787739082.826804:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fda00/0x7ee4453e4a1aed1d lrc: 0/0,0 mode: --/CR res: [0x200000402:0x152c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955a5 expref: 30 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.826827:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1aed24 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x152c:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955a5 expref: 29 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.826836:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152c:0x0].0x0 (ffffa07433235100) 00010000:00010000:0.0:1787739082.826839:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1787739082.826844:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fd600, rc 0) 00000100:00100000:0.0:1787739082.826921:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383f0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+29:18317:x1874579652558464:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 738us (1190us total) trans 4294983528 rc 0/0 00000100:00100000:0.0:1787739082.826930:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23722 00000100:00100000:3.0:1787739082.832316:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739082.832453:0:14660:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652558592 00000100:00100000:2.0:1787739082.832463:0:14660:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652558592 00000100:00100000:2.0:1787739082.832517:0:14660:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5200 00000100:00100000:2.0:1787739082.832537:0:14660:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:2406:x1874579652558592:12345-192.168.203.15@tcp:1:lfs.0 00010000:00010000:2.0:1787739082.832596:0:14660:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d1d739a/0x7ee4453e4a1aed32 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14660 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.832618:0:14660:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000000e0be3a3/0x7ee4453e4a1ae949 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954fd expref: 9 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.832630:0:14660:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000401:0x9c8:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1787739082.832634:0:14660:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9c8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1787739082.832639:0:14660:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d1d739a/0x7ee4453e4a1aed32 lrc: 4/0,1 mode: --/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14660 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.832684:0:14660:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000000e0be3a3/0x7ee4453e4a1ae949 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954fd expref: 9 pid: 8436 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.832698:0:14660:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000000e0be3a3/0x7ee4453e4a1ae949 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954fd expref: 9 pid: 8436 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787739082.832711:0:14660:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f783c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:14660:1874579665803904:192.168.203.15@tcp:104: 00000100:00100000:2.0:1787739082.832783:0:14660:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887900 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.838094:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739082.838172:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652558720 00000100:00100000:3.0:1787739082.838181:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652558720 00002000:00100000:3.0:1787739082.838232:0:8442:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x9c8:0x0] (0->1) req@ffffa07305f78b40 x1874579652558720/t0(0) o4->e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2@192.168.203.15@tcp:263/0 lens 496/0 e 0 to 0 dl 1787739093 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787739082.838243:0:8442:0:(ofd_dev.c:2572:ofd_prolong_extent_locks()) Prolong locks for req ffffa07305f78b40 with x1874579652558720 ext(0->1) 00010000:00010000:3.0:1787739082.838251:0:8442:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000000e0be3a3/0x7ee4453e4a1ae949 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954fd expref: 10 pid: 8436 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.838266:0:8442:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000000e0be3a3/0x7ee4453e4a1ae949 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954fd expref: 10 pid: 8436 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787739082.838274:0:8442:0:(ofd_dev.c:2721:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffffa07305f78b40 00000100:00100000:3.0:1787739082.838308:0:8442:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5 00000100:00100000:3.0:1787739082.838317:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f78b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652558720:12345-192.168.203.15@tcp:4:bash.0 00000100:00100000:3.0:1787739082.838701:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f78b40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2408:x1874579652558720:12345-192.168.203.15@tcp:4:bash.0 Request processed in 384us (617us total) trans 0 rc 0/0 00000100:00100000:3.0:1787739082.838708:0:8442:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5 00002000:00100000:3.0:1787739082.838716:0:8442:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x9c8:0x0] (0->1) req@ffffa07305f78b40 x1874579652558720/t0(0) o4->e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2@192.168.203.15@tcp:263/0 lens 496/448 e 0 to 0 dl 1787739093 ref 1 fl Complete:H/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1787739082.838724:0:8442:0:(ofd_dev.c:2572:ofd_prolong_extent_locks()) Prolong locks for req ffffa07305f78b40 with x1874579652558720 ext(0->1) 00010000:00010000:3.0:1787739082.838730:0:8442:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000000e0be3a3/0x7ee4453e4a1ae949 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954fd expref: 11 pid: 8436 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.838743:0:8442:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000000e0be3a3/0x7ee4453e4a1ae949 lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954fd expref: 11 pid: 8436 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1787739082.838750:0:8442:0:(ofd_dev.c:2721:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffffa07305f78b40 00000100:00100000:0.0:1787739082.839321:0:14660:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f783c0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:14660:1874579665803904:192.168.203.15@tcp:104: 00010000:00010000:0.0:1787739082.839348:0:14660:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000008d1d739a/0x7ee4453e4a1aed32 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14660 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739082.844774:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739082.844965:0:9525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652558848 00000100:00100000:1.0:1787739082.844975:0:9525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652558848 00010000:00010000:1.0:1787739082.845014:0:9525:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000e0be3a3/0x7ee4453e4a1ae949 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954fd expref: 11 pid: 8436 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739082.845056:0:9525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1603 00000100:00100000:1.0:1787739082.845066:0:9525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383f840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:2408:x1874579652558848:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787739082.845082:0:9525:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1787739082.845099:0:9525:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x9c8:0x0] updating lvb mtime from disk: 0 -> 1787739082 00002000:00010000:1.0:1787739082.845102:0:9525:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x9c8:0x0] updating lvb ctime from disk: 0 -> 1787739082 00010000:00010000:1.0:1787739082.845107:0:9525:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000000e0be3a3/0x7ee4453e4a1ae949 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954fd expref: 11 pid: 8436 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.845126:0:9525:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000000e0be3a3/0x7ee4453e4a1ae949 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954fd expref: 11 pid: 8436 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.845149:0:9525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000e0be3a3/0x7ee4453e4a1ae949 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954fd expref: 11 pid: 8436 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.845165:0:9525:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9c8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787739082.845169:0:9525:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9c8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1787739082.845174:0:9525:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008d1d739a/0x7ee4453e4a1aed32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 14660 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.845182:0:9525:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d1d739a/0x7ee4453e4a1aed32 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 14660 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.845217:0:9525:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1787739082.845273:0:9525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383f840 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652558848:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 208us (506us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.845281:0:9525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1603 00010000:00010000:0.0:1787739082.845861:0:14660:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000008d1d739a/0x7ee4453e4a1aed32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14660 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.845874:0:14660:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d1d739a/0x7ee4453e4a1aed32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14660 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.845909:0:14660:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d1d739a/0x7ee4453e4a1aed32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14660 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.845918:0:14660:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d1d739a/0x7ee4453e4a1aed32 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14660 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.845937:0:14660:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d1d739a/0x7ee4453e4a1aed32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14660 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.845946:0:14660:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0000 already unused, calling callback (ffffffffc0cce440) 00010000:00010000:0.0:1787739082.845950:0:14660:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d1d739a/0x7ee4453e4a1aed32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14660 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.845964:0:14660:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d1d739a/0x7ee4453e4a1aed32 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14660 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.845977:0:14660:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d1d739a/0x7ee4453e4a1aed32 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14660 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.845985:0:14660:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d1d739a/0x7ee4453e4a1aed32 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14660 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739082.846051:0:14660:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383e580 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:2406:x1874579652558592:12345-192.168.203.15@tcp:1:lfs.0 Request processed in 13523us (13742us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.846060:0:14660:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5200 00000100:00100000:3.0:1787739082.858379:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739082.858498:0:6536:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652559104 00000100:00100000:2.0:1787739082.858508:0:6536:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652559104 00000100:00100000:2.0:1787739082.858585:0:6536:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 6 00000100:00100000:2.0:1787739082.858595:0:6536:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+29:18317:x1874579652559104:12345-192.168.203.15@tcp:36:lfs.0 00010000:00010000:2.0:1787739082.858645:0:6536:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739082.858678:0:6536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840600/0x7ee4453e4a1ae93b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x152c:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0954f6 expref: 29 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.858695:0:6536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152c:0x0].0x0 (ffffa07433235100) 00010000:00010000:2.0:1787739082.858700:0:6536:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1787739082.858725:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aed39 lrc: 3/0,1 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.858738:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aed39 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.858748:0:6536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aed39 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1787739082.858764:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000402:0x186:0x0] 00000100:00100000:2.0:1787739082.858812:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f78780 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665804032:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.858913:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.858947:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787739082.859003:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665804032 00000100:00100000:0.0:1787739082.859008:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665804032 00000100:00100000:0.0:1787739082.859052:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3728 00000100:00100000:0.0:1787739082.859060:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+19:6536:x1874579665804032:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:0.0:1787739082.859164:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7a940 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+19:6536:x1874579665804032:12345-0@lo:1000:mdt_io00_000.0 Request processed in 105us (250us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.859171:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3728 00000100:00100000:2.0:1787739082.859391:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f78780 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665804032:0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.860050:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f78780 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665804160:0@lo:403:mdt_io00_000.0 00000100:00100000:2.0:1787739082.860096:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.860118:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887900 going to sleep for 11 seconds 00000100:00100000:0.0:1787739082.860333:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665804160 00000100:00100000:0.0:1787739082.860339:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665804160 00000100:00100000:0.0:1787739082.860382:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3729 00000100:00100000:0.0:1787739082.860390:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+19:6536:x1874579665804160:12345-0@lo:403:mdt_io00_000.0 00000100:00100000:1.0:1787739082.865966:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+19:6536:x1874579665804160:12345-0@lo:403:mdt_io00_000.0 Request processed in 5567us (5859us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.865981:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3729 00000100:00100000:3.0:1787739082.866169:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f78780 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665804160:0@lo:403:mdt_io00_000.0 00010000:00010000:3.0:1787739082.867054:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1aed40 lrc: 3/0,1 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.867066:0:6536:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1aed40 lrc: 3/0,1 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.867077:0:6536:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1aed40 lrc: 3/0,1 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739082.867098:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561a940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665804288:0@lo:101:mdt_io00_000.0 00000100:00100000:3.0:1787739082.867146:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739082.867186:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.870960:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665804288 00000100:00100000:1.0:1787739082.870975:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665804288 00000100:00100000:1.0:1787739082.871045:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3730 00000100:00100000:1.0:1787739082.871056:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305618000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+19:6536:x1874579665804288:12345-0@lo:101:mdt_io00_000.0 00010000:00010000:1.0:1787739082.871078:0:6529:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739082.871104:0:6529:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aed47 lrc: 2/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aed40 expref: -99 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.871134:0:6529:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aed47 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aed40 expref: 20 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.871150:0:6529:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aed47 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aed40 expref: 20 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.871159:0:6529:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa0730713a100) 00010000:00010000:1.0:1787739082.871163:0:6529:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.871166:0:6529:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306840800, rc 0) 00000100:00100000:1.0:1787739082.871239:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305618000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+20:6536:x1874579665804288:12345-0@lo:101:mdt_io00_000.0 Request processed in 186us (4093us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.871246:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3730 00000100:00100000:3.0:1787739082.871785:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561a940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665804288:0@lo:101:mdt_io00_000.0 00010000:00010000:3.0:1787739082.871805:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1aed40 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aed47 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.871817:0:6536:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1aed40 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aed47 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739082.871839:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0xcc:0x0] 00010000:00010000:3.0:1787739082.871872:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1aed4e lrc: 3/0,1 mode: --/PW res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.871892:0:6536:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1aed4e lrc: 3/0,1 mode: --/PW res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.871902:0:6536:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1aed4e lrc: 3/0,1 mode: --/PW res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x1000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739082.871920:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561a940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665804416:0@lo:101:mdt_io00_000.0 00000100:00100000:3.0:1787739082.871960:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739082.871982:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.872053:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665804416 00000100:00100000:1.0:1787739082.872058:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665804416 00000100:00100000:1.0:1787739082.872096:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3731 00000100:00100000:1.0:1787739082.872103:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305619a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+20:6536:x1874579665804416:12345-0@lo:101:mdt_io00_000.0 00010000:00010000:1.0:1787739082.872113:0:12410:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739082.872126:0:12410:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1aed55 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7ee4453e4a1aed4e expref: -99 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.872142:0:12410:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1aed55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 0@lo remote: 0x7ee4453e4a1aed4e expref: 21 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.872153:0:12410:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1aed55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aed4e expref: 21 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.872161:0:12410:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0xdcc87686 (ffffa0730713a500) 00010000:00010000:1.0:1787739082.872164:0:12410:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.872166:0:12410:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306840000, rc 0) 00000100:00100000:1.0:1787739082.872196:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305619a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+21:6536:x1874579665804416:12345-0@lo:101:mdt_io00_000.0 Request processed in 93us (236us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.872202:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3731 00000100:00100000:3.0:1787739082.872374:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561a940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665804416:0@lo:101:mdt_io00_000.0 00010000:00010000:3.0:1787739082.872386:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1aed4e lrc: 4/0,1 mode: PW/PW res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aed55 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.872395:0:6536:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1aed4e lrc: 4/0,1 mode: PW/PW res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aed55 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1787739082.872408:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0xcc:0x0] 00010000:00010000:3.0:1787739082.872451:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aed5c lrc: 3/0,1 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.872461:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aed5c lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.872468:0:6536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aed5c lrc: 3/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.872481:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aed63 lrc: 3/0,1 mode: --/PW res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.872491:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aed63 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.872497:0:6536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aed63 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.872560:0:6536:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aed6a lrc: 3/0,1 mode: --/EX res: [0x200000402:0x152c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.872573:0:6536:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1aed24 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x152c:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955a5 expref: 28 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.872581:0:6536:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aed6a lrc: 4/0,1 mode: --/EX res: [0x200000402:0x152c:0x0].0x0 bits 0x25/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.872603:0:6536:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1aed24 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x152c:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955a5 expref: 28 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.872616:0:6536:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1aed24 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x152c:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955a5 expref: 28 pid: 6524 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739082.872627:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561a940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000_UUID:6536:1874579665804544:192.168.203.15@tcp:104: 00000100:00100000:3.0:1787739082.872842:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234b40 going to sleep for 11 seconds 00010000:00010000:3.0:1787739082.875826:0:6536:0:(ldlm_lockd.c:741:ldlm_handle_ast_error()) ### client (nid 192.168.203.15@tcp) returned -22 from blocking AST (req@00000000a17a18f0 x1874579665804544) - normal race ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1aed24 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x152c:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955a5 expref: 28 pid: 6524 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.875852:0:6536:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1aed24 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x152c:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955a5 expref: 28 pid: 6524 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.875890:0:6536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd600/0x7ee4453e4a1aed24 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x152c:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955a5 expref: 28 pid: 6524 timeout: 769 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739082.875907:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561a940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000_UUID:6536:1874579665804544:192.168.203.15@tcp:104: 00010000:00010000:3.0:1787739082.875928:0:6536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152c:0x0].0x0 (ffffa07433235100) 00010000:00010000:3.0:1787739082.875934:0:6536:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aed6a lrc: 3/0,1 mode: --/EX res: [0x200000402:0x152c:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.875944:0:6536:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aed6a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x152c:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x50290000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.875952:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aed6a lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x152c:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x50290000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.875972:0:6536:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aed6a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x152c:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.875982:0:6536:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aed6a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x152c:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739082.875988:0:6536:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aed6a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x152c:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739082.876118:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561a940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665804672:0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.876168:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739082.876242:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234000 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.876281:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665804672 00000100:00100000:1.0:1787739082.876288:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665804672 00000100:00100000:1.0:1787739082.876378:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3732 00000100:00100000:1.0:1787739082.876387:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+21:6536:x1874579665804672:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:1.0:1787739082.876784:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+21:6536:x1874579665804672:12345-0@lo:1000:mdt_io00_000.0 Request processed in 397us (615us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739082.876792:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3732 00000100:00100000:3.0:1787739082.876850:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561a940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665804672:0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.876905:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561a940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665804800:0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.876931:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739082.877027:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.877102:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665804800 00000100:00100000:2.0:1787739082.877108:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665804800 00000100:00100000:2.0:1787739082.877137:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3733 00000100:00100000:2.0:1787739082.877145:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+21:6536:x1874579665804800:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.877211:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+21:6536:x1874579665804800:12345-0@lo:1000:mdt_io00_000.0 Request processed in 66us (278us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.877217:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3733 00000100:00100000:3.0:1787739082.877283:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561a940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665804800:0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.877372:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561a940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665804928:0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.877397:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739082.877440:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.877487:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665804928 00000100:00100000:2.0:1787739082.877492:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665804928 00000100:00100000:2.0:1787739082.877511:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3734 00000100:00100000:2.0:1787739082.877517:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073056183c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+21:6536:x1874579665804928:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.877595:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073056183c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+21:6536:x1874579665804928:12345-0@lo:1000:mdt_io00_000.0 Request processed in 77us (196us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.877602:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3734 00000100:00100000:3.0:1787739082.877725:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561a940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665804928:0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.877759:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561a940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665805056:0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.877782:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739082.877805:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234000 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.877909:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665805056 00000100:00100000:2.0:1787739082.877913:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665805056 00000100:00100000:2.0:1787739082.877932:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3735 00000100:00100000:2.0:1787739082.877939:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+21:6536:x1874579665805056:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.877993:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561ad00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+21:6536:x1874579665805056:12345-0@lo:1000:mdt_io00_000.0 Request processed in 54us (209us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.877999:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3735 00000100:00100000:3.0:1787739082.878059:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561a940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665805056:0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.878137:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561a940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665805184:0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.878162:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739082.878186:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234540 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.878229:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665805184 00000100:00100000:2.0:1787739082.878234:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665805184 00000100:00100000:2.0:1787739082.878254:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3736 00000100:00100000:2.0:1787739082.878259:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305619680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6536:x1874579665805184:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.878322:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305619680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6536:x1874579665805184:12345-0@lo:1000:mdt_io00_000.0 Request processed in 61us (158us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.878329:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3736 00000100:00100000:3.0:1787739082.878388:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561a940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665805184:0@lo:1000:mdt_io00_000.0 00000040:00080000:3.0:1787739082.878735:0:6536:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x200000400:0x3:0x0].410, 1264 off613368 00000040:00080000:3.0:1787739082.878788:0:6536:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x240000401:0x3:0x0].410, 1344 off665352 00000100:00100000:3.0:1787739082.878811:0:6536:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0730561a940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665805312:0@lo:1000:mdt_io00_000.0 00000100:00100000:3.0:1787739082.878838:0:6536:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739082.878866:0:6536:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787739082.879044:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665805312 00000100:00100000:2.0:1787739082.879049:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665805312 00000100:00100000:2.0:1787739082.879074:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3737 00000100:00100000:2.0:1787739082.879080:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305619e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+7:6536:x1874579665805312:12345-0@lo:1000:mdt_io00_000.0 00000100:00100000:2.0:1787739082.879186:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305619e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+8:6536:x1874579665805312:12345-0@lo:1000:mdt_io00_000.0 Request processed in 106us (348us total) trans 4294969883 rc 0/0 00000100:00100000:2.0:1787739082.879203:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3737 00000100:00100000:3.0:1787739082.879377:0:6536:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0730561a940 pname:cluuid:pid:xid:nid:opc:job mdt_io00_000:lustre-MDT0000-mdtlov_UUID:6536:1874579665805312:0@lo:1000:mdt_io00_000.0 00000004:00080000:3.0:1787739082.879387:0:6536:0:(osp_trans.c:1595:osp_trans_stop()) lustre-MDT0001-osp-MDT0000: add oth ffffa07306234240 with version 778 00000004:00080000:2.0:1787739082.879465:0:7740:0:(osp_trans.c:1336:osp_get_next_request()) ou ffffa07435539300 version 778 rpc_version 778 00000020:00100000:2.0:1787739082.879487:0:7740:0:(genops.c:2423:obd_get_mod_rpc_slot()) lustre-MDT0001-osp-MDT0000: modify RPC slot 1 is allocated opc 1000, max 8 00000100:00100000:2.0:1787739082.879495:0:7740:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffffa07305f792c0 x1874579665805568/t0(0) o1000->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 3088/4320 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787739082.879511:0:7740:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f792c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665805568:0@lo:1000:osp_up1-0.0 00000100:00100000:2.0:1787739082.879606:0:7740:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.879634:0:7740:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306887e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787739082.879768:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665805568 00000100:00100000:1.0:1787739082.879774:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665805568 00000100:00100000:1.0:1787739082.879805:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3738 00000100:00100000:1.0:1787739082.879812:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+21:7740:x1874579665805568:12345-0@lo:1000:osp_up1-0.0 00000100:00100000:1.0:1787739082.880131:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7b0c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+22:7740:x1874579665805568:12345-0@lo:1000:osp_up1-0.0 Request processed in 318us (524us total) trans 4294968494 rc 0/0 00000100:00100000:1.0:1787739082.880138:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3738 00000100:00100000:2.0:1787739082.880244:0:7740:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f792c0 pname:cluuid:pid:xid:nid:opc:job osp_up1-0:lustre-MDT0000-mdtlov_UUID:7740:1874579665805568:0@lo:1000:osp_up1-0.0 00000004:00010000:1.0:1787739082.880609:0:6536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005e447b06 reply state 00000000f11579cf transno 4294983529 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aed6a lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x152c:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.880632:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa000/0x7ee4453e4a1aed6a lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x152c:0x0].0x0 bits 0x25/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.880658:0:6536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x152c:0x0].0x0 (ffffa07433235100) 00010000:00010000:1.0:1787739082.880662:0:6536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 25 00000100:00100000:1.0:1787739082.880666:0:6536:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1787739082.880675:0:6536:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aed5c lrc: 4/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.880683:0:6536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aed5c lrc: 4/0,1 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.880689:0:6536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aed5c lrc: 3/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.880696:0:6536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aed5c lrc: 4/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.880702:0:6536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739082.880707:0:6536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aed5c lrc: 4/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.880714:0:6536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aed5c lrc: 5/0,0 mode: CW/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.880724:0:6536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa074331c2f00) 00010000:00010000:1.0:1787739082.880727:0:6536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.880730:0:6536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aed5c lrc: 3/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.880737:0:6536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aed5c lrc: 0/0,0 mode: --/CW res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1787739082.880755:0:6536:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005e447b06 reply state 00000000f11579cf transno 4294983529 ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aed63 lrc: 3/0,1 mode: PW/PW res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.880764:0:6536:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aac00/0x7ee4453e4a1aed63 lrc: 3/0,1 mode: TXN/TXN res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.880770:0:6536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0xdcc87686 (ffffa074331c2b00) 00010000:00010000:1.0:1787739082.880773:0:6536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1787739082.880775:0:6536:0:(service.c:179:ptlrpc_save_lock()) nlocks 1 00000004:00080000:1.0:1787739082.880779:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0xcc:0x0] 00010000:00010000:1.0:1787739082.880787:0:6536:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1aed40 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x7ee4453e4a1aed47 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.880795:0:6536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1aed40 lrc: 4/0,1 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aed47 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.880803:0:6536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1aed40 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000400000000 nid: local remote: 0x7ee4453e4a1aed47 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.880831:0:6536:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1787739082.880836:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000400:0xcc:0x0] 00010000:00010000:1.0:1787739082.880841:0:6536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1aed4e lrc: 4/0,1 mode: PW/PW res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aed55 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.880848:0:6536:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1aed4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000000000000 nid: local remote: 0x7ee4453e4a1aed55 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1787739082.880861:0:6536:0:(osp_object.c:1402:osp_invalidate()) Invalidate osp_object [0x240000402:0x186:0x0] 00010000:00010000:1.0:1787739082.880869:0:6536:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aed39 lrc: 3/0,1 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.880886:0:6536:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aed39 lrc: 2/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.880893:0:6536:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aed39 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.880898:0:6536:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739082.880901:0:6536:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aed39 lrc: 3/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.880908:0:6536:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aed39 lrc: 4/0,0 mode: EX/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.880916:0:6536:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000004:0x1:0x0].0x0 (ffffa07403a6f100) 00010000:00010000:1.0:1787739082.880918:0:6536:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.880922:0:6536:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aed39 lrc: 2/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.880928:0:6536:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aed39 lrc: 0/0,0 mode: --/EX res: [0x200000004:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6536 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787739082.881006:0:6536:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+29:18317:x1874579652559104:12345-192.168.203.15@tcp:36:lfs.0 Request processed in 22411us (22634us total) trans 4294983529 rc 0/0 00000100:00100000:1.0:1787739082.881015:0:6536:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 6 00010000:00010000:2.0:1787739082.881931:0:6511:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787739082.881940:0:6511:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1aed40 lrc: 2/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000400000000 nid: local remote: 0x7ee4453e4a1aed47 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.881949:0:6511:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aaa00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:2.0:1787739082.881954:0:6511:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1aed40 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008400000000 nid: local remote: 0x7ee4453e4a1aed47 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1787739082.881968:0:6511:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1aed40 lrc: 3/0,0 mode: CW/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009400000000 nid: local remote: 0x7ee4453e4a1aed47 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.882005:0:6511:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1aed40 lrc: 2/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aed47 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.882012:0:6511:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787739082.882047:0:6511:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1aed40 lrc: 1/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aed47 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.882055:0:6511:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069aaa00/0x7ee4453e4a1aed40 lrc: 0/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809400000020 nid: local remote: 0x7ee4453e4a1aed47 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.882072:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739082.882075:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787739082.882077:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787739082.882150:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665805696:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739082.882200:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739082.882267:0:6508:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665805696 00000100:00100000:2.0:1787739082.882274:0:6508:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665805696 00000100:00100000:2.0:1787739082.882322:0:6508:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1440 00000100:00100000:2.0:1787739082.882331:0:6508:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07410ba0450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+22:3651:x1874579665805696:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1787739082.882342:0:6508:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739082.882359:0:6508:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aed47 lrc: 0/0,0 mode: --/CW res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0x7ee4453e4a1aed40 expref: 22 pid: 6529 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.882370:0:6508:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa0730713a100) 00010000:00010000:2.0:1787739082.882378:0:6508:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739082.882440:0:6508:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07410ba0450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000-mdtlov_UUID+21:3651:x1874579665805696:12345-0@lo:103:ldlm_bl.0 Request processed in 110us (239us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.882447:0:6508:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1440 00000100:00100000:2.0:1787739082.882504:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305f7bc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665805696:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1787739082.887021:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739082.887130:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652559360 00000100:00100000:2.0:1787739082.887139:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652559360 00000100:00100000:2.0:1787739082.887198:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5084 00000100:00100000:2.0:1787739082.887208:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+7:18317:x1874579652559360:12345-192.168.203.15@tcp:35:lfs.0 00000100:00100000:2.0:1787739082.887478:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:18317:x1874579652559360:12345-192.168.203.15@tcp:35:lfs.0 Request processed in 271us (465us total) trans 4294968495 rc 0/0 00000100:00100000:2.0:1787739082.887486:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5084 00000100:00100000:0.0:1787739082.908510:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739082.908763:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652559616 00000100:00100000:1.0:1787739082.908772:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652559616 00000100:00100000:1.0:1787739082.908832:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23723 00000100:00100000:1.0:1787739082.908843:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be52c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+29:18318:x1874579652559616:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:1.0:1787739082.908871:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739082.908901:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840600/0x7ee4453e4a1aed71 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0955ac expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.908976:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841400/0x7ee4453e4a1aed78 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.908991:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841400/0x7ee4453e4a1aed78 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909003:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841400/0x7ee4453e4a1aed78 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909020:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1aed7f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909027:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1aed7f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909034:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1aed7f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909100:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aed86 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x186:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909109:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aed86 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x186:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909117:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aed86 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x186:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909130:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841400/0x7ee4453e4a1aed78 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909139:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841400/0x7ee4453e4a1aed78 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909145:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841400/0x7ee4453e4a1aed78 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909151:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841400/0x7ee4453e4a1aed78 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909157:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306841400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739082.909162:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841400/0x7ee4453e4a1aed78 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909168:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841400/0x7ee4453e4a1aed78 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909179:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa07401795300) 00010000:00010000:1.0:1787739082.909181:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.909185:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841400/0x7ee4453e4a1aed78 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909191:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841400/0x7ee4453e4a1aed78 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909204:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1aed7f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909211:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1aed7f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909217:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1aed7f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909223:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840c00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739082.909225:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1aed7f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909231:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1aed7f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909239:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xf5d6a2c1 (ffffa0730713a000) 00010000:00010000:1.0:1787739082.909241:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739082.909244:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1aed7f lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909250:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840c00/0x7ee4453e4a1aed7f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf5d6a2c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909283:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840600/0x7ee4453e4a1aed71 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955ac expref: 31 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909296:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aed86 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x186:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955ac expref: 30 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.909303:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x186:0x0].0x0 (ffffa0730713a200) 00010000:00010000:1.0:1787739082.909306:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739082.909308:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841000, rc 0) 00000100:00100000:1.0:1787739082.909375:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be52c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18318:x1874579652559616:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 533us (874us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739082.909382:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23723 00000100:00100000:1.0:1787739082.917504:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739082.917633:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739082.917715:0:10340:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652560000 00000100:00100000:2.0:1787739082.917725:0:10340:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652560000 00000100:00100000:2.0:1787739082.917788:0:10340:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 1441 00000100:00100000:2.0:1787739082.917798:0:10340:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c1e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:2407:x1874579652560000:12345-192.168.203.15@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787739082.917815:0:10340:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1787739082.917841:0:10340:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aed86 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x186:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955ac expref: 30 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.917859:0:10340:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x186:0x0].0x0 (ffffa0730713a200) 00010000:00010000:2.0:1787739082.917864:0:10340:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1787739082.917934:0:10340:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c1e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+29:2407:x1874579652560000:12345-192.168.203.15@tcp:103:ldlm_bl.0 Request processed in 137us (300us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.917942:0:10340:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 1441 00000100:00100000:0.0:1787739082.918613:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652559872 00000100:00100000:0.0:1787739082.918622:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652559872 00000100:00100000:0.0:1787739082.918676:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23724 00000100:00100000:0.0:1787739082.918684:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+8:18318:x1874579652559872:12345-192.168.203.15@tcp:101:lfs.0 00010000:00010000:0.0:1787739082.918704:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739082.918725:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd600/0x7ee4453e4a1aed8d lrc: 2/0,0 mode: --/CR res: [0x240000402:0x186:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0955b3 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739082.918864:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x186:0x0]: returning size 4096 00010000:00010000:0.0:1787739082.918961:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd600/0x7ee4453e4a1aed8d lrc: 1/0,0 mode: --/CR res: [0x240000402:0x186:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955b3 expref: 10 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.918973:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x186:0x0].0x0 (ffffa073068a9700) 00010000:00010000:0.0:1787739082.918977:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739082.918981:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd600/0x7ee4453e4a1aed8d lrc: 0/0,0 mode: --/CR res: [0x240000402:0x186:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955b3 expref: 10 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.918995:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fd600, rc 0) 00000100:00100000:0.0:1787739082.919167:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c0f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:18318:x1874579652559872:12345-192.168.203.15@tcp:101:lfs.0 Request processed in 483us (1672us total) trans 4294968496 rc 301/301 00000100:00100000:0.0:1787739082.919175:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23724 00000100:00100000:1.0:1787739082.925136:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.925216:0:6527:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652560128 00000100:00100000:0.0:1787739082.925226:0:6527:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652560128 00000100:00100000:0.0:1787739082.925292:0:6527:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5085 00000100:00100000:0.0:1787739082.925303:0:6527:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:18318:x1874579652560128:12345-192.168.203.15@tcp:33:lfs.0 00000004:00200000:0.0:1787739082.925471:0:6527:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x186:0x0]: returning size 4096 00000004:00200000:0.0:1787739082.925476:0:6527:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 3 MDT count 2 00000100:00100000:0.0:1787739082.925563:0:6527:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:18318:x1874579652560128:12345-192.168.203.15@tcp:33:lfs.0 Request processed in 260us (435us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.925571:0:6527:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5085 00000100:00100000:2.0:1787739082.930973:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.931081:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652560384 00000100:00100000:0.0:1787739082.931090:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652560384 00000100:00100000:0.0:1787739082.931190:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5086 00000100:00100000:0.0:1787739082.931200:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:18318:x1874579652560384:12345-192.168.203.15@tcp:35:lfs.0 00000100:00100000:0.0:1787739082.931461:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7bc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:18318:x1874579652560384:12345-192.168.203.15@tcp:35:lfs.0 Request processed in 262us (496us total) trans 4294968497 rc 0/0 00000100:00100000:0.0:1787739082.931477:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5086 00000100:00100000:3.0:1787739082.942513:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.943298:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652560512 00000100:00100000:0.0:1787739082.943318:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652560512 00000100:00100000:0.0:1787739082.943390:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23725 00000100:00100000:0.0:1787739082.943402:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305618b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:18160:x1874579652560512:12345-192.168.203.15@tcp:101:bash.0 00010000:00010000:0.0:1787739082.943455:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739082.943479:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd600/0x7ee4453e4a1aed9b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0955ba expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.943542:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd400/0x7ee4453e4a1aeda2 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x186:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.943554:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd400/0x7ee4453e4a1aeda2 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x186:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.943565:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd400/0x7ee4453e4a1aeda2 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x186:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1787739082.943647:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x186:0x0]: returning size 4096 00000004:00200000:0.0:1787739082.943652:0:6524:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 3 MDT count 2 00000004:00010000:0.0:1787739082.943675:0:6524:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd400/0x7ee4453e4a1aeda2 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x186:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.943701:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd600/0x7ee4453e4a1aed9b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955ba expref: 12 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.943726:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd400/0x7ee4453e4a1aeda2 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x186:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955ba expref: 11 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739082.943734:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x186:0x0].0x0 (ffffa073068a9c00) 00010000:00010000:0.0:1787739082.943737:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1787739082.943740:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fd400, rc 0) 00000100:00100000:0.0:1787739082.943794:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305618b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:18160:x1874579652560512:12345-192.168.203.15@tcp:101:bash.0 Request processed in 394us (1289us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739082.943801:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23725 00000100:00100000:1.0:1787739082.955480:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739082.955634:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652560640 00000100:00100000:2.0:1787739082.955644:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652560640 00000100:00100000:2.0:1787739082.955719:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23726 00000100:00100000:2.0:1787739082.955729:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:18160:x1874579652560640:12345-192.168.203.15@tcp:101:bash.0 00010000:00010000:2.0:1787739082.955750:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739082.955772:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1600/0x7ee4453e4a1aeda9 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0955c1 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.955828:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0600/0x7ee4453e4a1aedb0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.955841:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0600/0x7ee4453e4a1aedb0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.955852:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0600/0x7ee4453e4a1aedb0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739082.955939:0:6525:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000400:0xcc:0x0]: returning size 4096 00000004:00200000:2.0:1787739082.955945:0:6525:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 3 MDT count 2 00010000:00010000:2.0:1787739082.955989:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1600/0x7ee4453e4a1aeda9 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955c1 expref: 13 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.956007:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0600/0x7ee4453e4a1aedb0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955c1 expref: 12 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.956015:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:2.0:1787739082.956018:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739082.956021:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa0600, rc 0) 00000100:00100000:2.0:1787739082.956078:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c2d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18160:x1874579652560640:12345-192.168.203.15@tcp:101:bash.0 Request processed in 351us (608us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.956085:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23726 00000100:00100000:2.0:1787739082.964321:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739082.966491:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652560768 00000100:00100000:0.0:1787739082.966506:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652560768 00000100:00100000:2.0:1787739082.966653:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23727 00000100:00100000:2.0:1787739082.966665:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f78780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+29:18160:x1874579652560768:12345-192.168.203.15@tcp:101:bash.0 00010000:00010000:2.0:1787739082.966687:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739082.966709:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1aedb7 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0955c8 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.966789:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1aedbe lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.966803:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1aedbe lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.966820:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1aedbe lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1787739082.966867:0:13793:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000400:0xce:0x0]: returning size 4096 00000004:00200000:2.0:1787739082.966872:0:13793:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 3 MDT count 2 00010000:00010000:2.0:1787739082.966918:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1600/0x7ee4453e4a1aedb7 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955c8 expref: 31 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.966937:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1a00/0x7ee4453e4a1aedbe lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955c8 expref: 30 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739082.966945:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:2.0:1787739082.966948:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1787739082.966952:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa1a00, rc 0) 00000100:00100000:2.0:1787739082.967009:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f78780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18160:x1874579652560768:12345-192.168.203.15@tcp:101:bash.0 Request processed in 346us (2698us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739082.967017:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23727 00000100:00100000:3.0:1787739082.979955:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739082.980052:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652561024 00000100:00100000:1.0:1787739082.980062:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652561024 00000100:00100000:1.0:1787739082.980129:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23728 00000100:00100000:1.0:1787739082.980140:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18160:x1874579652561024:12345-192.168.203.15@tcp:101:bash.0 00010000:00010000:1.0:1787739082.980161:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739082.980182:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841400/0x7ee4453e4a1aedc5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0955cf expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.980333:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840c00/0x7ee4453e4a1aedcc lrc: 3/1,0 mode: --/CR res: [0x240000402:0x186:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.980347:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840c00/0x7ee4453e4a1aedcc lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x186:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.980360:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840c00/0x7ee4453e4a1aedcc lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x186:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1787739082.980373:0:6523:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x186:0x0]: returning size 4096 00010000:00010000:1.0:1787739082.980719:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841400/0x7ee4453e4a1aedc5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955cf expref: 15 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.980743:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840c00/0x7ee4453e4a1aedcc lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x186:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955cf expref: 14 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739082.980751:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x186:0x0].0x0 (ffffa073068a9c00) 00010000:00010000:1.0:1787739082.980754:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1787739082.980757:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306840c00, rc 0) 00000100:00100000:1.0:1787739082.980864:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561a580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:18160:x1874579652561024:12345-192.168.203.15@tcp:101:bash.0 Request processed in 726us (929us total) trans 4294968498 rc 0/0 00000100:00100000:1.0:1787739082.980872:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23728 00000100:00100000:1.0:1787739082.994794:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739082.994914:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652561152 00000100:00100000:2.0:1787739082.994924:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652561152 00000100:00100000:2.0:1787739082.994979:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5087 00000100:00100000:2.0:1787739082.994989:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:18160:x1874579652561152:12345-192.168.203.15@tcp:37:bash.0 00000100:00100000:2.0:1787739083.009644:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:18160:x1874579652561152:12345-192.168.203.15@tcp:37:bash.0 Request processed in 14646us (14849us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739083.009657:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5087 00000100:00100000:0.0:1787739083.022622:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739083.023747:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652561280 00000100:00100000:1.0:1787739083.023759:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652561280 00000100:00100000:1.0:1787739083.023868:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5088 00000100:00100000:1.0:1787739083.023890:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074332ab050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18160:x1874579652561280:12345-192.168.203.15@tcp:37:bash.0 00000100:00080000:2.0:1787739083.026954:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1787739083.028818:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074332ab050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18160:x1874579652561280:12345-192.168.203.15@tcp:37:bash.0 Request processed in 4931us (6194us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739083.028831:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5088 00000100:00100000:0.0:1787739083.058183:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739083.058287:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652561408 00000100:00100000:1.0:1787739083.058297:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652561408 00000100:00100000:1.0:1787739083.058358:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23729 00000100:00100000:1.0:1787739083.058368:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18320:x1874579652561408:12345-192.168.203.15@tcp:101:cmp.0 00010000:00010000:1.0:1787739083.058396:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739083.058451:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840600/0x7ee4453e4a1aedda lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0955d6 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.058511:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aede1 lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.058725:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aede1 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.058739:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aede1 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.058758:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aede8 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0xeddcfc6c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.058769:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aede8 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xeddcfc6c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.058776:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aede8 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xeddcfc6c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.058811:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aede1 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.058818:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aede1 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.058823:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aede1 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.058829:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aede1 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.058834:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306841000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739083.058838:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aede1 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.058845:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aede1 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.058856:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:1.0:1787739083.058859:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739083.058864:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aede1 lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.058871:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aede1 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.059119:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aede8 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xeddcfc6c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.059126:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aede8 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xeddcfc6c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.059131:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aede8 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xeddcfc6c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.059137:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840a00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739083.059140:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aede8 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xeddcfc6c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.059146:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aede8 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xeddcfc6c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.059155:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0xeddcfc6c (ffffa0730713a000) 00010000:00010000:1.0:1787739083.059158:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739083.059161:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aede8 lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xeddcfc6c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.059167:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aede8 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xeddcfc6c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.059201:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840600/0x7ee4453e4a1aedda lrc: 1/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955d6 expref: 31 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.059212:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:1.0:1787739083.059214:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739083.059219:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840600/0x7ee4453e4a1aedda lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955d6 expref: 31 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.059228:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306840600, rc 0) 00000100:00100000:1.0:1787739083.059292:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be61c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18320:x1874579652561408:12345-192.168.203.15@tcp:101:cmp.0 Request processed in 926us (1119us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739083.059300:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23729 00000100:00100000:3.0:1787739083.063916:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739083.066042:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652561536 00000100:00100000:0.0:1787739083.066058:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652561536 00000100:00100000:0.0:1787739083.066173:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23730 00000100:00100000:0.0:1787739083.066186:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073056192c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:18320:x1874579652561536:12345-192.168.203.15@tcp:101:cmp.0 00010000:00010000:0.0:1787739083.066206:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739083.066226:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc400/0x7ee4453e4a1aedef lrc: 2/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0955dd expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.066294:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd200/0x7ee4453e4a1aedf6 lrc: 3/1,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.066309:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd200/0x7ee4453e4a1aedf6 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.066323:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd200/0x7ee4453e4a1aedf6 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.066338:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc200/0x7ee4453e4a1aedfd lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xeddcfc6c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.066346:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc200/0x7ee4453e4a1aedfd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xeddcfc6c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.066352:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc200/0x7ee4453e4a1aedfd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xeddcfc6c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.066408:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd200/0x7ee4453e4a1aedf6 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.066432:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd200/0x7ee4453e4a1aedf6 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.066438:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd200/0x7ee4453e4a1aedf6 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.066444:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd200/0x7ee4453e4a1aedf6 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.066450:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fd200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739083.066455:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd200/0x7ee4453e4a1aedf6 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.066463:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd200/0x7ee4453e4a1aedf6 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.066473:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:0.0:1787739083.066476:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739083.066481:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd200/0x7ee4453e4a1aedf6 lrc: 3/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.066487:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fd200/0x7ee4453e4a1aedf6 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.066499:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc200/0x7ee4453e4a1aedfd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xeddcfc6c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.066504:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc200/0x7ee4453e4a1aedfd lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xeddcfc6c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.066510:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc200/0x7ee4453e4a1aedfd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xeddcfc6c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.066514:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fc200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739083.066517:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc200/0x7ee4453e4a1aedfd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xeddcfc6c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.066531:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc200/0x7ee4453e4a1aedfd lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xeddcfc6c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.066538:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0xeddcfc6c (ffffa073068a9700) 00010000:00010000:0.0:1787739083.066541:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739083.066544:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc200/0x7ee4453e4a1aedfd lrc: 2/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xeddcfc6c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.066549:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc200/0x7ee4453e4a1aedfd lrc: 0/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xeddcfc6c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.066576:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc400/0x7ee4453e4a1aedef lrc: 1/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955dd expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.066585:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:0.0:1787739083.066587:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739083.066591:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc400/0x7ee4453e4a1aedef lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955dd expref: 15 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.066599:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fc400, rc 0) 00000100:00100000:0.0:1787739083.066668:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073056192c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:18320:x1874579652561536:12345-192.168.203.15@tcp:101:cmp.0 Request processed in 485us (2761us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739083.066675:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23730 00000100:00100000:3.0:1787739083.074347:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739083.076740:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652561664 00000100:00100000:2.0:1787739083.076755:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652561664 00000100:00100000:2.0:1787739083.076823:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23731 00000100:00100000:2.0:1787739083.076834:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18320:x1874579652561664:12345-192.168.203.15@tcp:101:cmp.0 00010000:00010000:2.0:1787739083.076854:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739083.076884:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0800/0x7ee4453e4a1aee04 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0955e4 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.077047:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aee0b lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.077062:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aee0b lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.077073:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aee0b lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.077089:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aee12 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0x3491cd4c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.077097:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aee12 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x3491cd4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.077104:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aee12 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x3491cd4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.077142:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aee0b lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.077150:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aee0b lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.077156:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aee0b lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.077162:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aee0b lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.077169:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739083.077174:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aee0b lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.077182:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aee0b lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.077193:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:2.0:1787739083.077196:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739083.077201:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aee0b lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.077208:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aee0b lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.077221:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aee12 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x3491cd4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.077228:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aee12 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x3491cd4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.077234:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aee12 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x3491cd4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.077239:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739083.077242:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aee12 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x3491cd4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.077248:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aee12 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x3491cd4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.077259:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x3491cd4c (ffffa07403a6f100) 00010000:00010000:2.0:1787739083.077262:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739083.077264:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aee12 lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x3491cd4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.077270:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aee12 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x3491cd4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.077299:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0800/0x7ee4453e4a1aee04 lrc: 1/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955e4 expref: 31 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.077308:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:2.0:1787739083.077311:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739083.077315:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0800/0x7ee4453e4a1aee04 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955e4 expref: 31 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.077324:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa0800, rc 0) 00010000:00010000:1.0:1787739083.077473:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739083.077620:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18320:x1874579652561664:12345-192.168.203.15@tcp:101:cmp.0 Request processed in 786us (3281us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739083.077628:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23731 00000100:00100000:2.0:1787739083.098241:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739083.098332:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652561792 00000100:00100000:3.0:1787739083.098342:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652561792 00000100:00100000:3.0:1787739083.098406:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23732 00000100:00100000:3.0:1787739083.098465:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f783c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:18320:x1874579652561792:12345-192.168.203.15@tcp:101:cmp.0 00010000:00010000:3.0:1787739083.098489:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739083.098512:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1aee19 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0955eb expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.098593:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee20 lrc: 3/1,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.098608:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee20 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.098619:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee20 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.098634:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee27 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x3491cd4c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.098641:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee27 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x3491cd4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.098647:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee27 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x3491cd4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.098911:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee20 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.098923:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee20 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.098929:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee20 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.098935:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee20 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.098941:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739083.098945:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee20 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.098953:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee20 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.098965:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739083.098967:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739083.098971:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee20 lrc: 3/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.098977:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee20 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.098989:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee27 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x3491cd4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.098994:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee27 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x3491cd4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.099000:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee27 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x3491cd4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.099005:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739083.099008:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee27 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x3491cd4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.099013:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee27 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x3491cd4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.099019:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x3491cd4c (ffffa074331c2200) 00010000:00010000:3.0:1787739083.099021:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739083.099024:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee27 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x3491cd4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.099029:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee27 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x3491cd4c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.099058:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1aee19 lrc: 1/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955eb expref: 15 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.099068:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739083.099071:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739083.099074:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1aee19 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955eb expref: 15 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.099083:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739083.099196:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f783c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:18320:x1874579652561792:12345-192.168.203.15@tcp:101:cmp.0 Request processed in 780us (964us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739083.099204:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23732 00000100:00100000:0.0:1787739083.111897:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739083.111983:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652561920 00000100:00100000:3.0:1787739083.111994:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652561920 00000100:00100000:3.0:1787739083.112053:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23733 00000100:00100000:3.0:1787739083.112064:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18320:x1874579652561920:12345-192.168.203.15@tcp:101:cmp.0 00010000:00010000:3.0:1787739083.112083:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739083.112110:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aee2e lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0955f2 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.112173:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee35 lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.112187:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee35 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.112198:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee35 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.112214:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee3c lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0x3735a305 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.112221:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee3c lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x3735a305 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.112227:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee3c lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x3735a305 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.112264:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee35 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.112273:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee35 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.112279:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee35 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.112287:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee35 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.112293:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739083.112297:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee35 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.112306:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee35 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.112317:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:3.0:1787739083.112320:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739083.112325:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee35 lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.112332:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee35 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.112346:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee3c lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x3735a305 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.112353:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee3c lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x3735a305 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.112357:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee3c lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x3735a305 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.112362:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739083.112365:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee3c lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x3735a305 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.112375:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee3c lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x3735a305 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.112383:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x3735a305 (ffffa074331c2200) 00010000:00010000:3.0:1787739083.112385:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739083.112388:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee3c lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x3735a305 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.112394:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee3c lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x3735a305 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.112895:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aee2e lrc: 1/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955f2 expref: 31 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.112907:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:3.0:1787739083.112909:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739083.112913:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aee2e lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955f2 expref: 31 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.112923:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739083.112986:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07306be7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18320:x1874579652561920:12345-192.168.203.15@tcp:101:cmp.0 Request processed in 923us (1096us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739083.112994:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23733 00000100:00100000:0.0:1787739083.118406:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739083.118521:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652562048 00000100:00100000:2.0:1787739083.118531:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652562048 00000100:00100000:2.0:1787739083.118583:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23734 00000100:00100000:2.0:1787739083.118593:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305656940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:18320:x1874579652562048:12345-192.168.203.15@tcp:101:cmp.0 00010000:00010000:2.0:1787739083.118612:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739083.118633:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aee43 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0955f9 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.118696:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aee4a lrc: 3/1,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.118710:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aee4a lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.118721:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aee4a lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.118735:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aee51 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x3735a305 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.118742:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aee51 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x3735a305 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.118750:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aee51 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x3735a305 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.118786:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aee4a lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.118793:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aee4a lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.118798:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aee4a lrc: 3/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.118804:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aee4a lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.118809:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739083.118813:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aee4a lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.118819:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aee4a lrc: 5/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.118830:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:2.0:1787739083.118833:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739083.118837:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aee4a lrc: 3/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.118844:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aee4a lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.118856:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aee51 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x3735a305 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.118861:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aee51 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x3735a305 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.118866:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aee51 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x3735a305 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.118871:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739083.118884:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aee51 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x3735a305 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.118891:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aee51 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x3735a305 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.118898:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x3735a305 (ffffa07403a6f100) 00010000:00010000:2.0:1787739083.118900:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739083.118904:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aee51 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x3735a305 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.118910:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aee51 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x3735a305 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.118938:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aee43 lrc: 1/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955f9 expref: 15 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.118947:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:2.0:1787739083.118949:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739083.118952:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0000/0x7ee4453e4a1aee43 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0955f9 expref: 15 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.118960:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa0000, rc 0) 00000100:00100000:2.0:1787739083.119031:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305656940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:18320:x1874579652562048:12345-192.168.203.15@tcp:101:cmp.0 Request processed in 439us (635us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739083.119038:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23734 00000100:00100000:2.0:1787739083.144755:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739083.144912:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652562304 00000100:00100000:1.0:1787739083.144921:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652562304 00000100:00100000:1.0:1787739083.144989:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23735 00000100:00100000:1.0:1787739083.144998:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f79a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18320:x1874579652562304:12345-192.168.203.15@tcp:101:cmp.0 00010000:00010000:1.0:1787739083.145017:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739083.145036:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840600/0x7ee4453e4a1aee58 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095600 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.145101:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aee5f lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.145114:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aee5f lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.145123:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aee5f lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.145133:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aee66 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.145139:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aee66 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.145148:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aee66 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.145183:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aee5f lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.145190:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aee5f lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.145196:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aee5f lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.145202:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aee5f lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.145208:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840a00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739083.145211:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aee5f lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.145219:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aee5f lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.145229:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:1.0:1787739083.145232:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739083.145235:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aee5f lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.145242:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aee5f lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.145254:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aee66 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.145260:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aee66 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.145265:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aee66 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.145270:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306841000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739083.145272:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aee66 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.145279:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aee66 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.145285:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0xdcc87686 (ffffa074331c2b00) 00010000:00010000:1.0:1787739083.145287:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739083.145290:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aee66 lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.145296:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aee66 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xdcc87686 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.145319:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840600/0x7ee4453e4a1aee58 lrc: 1/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095600 expref: 31 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.145329:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:1.0:1787739083.145332:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739083.145335:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840600/0x7ee4453e4a1aee58 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095600 expref: 31 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.145349:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306840600, rc 0) 00000100:00100000:1.0:1787739083.145680:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f79a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18320:x1874579652562304:12345-192.168.203.15@tcp:101:cmp.0 Request processed in 457us (711us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739083.145688:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23735 00000100:00100000:0.0:1787739083.156911:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739083.157815:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652562560 00000100:00100000:3.0:1787739083.157827:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652562560 00000100:00100000:3.0:1787739083.157900:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23736 00000100:00100000:3.0:1787739083.157910:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303f743c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+14:18320:x1874579652562560:12345-192.168.203.15@tcp:101:cmp.0 00010000:00010000:3.0:1787739083.157934:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739083.157954:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1aee6d lrc: 2/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095607 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.158024:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee74 lrc: 3/1,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.158037:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee74 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.158048:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee74 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.158060:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee7b lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.158070:0:6524:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1aed55 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 0@lo remote: 0x7ee4453e4a1aed4e expref: 20 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.158078:0:6524:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee7b lrc: 4/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.158108:0:6524:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1aed55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 0@lo remote: 0x7ee4453e4a1aed4e expref: 20 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787739083.158122:0:6524:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07305618f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665805824:0@lo:104: 00000100:00100000:3.0:1787739083.158173:0:6524:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739083.158238:0:6524:0:(client.c:2715:ptlrpc_set_wait()) set ffffa07306234a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787739083.160165:0:6507:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665805824 00000100:00100000:1.0:1787739083.160180:0:6507:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665805824 00000100:00100000:1.0:1787739083.160244:0:6507:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1216 00000100:00100000:1.0:1787739083.160253:0:6507:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6524:x1874579665805824:12345-0@lo:104: 00010000:00010000:1.0:1787739083.160270:0:6507:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1aed4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aed55 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.160384:0:6507:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787739083.160390:0:6507:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561a1c0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0000-mdtlov_UUID+4:6524:x1874579665805824:12345-0@lo:104: Request processed in 139us (2217us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739083.160397:0:6507:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1216 00010000:00010000:0.0:1787739083.160797:0:14665:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787739083.160805:0:14665:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1aed4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x1400000000000 nid: local remote: 0x7ee4453e4a1aed55 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.160815:0:14665:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18e6ee0) 00010000:00010000:0.0:1787739083.160821:0:14665:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1aed4e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1408400000000 nid: local remote: 0x7ee4453e4a1aed55 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1787739083.160835:0:14665:0:(mdt_handler.c:3973:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1aed4e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x1409400000000 nid: local remote: 0x7ee4453e4a1aed55 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.160869:0:14665:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1aed4e lrc: 2/0,0 mode: --/PW res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aed55 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.160886:0:14665:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787739083.160917:0:14665:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1aed4e lrc: 1/0,0 mode: --/PW res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aed55 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.160924:0:14665:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-osp-MDT0000 lock: ffffa073069abe00/0x7ee4453e4a1aed4e lrc: 0/0,0 mode: --/PW res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x2 rrc: 2 type: IBT gid 0 flags: 0x5c09400000020 nid: local remote: 0x7ee4453e4a1aed55 expref: -99 pid: 6536 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.160951:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739083.160953:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787739083.160955:0:14665:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787739083.160993:0:3653:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1787739083.161009:0:3653:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa07303a32580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665805952:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1787739083.161050:0:3653:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1787739083.161298:0:6524:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07305618f00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001_UUID:6524:1874579665805824:0@lo:104: 00000100:00100000:0.0:1787739083.161298:0:6509:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665805952 00000100:00100000:0.0:1787739083.161302:0:6509:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665805952 00010000:00010000:3.0:1787739083.161320:0:6524:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee7b lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.161323:0:6509:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1aed55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000400000020 nid: 0@lo remote: 0x7ee4453e4a1aed4e expref: 21 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787739083.161351:0:6509:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1604 00000100:00100000:0.0:1787739083.161358:0:6509:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305f7bc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+21:3653:x1874579665805952:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1787739083.161370:0:6509:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:3.0:1787739083.168277:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968498 is committed 00000001:00080000:3.0:1787739083.168304:0:7453:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0001: transno 4294968494 is committed 00010000:00010000:0.0:1787739083.168610:0:6509:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1aed55 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 0@lo remote: 0x7ee4453e4a1aed4e expref: 15 pid: 12410 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.168640:0:6509:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0xdcc87686 (ffffa0730713a500) 00010000:00010000:0.0:1787739083.168646:0:6509:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee7b lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.168656:0:6509:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee7b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.168664:0:6509:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee7b lrc: 4/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739083.168703:0:6509:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1787739083.168780:0:6509:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305f7bc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000-mdtlov_UUID+14:3653:x1874579665805952:12345-0@lo:103:ldlm_bl.0 Request processed in 7420us (7727us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739083.168789:0:6509:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1604 00000100:00100000:2.0:1787739083.168938:0:3653:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa07303a32580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3653:1874579665805952:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1787739083.170557:0:6524:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee7b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.170572:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee7b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.170685:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aee82 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x18c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.170707:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aee82 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x18c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.170718:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aee82 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x18c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1787739083.170729:0:6524:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x240000402:0x18c:0x0]: returning size 2 00010000:00010000:3.0:1787739083.170818:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee74 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.170826:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee74 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.170833:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee74 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.170839:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee74 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.170845:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739083.170851:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee74 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.170858:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee74 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.170869:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739083.170872:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739083.170885:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee74 lrc: 3/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.170891:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aee74 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.170903:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee7b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.170909:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee7b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.170915:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee7b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.170920:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739083.170923:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee7b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.170928:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee7b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.170934:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0xdcc87686 (ffffa0730713a500) 00010000:00010000:3.0:1787739083.170937:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739083.170940:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee7b lrc: 2/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.170945:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aee7b lrc: 0/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xdcc87686 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.171008:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1aee6d lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095607 expref: 12 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.171022:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab200/0x7ee4453e4a1aee82 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x18c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095607 expref: 11 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.171029:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x18c:0x0].0x0 (ffffa07305eae500) 00010000:00010000:3.0:1787739083.171032:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1787739083.171034:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069ab200, rc 0) 00000100:00100000:3.0:1787739083.171104:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303f743c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:18320:x1874579652562560:12345-192.168.203.15@tcp:101:cmp.0 Request processed in 13195us (14200us total) trans 4294968499 rc 0/0 00000100:00100000:3.0:1787739083.171113:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23736 00000100:00100000:3.0:1787739083.184046:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739083.184276:0:14679:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652562688 00000100:00100000:1.0:1787739083.184286:0:14679:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652562688 00000100:00100000:1.0:1787739083.184361:0:14679:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5201 00000100:00100000:1.0:1787739083.184386:0:14679:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07305618f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+9:2409:x1874579652562688:12345-192.168.203.15@tcp:101:cmp.0 00010000:00010000:1.0:1787739083.184404:0:14679:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739083.184605:0:14679:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c3f34a74/0x7ee4453e4a1aee90 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09560e expref: -99 pid: 14679 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1787739083.184631:0:14679:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x9c8:0x0] initial LVB size: 0, mtime: 0x6a8ebbca, atime: 0x0, ctime: 0x6a8ebbca, blocks: 0x0 00010000:00010000:1.0:1787739083.184652:0:14679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9c8:0x0].0x0 since=670s v=(0 0) 00010000:00010000:1.0:1787739083.184657:0:14679:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x9c8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1787739083.184667:0:14679:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c3f34a74/0x7ee4453e4a1aee90 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09560e expref: 10 pid: 14679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.184684:0:14679:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c3f34a74/0x7ee4453e4a1aee90 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x9c8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09560e expref: 10 pid: 14679 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.184764:0:14679:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306840600, rc 0) 00000100:00100000:1.0:1787739083.184829:0:14679:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07305618f00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2409:x1874579652562688:12345-192.168.203.15@tcp:101:cmp.0 Request processed in 460us (793us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739083.184837:0:14679:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5201 00000100:00080000:2.0:1787739083.184913:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787739083.204466:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739083.204923:0:8444:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652562816 00000100:00100000:0.0:1787739083.204933:0:8444:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652562816 00002000:00100000:0.0:1787739083.204985:0:8444:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x9c8:0x0] (0->4095) req@ffffa0730383fc00 x1874579652562816/t0(0) o3->e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2@192.168.203.15@tcp:264/0 lens 488/0 e 0 to 0 dl 1787739094 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cmp.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787739083.204995:0:8444:0:(ofd_dev.c:2572:ofd_prolong_extent_locks()) Prolong locks for req ffffa0730383fc00 with x1874579652562816 ext(0->4095) 00002000:00010000:0.0:1787739083.205000:0:8444:0:(ofd_dev.c:2721:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffffa0730383fc00 00000100:00100000:0.0:1787739083.205033:0:8444:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 2 00000100:00100000:0.0:1787739083.205044:0:8444:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652562816:12345-192.168.203.15@tcp:3:cmp.0 00000100:00100000:0.0:1787739083.205276:0:8444:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383fc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+10:2408:x1874579652562816:12345-192.168.203.15@tcp:3:cmp.0 Request processed in 233us (861us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739083.205284:0:8444:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 2 00002000:00100000:0.0:1787739083.205291:0:8444:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x9c8:0x0] (0->4095) req@ffffa0730383fc00 x1874579652562816/t0(0) o3->e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2@192.168.203.15@tcp:264/0 lens 488/440 e 0 to 0 dl 1787739094 ref 1 fl Complete:/600/0 rc 0/0 job:'cmp.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1787739083.205300:0:8444:0:(ofd_dev.c:2572:ofd_prolong_extent_locks()) Prolong locks for req ffffa0730383fc00 with x1874579652562816 ext(0->4095) 00002000:00010000:0.0:1787739083.205304:0:8444:0:(ofd_dev.c:2721:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffffa0730383fc00 00000100:00100000:3.0:1787739083.211596:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739083.211679:0:14475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652563072 00000100:00100000:2.0:1787739083.211700:0:14475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652563072 00000100:00100000:2.0:1787739083.211756:0:14475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 5089 00000100:00100000:2.0:1787739083.211766:0:14475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730561b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+11:18320:x1874579652563072:12345-192.168.203.15@tcp:35:cmp.0 00000100:00100000:2.0:1787739083.212009:0:14475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730561b840 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18320:x1874579652563072:12345-192.168.203.15@tcp:35:cmp.0 Request processed in 244us (422us total) trans 4294968500 rc 0/0 00000100:00100000:2.0:1787739083.212017:0:14475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 5089 00000100:00100000:3.0:1787739083.248283:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739083.248402:0:6528:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652563200 00000100:00100000:1.0:1787739083.248444:0:6528:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652563200 00000100:00100000:1.0:1787739083.248501:0:6528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 3739 00000100:00100000:1.0:1787739083.248511:0:6528:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074074f5a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:2409:x1874579652563200:12345-192.168.203.15@tcp:63:ll_sa.0 00010000:00010000:1.0:1787739083.248579:0:6528:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739083.248599:0:6528:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aee97 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095615 expref: -99 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.248660:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aee9e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.248673:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aee9e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.248685:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aee9e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.248700:0:6528:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aeea5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x50441c1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.248707:0:6528:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aeea5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x50441c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.248712:0:6528:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aeea5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x50441c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.248765:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aee9e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.248772:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aee9e lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.248778:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aee9e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.248785:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840a00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739083.248789:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aee9e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.248797:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aee9e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.248809:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa07401795300) 00010000:00010000:1.0:1787739083.248812:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739083.248817:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aee9e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.248824:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aee9e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.248836:0:6528:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aeea5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x50441c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.248842:0:6528:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aeea5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x50441c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.248848:0:6528:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aeea5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x50441c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.248853:0:6528:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739083.248856:0:6528:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aeea5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x50441c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.248862:0:6528:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aeea5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x50441c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.248868:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x50441c1 (ffffa0730713a700) 00010000:00010000:1.0:1787739083.248871:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739083.248882:0:6528:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aeea5 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x50441c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.248889:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aeea5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x50441c1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6528 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.248919:0:6528:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aee97 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095615 expref: 31 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.248929:0:6528:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffffa07401795300) 00010000:00010000:1.0:1787739083.248931:0:6528:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1787739083.248935:0:6528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aee97 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095615 expref: 31 pid: 6528 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.249157:0:6528:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841000, rc -2) 00000100:00100000:1.0:1787739083.249218:0:6528:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074074f5a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:2409:x1874579652563200:12345-192.168.203.15@tcp:63:ll_sa.0 Request processed in 708us (944us total) trans 0 rc -2/-2 00000100:00100000:1.0:1787739083.249226:0:6528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 3739 00000100:00100000:0.0:1787739083.285600:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739083.285792:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652563456 00000100:00100000:2.0:1787739083.285803:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652563456 00000100:00100000:2.0:1787739083.285864:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23737 00000100:00100000:2.0:1787739083.285884:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18326:x1874579652563456:12345-192.168.203.15@tcp:101:awk.0 00010000:00010000:2.0:1787739083.285905:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739083.285927:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aeeac lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09564d expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.286005:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aeeb3 lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.286020:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aeeb3 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.286032:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aeeb3 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.286047:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1aeeba lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0xcd19b99d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.286054:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1aeeba lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xcd19b99d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.286060:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1aeeba lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xcd19b99d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.286098:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aeeb3 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.286106:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aeeb3 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.286112:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aeeb3 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.286118:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aeeb3 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.286124:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739083.286128:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aeeb3 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.286136:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aeeb3 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.286148:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:2.0:1787739083.286150:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739083.286155:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aeeb3 lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.286161:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aeeb3 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.286175:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1aeeba lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xcd19b99d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.286181:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1aeeba lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xcd19b99d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.286185:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1aeeba lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xcd19b99d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.286190:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0a00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739083.286192:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1aeeba lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xcd19b99d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.286197:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1aeeba lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xcd19b99d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.286202:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0xcd19b99d (ffffa07403a6f100) 00010000:00010000:2.0:1787739083.286204:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739083.286206:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1aeeba lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xcd19b99d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.286211:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1aeeba lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xcd19b99d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.286240:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aeeac lrc: 1/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09564d expref: 31 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.286250:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:2.0:1787739083.286252:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739083.286255:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0200/0x7ee4453e4a1aeeac lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09564d expref: 31 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.286264:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa0200, rc 0) 00000100:00100000:2.0:1787739083.286332:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18326:x1874579652563456:12345-192.168.203.15@tcp:101:awk.0 Request processed in 458us (741us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739083.286339:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23737 00000100:00100000:0.0:1787739083.298071:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739083.298294:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652563712 00000100:00100000:1.0:1787739083.298303:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652563712 00000100:00100000:1.0:1787739083.298386:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23738 00000100:00100000:1.0:1787739083.298397:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a33c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18325:x1874579652563712:12345-192.168.203.15@tcp:101:lsmod.0 00010000:00010000:1.0:1787739083.298464:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739083.298486:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aeec1 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095654 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.298566:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aeec8 lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.298580:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aeec8 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.298592:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aeec8 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.298605:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aeecf lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0x907a850 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.298613:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aeecf lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x907a850 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.298620:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aeecf lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x907a850 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.298665:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aeec8 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.298673:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aeec8 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.298680:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aeec8 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.298964:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aeec8 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.298973:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739083.298978:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aeec8 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.298987:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aeec8 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.299000:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:1.0:1787739083.299003:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739083.299008:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aeec8 lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.299015:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aeec8 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.299030:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aeecf lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x907a850 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.299037:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aeecf lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x907a850 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.299043:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aeecf lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x907a850 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.299049:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840a00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739083.299051:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aeecf lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x907a850 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.299057:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aeecf lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x907a850 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.299064:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x907a850 (ffffa0730713a700) 00010000:00010000:1.0:1787739083.299066:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739083.299069:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aeecf lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x907a850 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.299074:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840a00/0x7ee4453e4a1aeecf lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x907a850 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.299110:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aeec1 lrc: 1/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095654 expref: 31 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.299121:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:1.0:1787739083.299123:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739083.299127:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aeec1 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095654 expref: 31 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.299136:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841000, rc 0) 00000100:00100000:1.0:1787739083.299201:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a33c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18325:x1874579652563712:12345-192.168.203.15@tcp:101:lsmod.0 Request processed in 806us (1140us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739083.299209:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23738 00000100:00100000:2.0:1787739083.304563:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739083.304790:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652563968 00000100:00100000:3.0:1787739083.304801:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652563968 00000100:00100000:3.0:1787739083.304857:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23739 00000100:00100000:3.0:1787739083.304867:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18325:x1874579652563968:12345-192.168.203.15@tcp:101:lsmod.0 00010000:00010000:3.0:1787739083.304899:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739083.304919:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1aeed6 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09565b expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.304996:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aeedd lrc: 3/1,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.305010:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aeedd lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.305021:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aeedd lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.305035:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aeee4 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x907a850 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.305042:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aeee4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x907a850 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.305049:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aeee4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x907a850 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.305083:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aeedd lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.305089:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aeedd lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.305094:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aeedd lrc: 3/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.305100:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aeedd lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.305106:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739083.305110:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aeedd lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.305118:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aeedd lrc: 5/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.305129:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739083.305132:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739083.305136:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aeedd lrc: 3/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.305141:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1aeedd lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.305153:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aeee4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x907a850 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.305159:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aeee4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x907a850 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.305164:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aeee4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x907a850 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.305169:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739083.305171:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aeee4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x907a850 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.305176:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aeee4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x907a850 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.305182:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x907a850 (ffffa07305eae100) 00010000:00010000:3.0:1787739083.305184:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739083.305188:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aeee4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x907a850 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.305193:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1aeee4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x907a850 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.305222:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1aeed6 lrc: 1/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09565b expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.305231:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739083.305234:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739083.305237:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1aeed6 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09565b expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.305246:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739083.305306:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383ed00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18325:x1874579652563968:12345-192.168.203.15@tcp:101:lsmod.0 Request processed in 440us (754us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739083.305313:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23739 00000100:00100000:0.0:1787739083.309939:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739083.310028:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652564224 00000100:00100000:1.0:1787739083.310038:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652564224 00000100:00100000:1.0:1787739083.310096:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23740 00000100:00100000:1.0:1787739083.310106:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a321c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18326:x1874579652564224:12345-192.168.203.15@tcp:101:awk.0 00010000:00010000:1.0:1787739083.310125:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739083.310142:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841000/0x7ee4453e4a1aeeeb lrc: 2/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095662 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.310205:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1aeef2 lrc: 3/1,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.310219:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1aeef2 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.310230:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1aeef2 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.310244:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aeef9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xcd19b99d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.310251:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aeef9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xcd19b99d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.310258:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aeef9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xcd19b99d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.310295:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1aeef2 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.310302:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1aeef2 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.310308:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1aeef2 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.310315:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1aeef2 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.310321:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840a00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739083.310338:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1aeef2 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.310346:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1aeef2 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.310358:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:1.0:1787739083.310361:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739083.310365:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1aeef2 lrc: 3/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.310372:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1aeef2 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.310384:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aeef9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xcd19b99d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.310390:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aeef9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xcd19b99d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.310396:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aeef9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xcd19b99d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.310401:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739083.310403:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aeef9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xcd19b99d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.310461:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aeef9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xcd19b99d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.310470:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0xcd19b99d (ffffa0730713a700) 00010000:00010000:1.0:1787739083.310472:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739083.310475:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aeef9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xcd19b99d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.310481:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aeef9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xcd19b99d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.310511:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841000/0x7ee4453e4a1aeeeb lrc: 1/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095662 expref: 13 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.310764:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:1.0:1787739083.310768:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739083.310773:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841000/0x7ee4453e4a1aeeeb lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095662 expref: 13 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.310783:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841000, rc 0) 00000100:00100000:1.0:1787739083.310845:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a321c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18326:x1874579652564224:12345-192.168.203.15@tcp:101:awk.0 Request processed in 739us (913us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739083.310852:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23740 00000100:00100000:0.0:1787739083.322339:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739083.322465:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652564480 00000100:00100000:2.0:1787739083.322476:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652564480 00000100:00100000:2.0:1787739083.322541:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23741 00000100:00100000:2.0:1787739083.322552:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a330c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18326:x1874579652564480:12345-192.168.203.15@tcp:101:awk.0 00010000:00010000:2.0:1787739083.322573:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739083.322594:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1aef00 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095669 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.322803:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aef07 lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.322817:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aef07 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.322829:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aef07 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.322843:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aef0e lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0xd53f944c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.322849:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aef0e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xd53f944c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.322856:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aef0e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xd53f944c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.322901:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aef07 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.322908:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aef07 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.322918:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aef07 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.322925:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aef07 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.322931:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739083.322935:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aef07 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.322942:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aef07 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.322954:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:2.0:1787739083.322957:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739083.322961:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aef07 lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.322967:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aef07 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.322981:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aef0e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xd53f944c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.322987:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aef0e lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xd53f944c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.322993:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aef0e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xd53f944c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.322998:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739083.323002:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aef0e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xd53f944c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.323008:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aef0e lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xd53f944c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.323015:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0xd53f944c (ffffa07403a6f100) 00010000:00010000:2.0:1787739083.323018:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739083.323020:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aef0e lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xd53f944c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.323026:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aef0e lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xd53f944c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.323056:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1aef00 lrc: 1/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095669 expref: 31 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.323066:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:2.0:1787739083.323069:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739083.323072:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1aef00 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095669 expref: 31 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.323082:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa0a00, rc 0) 00000100:00100000:2.0:1787739083.323183:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a330c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18326:x1874579652564480:12345-192.168.203.15@tcp:101:awk.0 Request processed in 633us (853us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739083.323191:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23741 00000100:00100000:2.0:1787739083.331265:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739083.331558:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652564736 00000100:00100000:3.0:1787739083.331567:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652564736 00000100:00100000:3.0:1787739083.331674:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23742 00000100:00100000:3.0:1787739083.331684:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0730383f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18325:x1874579652564736:12345-192.168.203.15@tcp:101:lsmod.0 00010000:00010000:3.0:1787739083.331703:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739083.331724:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aef15 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095670 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.331796:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aef1c lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.331817:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aef1c lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.331828:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aef1c lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.331843:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1aef23 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0x90ba0166 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.331850:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1aef23 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x90ba0166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.331857:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1aef23 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x90ba0166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.331904:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aef1c lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.331911:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aef1c lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.331918:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aef1c lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.331924:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aef1c lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.331931:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739083.331935:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aef1c lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.331943:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aef1c lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.331955:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:3.0:1787739083.331958:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739083.331962:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aef1c lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.331968:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1aef1c lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.331982:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1aef23 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x90ba0166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.331988:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1aef23 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x90ba0166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.331993:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1aef23 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x90ba0166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.331998:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739083.332000:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1aef23 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x90ba0166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.332006:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1aef23 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x90ba0166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.332013:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x90ba0166 (ffffa07305eae100) 00010000:00010000:3.0:1787739083.332015:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739083.332017:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1aef23 lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x90ba0166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.332021:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1aef23 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x90ba0166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.332054:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aef15 lrc: 1/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095670 expref: 31 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.332065:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:3.0:1787739083.332067:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739083.332071:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1aef15 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095670 expref: 31 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.332080:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739083.332141:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0730383f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18325:x1874579652564736:12345-192.168.203.15@tcp:101:lsmod.0 Request processed in 458us (886us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739083.332148:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23742 00000100:00100000:0.0:1787739083.334855:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739083.334938:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652564992 00000100:00100000:1.0:1787739083.334947:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652564992 00000100:00100000:1.0:1787739083.335001:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23743 00000100:00100000:1.0:1787739083.335010:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07303a32580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18326:x1874579652564992:12345-192.168.203.15@tcp:101:awk.0 00010000:00010000:1.0:1787739083.335032:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739083.335049:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841000/0x7ee4453e4a1aef2a lrc: 2/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095677 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.335113:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aef31 lrc: 3/1,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.335125:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aef31 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.335136:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aef31 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.335148:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1aef38 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xd53f944c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.335154:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1aef38 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xd53f944c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.335159:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1aef38 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xd53f944c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.335192:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aef31 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.335198:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aef31 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.335203:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aef31 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.335209:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aef31 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.335215:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739083.335220:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aef31 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.335226:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aef31 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.335237:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:1.0:1787739083.335239:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739083.335243:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aef31 lrc: 3/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.335250:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aef31 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.335262:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1aef38 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xd53f944c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.335267:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1aef38 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xd53f944c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.335272:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1aef38 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xd53f944c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.335276:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840a00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739083.335279:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1aef38 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xd53f944c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.335284:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1aef38 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xd53f944c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.335293:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0xd53f944c (ffffa0730713a700) 00010000:00010000:1.0:1787739083.335295:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739083.335298:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1aef38 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xd53f944c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.335304:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1aef38 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xd53f944c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.335623:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841000/0x7ee4453e4a1aef2a lrc: 1/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095677 expref: 13 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.335634:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:1.0:1787739083.335636:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739083.335640:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841000/0x7ee4453e4a1aef2a lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095677 expref: 13 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.335715:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841000, rc 0) 00000100:00100000:1.0:1787739083.335814:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07303a32580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18326:x1874579652564992:12345-192.168.203.15@tcp:101:awk.0 Request processed in 804us (966us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739083.335821:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23743 00000100:00100000:0.0:1787739083.338849:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739083.338936:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652565248 00000100:00100000:3.0:1787739083.338944:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652565248 00000100:00100000:3.0:1787739083.339030:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23744 00000100:00100000:3.0:1787739083.339040:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c417480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18325:x1874579652565248:12345-192.168.203.15@tcp:101:lsmod.0 00010000:00010000:3.0:1787739083.339064:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739083.339084:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1aef3f lrc: 2/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09567e expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.339152:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef46 lrc: 3/1,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.339165:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef46 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.339175:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef46 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.339189:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef4d lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x90ba0166 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.339196:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef4d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x90ba0166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.339202:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef4d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x90ba0166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.339236:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef46 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.339243:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef46 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.339249:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef46 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.339255:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef46 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.339261:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739083.339265:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef46 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.339271:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef46 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.339281:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739083.339284:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739083.339288:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef46 lrc: 3/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.339294:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef46 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.339306:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef4d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x90ba0166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.339312:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef4d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x90ba0166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.339317:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef4d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x90ba0166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.339322:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739083.339385:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef4d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x90ba0166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.339392:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef4d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x90ba0166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.339400:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x90ba0166 (ffffa07305eae900) 00010000:00010000:3.0:1787739083.339403:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739083.339406:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef4d lrc: 2/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x90ba0166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.339677:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef4d lrc: 0/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x90ba0166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.339714:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1aef3f lrc: 1/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09567e expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.339730:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739083.339732:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739083.339736:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1aef3f lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09567e expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.339745:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aae00, rc 0) 00000100:00100000:3.0:1787739083.339795:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c417480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18325:x1874579652565248:12345-192.168.203.15@tcp:101:lsmod.0 Request processed in 757us (954us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739083.339803:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23744 00000100:00100000:0.0:1787739083.348139:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739083.352994:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652565504 00000100:00100000:3.0:1787739083.353010:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652565504 00000100:00100000:3.0:1787739083.353085:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23745 00000100:00100000:3.0:1787739083.353096:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c417840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18326:x1874579652565504:12345-192.168.203.15@tcp:101:awk.0 00010000:00010000:3.0:1787739083.353116:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739083.353136:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1aef54 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095685 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.353234:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef5b lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.353249:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef5b lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.353260:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef5b lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.353273:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef62 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0xfe94d89f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.353280:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef62 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xfe94d89f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.353286:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef62 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xfe94d89f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.353321:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef5b lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.353328:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef5b lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.353334:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef5b lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.353340:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef5b lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.353345:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739083.353349:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef5b lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.353355:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef5b lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.353366:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:3.0:1787739083.353368:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739083.353372:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef5b lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.353378:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef5b lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.353390:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef62 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xfe94d89f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.353395:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef62 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xfe94d89f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.353400:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef62 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xfe94d89f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.353404:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739083.353407:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef62 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xfe94d89f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.353442:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef62 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xfe94d89f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.353450:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0xfe94d89f (ffffa07305eae900) 00010000:00010000:3.0:1787739083.353453:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739083.353456:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef62 lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xfe94d89f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.353462:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef62 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xfe94d89f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.353493:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1aef54 lrc: 1/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095685 expref: 31 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.353503:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:3.0:1787739083.353506:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739083.353511:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1aef54 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095685 expref: 31 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.353520:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aae00, rc 0) 00000100:00100000:3.0:1787739083.353596:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c417840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18326:x1874579652565504:12345-192.168.203.15@tcp:101:awk.0 Request processed in 501us (5466us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739083.353604:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23745 00000100:00100000:0.0:1787739083.366992:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739083.367173:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652565760 00000100:00100000:1.0:1787739083.367183:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652565760 00000100:00100000:1.0:1787739083.367233:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23746 00000100:00100000:1.0:1787739083.367243:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c4170c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18326:x1874579652565760:12345-192.168.203.15@tcp:101:awk.0 00010000:00010000:1.0:1787739083.367262:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739083.368897:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1aef69 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09568c expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.368991:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aef70 lrc: 3/1,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.369004:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aef70 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.369014:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aef70 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.369029:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841000/0x7ee4453e4a1aef77 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xfe94d89f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.369035:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841000/0x7ee4453e4a1aef77 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xfe94d89f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.369042:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841000/0x7ee4453e4a1aef77 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xfe94d89f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.369081:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aef70 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.369088:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aef70 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.369094:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aef70 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.369110:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aef70 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.369116:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739083.369121:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aef70 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.369129:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aef70 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.369141:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:1.0:1787739083.369144:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739083.369149:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aef70 lrc: 3/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.369155:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aef70 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.369171:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841000/0x7ee4453e4a1aef77 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xfe94d89f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.369177:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841000/0x7ee4453e4a1aef77 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xfe94d89f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.369183:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841000/0x7ee4453e4a1aef77 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xfe94d89f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.369189:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306841000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739083.369192:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841000/0x7ee4453e4a1aef77 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xfe94d89f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.369199:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841000/0x7ee4453e4a1aef77 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xfe94d89f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.369207:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0xfe94d89f (ffffa0730713a700) 00010000:00010000:1.0:1787739083.369209:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739083.369212:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841000/0x7ee4453e4a1aef77 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xfe94d89f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.369219:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841000/0x7ee4453e4a1aef77 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xfe94d89f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.369251:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1aef69 lrc: 1/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09568c expref: 13 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.369262:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:1.0:1787739083.369265:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739083.369269:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1aef69 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09568c expref: 13 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.369291:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306840a00, rc 0) 00000100:00100000:1.0:1787739083.369355:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c4170c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18326:x1874579652565760:12345-192.168.203.15@tcp:101:awk.0 Request processed in 2112us (2370us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739083.369364:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23746 00000100:00100000:1.0:1787739083.957499:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739083.957802:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652566016 00000100:00100000:3.0:1787739083.957812:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652566016 00000100:00100000:3.0:1787739083.957888:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23747 00000100:00100000:3.0:1787739083.957899:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18382:x1874579652566016:12345-192.168.203.15@tcp:101:sed.0 00010000:00010000:3.0:1787739083.957929:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739083.957950:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1aef7e lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095693 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.958023:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef85 lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.958038:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef85 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.958049:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef85 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.958064:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef8c lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0x8c70ca4d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.958071:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef8c lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x8c70ca4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.958077:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef8c lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x8c70ca4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.958237:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef85 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.958246:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef85 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.958252:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef85 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.958258:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef85 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.958264:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739083.958267:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef85 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.958273:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef85 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.958282:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:3.0:1787739083.958284:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739083.958287:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef85 lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.958291:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1aef85 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.958300:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef8c lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x8c70ca4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.958304:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef8c lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x8c70ca4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.958308:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef8c lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x8c70ca4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.958311:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739083.958312:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef8c lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x8c70ca4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.958315:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef8c lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x8c70ca4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.958319:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x8c70ca4d (ffffa07305eae900) 00010000:00010000:3.0:1787739083.958320:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739083.958322:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef8c lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x8c70ca4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.958325:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1aef8c lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x8c70ca4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.958348:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1aef7e lrc: 1/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095693 expref: 31 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.958354:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:3.0:1787739083.958355:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739083.958357:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1aef7e lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095693 expref: 31 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.958362:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aae00, rc 0) 00000100:00100000:3.0:1787739083.958488:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c30c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18382:x1874579652566016:12345-192.168.203.15@tcp:101:sed.0 Request processed in 591us (1000us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739083.958500:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23747 00000100:00100000:0.0:1787739083.965279:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739083.965363:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652566272 00000100:00100000:2.0:1787739083.965374:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652566272 00000100:00100000:2.0:1787739083.965452:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23748 00000100:00100000:2.0:1787739083.965463:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c415e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18382:x1874579652566272:12345-192.168.203.15@tcp:101:sed.0 00010000:00010000:2.0:1787739083.965482:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739083.965503:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1aef93 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09569a expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.965586:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aef9a lrc: 3/1,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.965600:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aef9a lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.965611:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aef9a lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.965624:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aefa1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x8c70ca4d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.965631:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aefa1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x8c70ca4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.965638:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aefa1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x8c70ca4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.965677:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aef9a lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.965685:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aef9a lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.965691:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aef9a lrc: 3/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.965704:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aef9a lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.965710:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739083.965714:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aef9a lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.965722:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aef9a lrc: 5/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.965735:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:2.0:1787739083.965737:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739083.965742:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aef9a lrc: 3/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.965748:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1aef9a lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.965759:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aefa1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x8c70ca4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.965765:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aefa1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x8c70ca4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.965771:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aefa1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x8c70ca4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.965775:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739083.965778:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aefa1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x8c70ca4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.965788:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aefa1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x8c70ca4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.965794:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x8c70ca4d (ffffa07403a6f100) 00010000:00010000:2.0:1787739083.965809:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739083.965812:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aefa1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x8c70ca4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.965818:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1aefa1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x8c70ca4d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.965848:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1aef93 lrc: 1/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09569a expref: 13 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.965858:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:2.0:1787739083.965860:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739083.965864:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1aef93 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09569a expref: 13 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739083.965886:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa0a00, rc 0) 00000100:00100000:2.0:1787739083.965947:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c415e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18382:x1874579652566272:12345-192.168.203.15@tcp:101:sed.0 Request processed in 485us (676us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739083.965954:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23748 00000100:00100000:0.0:1787739083.973408:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739083.974105:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652566528 00000100:00100000:1.0:1787739083.974116:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652566528 00000100:00100000:1.0:1787739083.974175:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23749 00000100:00100000:1.0:1787739083.974185:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c416d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18382:x1874579652566528:12345-192.168.203.15@tcp:101:sed.0 00010000:00010000:1.0:1787739083.974205:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739083.974225:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aefa8 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0956a1 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.974293:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aefaf lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.974311:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aefaf lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.974327:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aefaf lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.974341:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aefb6 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0xfb43d54d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.974349:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aefb6 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xfb43d54d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.974355:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aefb6 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xfb43d54d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.974391:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aefaf lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.974399:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aefaf lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.974406:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aefaf lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.974437:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aefaf lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.974445:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739083.974450:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aefaf lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.974457:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aefaf lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.974471:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:1.0:1787739083.974473:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739083.974478:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aefaf lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.974484:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aefaf lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.974498:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aefb6 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xfb43d54d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.974504:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aefb6 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xfb43d54d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.974510:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aefb6 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xfb43d54d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.974514:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739083.974517:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aefb6 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xfb43d54d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.974523:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aefb6 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xfb43d54d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.974530:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0xfb43d54d (ffffa0730713a700) 00010000:00010000:1.0:1787739083.974532:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739083.974535:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aefb6 lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xfb43d54d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.974540:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aefb6 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xfb43d54d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.974590:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aefa8 lrc: 1/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956a1 expref: 31 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.974600:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:1.0:1787739083.974602:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739083.974605:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aefa8 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956a1 expref: 31 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739083.974614:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841000, rc 0) 00000100:00100000:1.0:1787739083.974675:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c416d00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18382:x1874579652566528:12345-192.168.203.15@tcp:101:sed.0 Request processed in 492us (1275us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739083.974682:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23749 00000100:00100000:1.0:1787739083.989712:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739083.989865:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652566784 00000100:00100000:3.0:1787739083.989884:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652566784 00000100:00100000:3.0:1787739083.989953:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23750 00000100:00100000:3.0:1787739083.989963:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18382:x1874579652566784:12345-192.168.203.15@tcp:101:sed.0 00010000:00010000:3.0:1787739083.989984:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739083.990005:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1aefbd lrc: 2/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0956a8 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.990077:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aefc4 lrc: 3/1,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.990091:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aefc4 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.990102:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aefc4 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.990116:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aefcb lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xfb43d54d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.990123:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aefcb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xfb43d54d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.990129:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aefcb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xfb43d54d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.990164:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aefc4 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.990174:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aefc4 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.990180:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aefc4 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.990186:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aefc4 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.990192:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739083.990195:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aefc4 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.990202:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aefc4 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.990212:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739083.990215:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739083.990219:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aefc4 lrc: 3/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.990224:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1aefc4 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.990237:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aefcb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xfb43d54d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.990242:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aefcb lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xfb43d54d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.990247:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aefcb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xfb43d54d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.990252:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739083.990254:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aefcb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xfb43d54d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.990260:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aefcb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xfb43d54d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.990265:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0xfb43d54d (ffffa07305eae900) 00010000:00010000:3.0:1787739083.990268:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739083.990270:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aefcb lrc: 2/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xfb43d54d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.990276:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1aefcb lrc: 0/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xfb43d54d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.990305:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1aefbd lrc: 1/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956a8 expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.990314:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739083.990316:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739083.990320:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1aefbd lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956a8 expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739083.990329:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aae00, rc 0) 00000100:00100000:3.0:1787739083.990393:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18382:x1874579652566784:12345-192.168.203.15@tcp:101:sed.0 Request processed in 431us (690us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739083.990401:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23750 00000100:00100000:0.0:1787739084.011957:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739084.012172:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652567040 00000100:00100000:1.0:1787739084.012181:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652567040 00000100:00100000:1.0:1787739084.012238:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23751 00000100:00100000:1.0:1787739084.012248:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c416940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18382:x1874579652567040:12345-192.168.203.15@tcp:101:sed.0 00010000:00010000:1.0:1787739084.012268:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739084.012288:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aefd2 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0956af expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.012600:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aefd9 lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.012614:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aefd9 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.012629:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aefd9 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.012655:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aefe0 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0xe2c88e60 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.012663:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aefe0 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xe2c88e60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.012670:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aefe0 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xe2c88e60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.012708:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aefd9 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.012714:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aefd9 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.012720:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aefd9 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.012727:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aefd9 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.012732:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739084.012736:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aefd9 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.012742:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aefd9 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.012753:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:1.0:1787739084.012757:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739084.012761:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aefd9 lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.012766:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1aefd9 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.012778:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aefe0 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xe2c88e60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.012783:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aefe0 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xe2c88e60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.012790:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aefe0 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xe2c88e60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.012795:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739084.012978:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aefe0 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xe2c88e60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.012986:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aefe0 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xe2c88e60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.012995:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0xe2c88e60 (ffffa0730713a700) 00010000:00010000:1.0:1787739084.012998:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739084.013001:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aefe0 lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xe2c88e60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.013008:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1aefe0 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xe2c88e60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.013198:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aefd2 lrc: 1/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956af expref: 31 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.013211:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:1.0:1787739084.013214:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739084.013218:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841000/0x7ee4453e4a1aefd2 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956af expref: 31 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.013229:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841000, rc 0) 00000100:00100000:1.0:1787739084.014754:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c416940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18382:x1874579652567040:12345-192.168.203.15@tcp:101:sed.0 Request processed in 2499us (2797us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739084.014765:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23751 00000100:00100000:0.0:1787739084.029782:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739084.032377:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652567296 00000100:00100000:1.0:1787739084.032392:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652567296 00000100:00100000:1.0:1787739084.032540:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23752 00000100:00100000:1.0:1787739084.032553:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c4152c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18382:x1874579652567296:12345-192.168.203.15@tcp:101:sed.0 00010000:00010000:1.0:1787739084.032573:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739084.032594:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841000/0x7ee4453e4a1aefe7 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0956b6 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.032696:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aefee lrc: 3/1,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.032711:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aefee lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.032723:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aefee lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.032737:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1aeff5 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xe2c88e60 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.032745:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1aeff5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xe2c88e60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.032752:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1aeff5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xe2c88e60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.032790:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aefee lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.032965:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aefee lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.032973:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aefee lrc: 3/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.032979:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aefee lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.032985:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739084.032989:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aefee lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.032996:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aefee lrc: 5/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.033009:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:1.0:1787739084.033012:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739084.033017:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aefee lrc: 3/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.033023:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1aefee lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.033049:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1aeff5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xe2c88e60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.033056:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1aeff5 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xe2c88e60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.033062:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1aeff5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xe2c88e60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.033067:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739084.033070:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1aeff5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xe2c88e60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.033076:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1aeff5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xe2c88e60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.033083:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0xe2c88e60 (ffffa0730713a700) 00010000:00010000:1.0:1787739084.033086:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739084.033089:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1aeff5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xe2c88e60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.033095:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1aeff5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xe2c88e60 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.033127:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841000/0x7ee4453e4a1aefe7 lrc: 1/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956b6 expref: 13 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.033137:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:1.0:1787739084.033139:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739084.033143:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841000/0x7ee4453e4a1aefe7 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956b6 expref: 13 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.033152:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841000, rc 0) 00000100:00100000:1.0:1787739084.033220:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c4152c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18382:x1874579652567296:12345-192.168.203.15@tcp:101:sed.0 Request processed in 670us (3448us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739084.033227:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23752 00000100:00080000:2.0:1787739084.037965:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00010000:00010000:2.0:1787739084.099004:0:6512:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739084.203481:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739084.204130:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652567552 00000100:00100000:2.0:1787739084.204141:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652567552 00000100:00100000:2.0:1787739084.204210:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23753 00000100:00100000:2.0:1787739084.204221:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743c416580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18380:x1874579652567552:12345-192.168.203.15@tcp:101:pdsh.0 00010000:00010000:2.0:1787739084.204246:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739084.204268:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1aeffc lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0956bd expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.204340:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af003 lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.204354:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af003 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.204366:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af003 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.204381:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af00a lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0xbe63e774 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.204388:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af00a lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xbe63e774 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.204395:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af00a lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xbe63e774 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.204476:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af003 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.204484:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af003 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.204490:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af003 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.204502:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af003 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.204509:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739084.204513:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af003 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.204520:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af003 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.204531:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:2.0:1787739084.204534:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739084.204538:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af003 lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.204544:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af003 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.204558:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af00a lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xbe63e774 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.204564:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af00a lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xbe63e774 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.204570:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af00a lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xbe63e774 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.204574:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739084.204577:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af00a lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xbe63e774 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.204587:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af00a lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xbe63e774 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.204595:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0xbe63e774 (ffffa07403a6f100) 00010000:00010000:2.0:1787739084.204597:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739084.204600:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af00a lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xbe63e774 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.204606:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af00a lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xbe63e774 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.204640:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1aeffc lrc: 1/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956bd expref: 31 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.204650:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:2.0:1787739084.204652:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739084.204656:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1aeffc lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956bd expref: 31 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.204666:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa0a00, rc 0) 00000100:00100000:2.0:1787739084.204721:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743c416580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18380:x1874579652567552:12345-192.168.203.15@tcp:101:pdsh.0 Request processed in 501us (1248us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739084.204729:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23753 00000100:00100000:0.0:1787739084.211955:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739084.217185:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652567808 00000100:00100000:1.0:1787739084.217199:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652567808 00000100:00100000:1.0:1787739084.217262:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23754 00000100:00100000:1.0:1787739084.217271:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18380:x1874579652567808:12345-192.168.203.15@tcp:101:pdsh.0 00010000:00010000:1.0:1787739084.217292:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739084.217312:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841000/0x7ee4453e4a1af011 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0956c4 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.218015:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1af018 lrc: 3/1,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.218031:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1af018 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.218043:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1af018 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.218057:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1af01f lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xbe63e774 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.218064:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1af01f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xbe63e774 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.218070:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1af01f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xbe63e774 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.218110:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1af018 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.218116:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1af018 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.218122:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1af018 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.218128:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1af018 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.218134:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739084.218139:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1af018 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.218147:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1af018 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.218159:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:1.0:1787739084.218162:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739084.218166:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1af018 lrc: 3/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.218173:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1af018 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.218186:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1af01f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xbe63e774 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.218192:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1af01f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xbe63e774 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.218198:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1af01f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xbe63e774 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.218203:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739084.218205:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1af01f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xbe63e774 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.218211:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1af01f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xbe63e774 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.218217:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0xbe63e774 (ffffa0730713a700) 00010000:00010000:1.0:1787739084.218219:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739084.218222:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1af01f lrc: 2/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xbe63e774 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.218226:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1af01f lrc: 0/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xbe63e774 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.218261:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841000/0x7ee4453e4a1af011 lrc: 1/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956c4 expref: 13 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.218271:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:1.0:1787739084.218274:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739084.218277:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306841000/0x7ee4453e4a1af011 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956c4 expref: 13 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.218286:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841000, rc 0) 00000100:00100000:1.0:1787739084.218347:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562c3c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18380:x1874579652567808:12345-192.168.203.15@tcp:101:pdsh.0 Request processed in 1076us (6400us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739084.218358:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23754 00000100:00100000:1.0:1787739084.226750:0:3652:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1787739084.226773:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0743562e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665806080:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1787739084.226847:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739084.226977:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665806080 00000100:00100000:0.0:1787739084.226986:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665806080 00000100:00100000:0.0:1787739084.227054:0:8441:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 374 00000100:00100000:0.0:1787739084.227066:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c12c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874579665806080:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1787739084.227093:0:8441:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 33959360 pending 86016 free 3923529728 avail 3692007424 00000100:00100000:0.0:1787739084.227139:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c12c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3652:x1874579665806080:12345-0@lo:13:osp-pre-0-1.0 Request processed in 76us (293us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739084.227146:0:8441:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 374 00000100:00100000:0.0:1787739084.227186:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0743562e940 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665806080:0@lo:13:osp-pre-0-1.0 00000100:00080000:1.0:1787739084.227355:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1787739084.228694:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739084.228833:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652568064 00000100:00100000:3.0:1787739084.228841:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652568064 00000100:00100000:3.0:1787739084.228901:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23755 00000100:00100000:3.0:1787739084.228910:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18380:x1874579652568064:12345-192.168.203.15@tcp:101:pdsh.0 00010000:00010000:3.0:1787739084.228929:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739084.228950:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1af026 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0956cb expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.229024:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1af02d lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.229038:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1af02d lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.229049:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1af02d lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.229063:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1af034 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0x897eb166 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.229070:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1af034 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x897eb166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.229076:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1af034 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x897eb166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.229113:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1af02d lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.229120:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1af02d lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.229126:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1af02d lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.229132:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1af02d lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.229139:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739084.229143:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1af02d lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.229150:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1af02d lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.229160:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:3.0:1787739084.229163:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739084.229167:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1af02d lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.229173:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1af02d lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.229185:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1af034 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x897eb166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.229192:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1af034 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x897eb166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.229197:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1af034 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x897eb166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.229202:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739084.229204:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1af034 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x897eb166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.229213:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1af034 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x897eb166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.229219:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x897eb166 (ffffa07305eae900) 00010000:00010000:3.0:1787739084.229222:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739084.229225:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1af034 lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x897eb166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.229230:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1af034 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x897eb166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.229263:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1af026 lrc: 1/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956cb expref: 31 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.229274:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:3.0:1787739084.229276:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739084.229279:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1af026 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956cb expref: 31 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.229288:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aae00, rc 0) 00000100:00100000:3.0:1787739084.229349:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562e940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18380:x1874579652568064:12345-192.168.203.15@tcp:101:pdsh.0 Request processed in 440us (663us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739084.229357:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23755 00000100:00100000:2.0:1787739084.230836:0:3651:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787739084.230854:0:3651:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa073043c2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665806208:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1787739084.230926:0:3651:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1787739084.231459:0:12859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665806208 00000100:00100000:1.0:1787739084.231466:0:12859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665806208 00000100:00100000:1.0:1787739084.231534:0:12859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 375 00000100:00100000:1.0:1787739084.231542:0:12859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156eda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+8:3651:x1874579665806208:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1787739084.231562:0:12859:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 33959360 pending 86016 free 3923529728 avail 3692007424 00000100:00100000:1.0:1787739084.231606:0:12859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156eda40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+8:3651:x1874579665806208:12345-0@lo:13:osp-pre-0-0.0 Request processed in 65us (680us total) trans 0 rc 0/0 00000100:00100000:1.0:1787739084.231612:0:12859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 375 00000100:00080000:2.0:1787739084.231755:0:6519:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1787739084.231794:0:3651:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa073043c2580 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3651:1874579665806208:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1787739084.237963:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739084.238054:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652568320 00000100:00100000:2.0:1787739084.238063:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652568320 00000100:00100000:2.0:1787739084.238119:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23756 00000100:00100000:2.0:1787739084.238129:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0743562f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18380:x1874579652568320:12345-192.168.203.15@tcp:101:pdsh.0 00010000:00010000:2.0:1787739084.238149:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739084.238169:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1af03b lrc: 2/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0956d2 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.238238:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af042 lrc: 3/1,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.238606:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af042 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.238617:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af042 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.238632:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af049 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x897eb166 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.238638:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af049 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x897eb166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.238644:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af049 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x897eb166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.238677:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af042 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.238683:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af042 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.238693:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af042 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.238699:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af042 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.238705:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739084.238708:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af042 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.238715:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af042 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.238724:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:2.0:1787739084.238727:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739084.238730:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af042 lrc: 3/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.238736:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af042 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.238748:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af049 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x897eb166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.238754:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af049 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x897eb166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.238759:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af049 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x897eb166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.238763:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739084.238766:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af049 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x897eb166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.238771:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af049 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x897eb166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.238776:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x897eb166 (ffffa07403eeb900) 00010000:00010000:2.0:1787739084.238778:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739084.238780:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af049 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x897eb166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.238784:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af049 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x897eb166 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.238810:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1af03b lrc: 1/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956d2 expref: 13 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.238819:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:2.0:1787739084.238821:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739084.238823:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1af03b lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956d2 expref: 13 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.238830:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa0a00, rc 0) 00000100:00100000:2.0:1787739084.238888:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0743562f840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18380:x1874579652568320:12345-192.168.203.15@tcp:101:pdsh.0 Request processed in 761us (934us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739084.238896:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23756 00000100:00100000:1.0:1787739084.256081:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739084.256170:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652568576 00000100:00100000:0.0:1787739084.256186:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652568576 00000100:00100000:0.0:1787739084.256262:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23757 00000100:00100000:0.0:1787739084.256274:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18380:x1874579652568576:12345-192.168.203.15@tcp:101:pdsh.0 00010000:00010000:0.0:1787739084.256296:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739084.256316:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1af050 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0956d9 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.256399:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc200/0x7ee4453e4a1af057 lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.256450:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc200/0x7ee4453e4a1af057 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.256463:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc200/0x7ee4453e4a1af057 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.256479:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc400/0x7ee4453e4a1af05e lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0x702ef35c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.256487:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc400/0x7ee4453e4a1af05e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x702ef35c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.256493:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc400/0x7ee4453e4a1af05e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x702ef35c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.256536:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc200/0x7ee4453e4a1af057 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.256548:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc200/0x7ee4453e4a1af057 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.256554:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc200/0x7ee4453e4a1af057 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.256561:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc200/0x7ee4453e4a1af057 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.256566:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fc200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739084.256570:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc200/0x7ee4453e4a1af057 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.256578:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc200/0x7ee4453e4a1af057 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.256590:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:0.0:1787739084.256593:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739084.256597:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc200/0x7ee4453e4a1af057 lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.256604:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc200/0x7ee4453e4a1af057 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.256618:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc400/0x7ee4453e4a1af05e lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x702ef35c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.256624:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc400/0x7ee4453e4a1af05e lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x702ef35c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.256631:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc400/0x7ee4453e4a1af05e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x702ef35c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.256636:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fc400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739084.256639:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc400/0x7ee4453e4a1af05e lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x702ef35c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.256645:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc400/0x7ee4453e4a1af05e lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x702ef35c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.256652:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x702ef35c (ffffa073068a9700) 00010000:00010000:0.0:1787739084.256655:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739084.256657:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc400/0x7ee4453e4a1af05e lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x702ef35c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.256662:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc400/0x7ee4453e4a1af05e lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x702ef35c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.256696:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1af050 lrc: 1/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956d9 expref: 31 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.256707:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:0.0:1787739084.256709:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739084.256713:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fd000/0x7ee4453e4a1af050 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956d9 expref: 31 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.256722:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fd000, rc 0) 00000100:00100000:0.0:1787739084.256783:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18380:x1874579652568576:12345-192.168.203.15@tcp:101:pdsh.0 Request processed in 511us (716us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739084.256791:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23757 00000100:00100000:0.0:1787739084.265587:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739084.266999:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652568832 00000100:00100000:3.0:1787739084.267012:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652568832 00000100:00100000:3.0:1787739084.267075:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23758 00000100:00100000:3.0:1787739084.267085:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304000f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18380:x1874579652568832:12345-192.168.203.15@tcp:101:pdsh.0 00010000:00010000:3.0:1787739084.267108:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739084.267127:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1af065 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0956e0 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.267196:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1af06c lrc: 3/1,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.267210:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1af06c lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.267221:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1af06c lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.267235:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1af073 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x702ef35c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.267242:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1af073 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x702ef35c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.267249:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1af073 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x702ef35c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.267283:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1af06c lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.267290:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1af06c lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.267297:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1af06c lrc: 3/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.267303:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1af06c lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.267310:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739084.267314:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1af06c lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.267322:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1af06c lrc: 5/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.267333:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739084.267336:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739084.267339:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1af06c lrc: 3/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.267345:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1af06c lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.267358:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1af073 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x702ef35c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.267364:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1af073 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x702ef35c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.267385:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1af073 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x702ef35c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.267390:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739084.267393:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1af073 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x702ef35c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.267399:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1af073 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x702ef35c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.267444:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x702ef35c (ffffa0730608b300) 00010000:00010000:3.0:1787739084.267447:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739084.267450:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1af073 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x702ef35c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.267456:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1af073 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x702ef35c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.267487:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1af065 lrc: 1/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956e0 expref: 13 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.267497:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739084.267499:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739084.267503:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1af065 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956e0 expref: 13 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.267512:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aae00, rc 0) 00000100:00100000:3.0:1787739084.267568:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304000f00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18380:x1874579652568832:12345-192.168.203.15@tcp:101:pdsh.0 Request processed in 485us (1992us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739084.267576:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23758 00000100:00100000:1.0:1787739084.300719:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739084.305159:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652569088 00000100:00100000:2.0:1787739084.305188:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652569088 00000100:00100000:2.0:1787739084.305309:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23759 00000100:00100000:2.0:1787739084.305322:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18412:x1874579652569088:12345-192.168.203.15@tcp:101:ssh.0 00010000:00010000:2.0:1787739084.305350:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739084.305377:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1af07a lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0956e7 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.305547:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af081 lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.305564:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af081 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.305578:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af081 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.305596:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af088 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0xa22460e7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.305603:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af088 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.305610:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af088 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.305660:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af081 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.305668:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af081 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.305675:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af081 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.305682:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af081 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.305689:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739084.305694:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af081 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.305702:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af081 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.305722:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:2.0:1787739084.305725:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739084.305730:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af081 lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.306004:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af081 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.306213:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af088 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.306221:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af088 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.306227:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af088 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.306555:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739084.306559:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af088 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.306924:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af088 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.306954:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0xa22460e7 (ffffa07403eeb900) 00010000:00010000:2.0:1787739084.306957:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739084.306964:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af088 lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.306976:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af088 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.307658:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1af07a lrc: 1/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956e7 expref: 31 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.307673:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:2.0:1787739084.307675:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739084.307687:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0a00/0x7ee4453e4a1af07a lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956e7 expref: 31 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.307707:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa0a00, rc 0) 00000100:00100000:2.0:1787739084.308046:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18412:x1874579652569088:12345-192.168.203.15@tcp:101:ssh.0 Request processed in 2718us (7334us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739084.308065:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23759 00000100:00100000:1.0:1787739084.319409:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739084.319729:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652569344 00000100:00100000:3.0:1787739084.319741:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652569344 00000100:00100000:3.0:1787739084.319810:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23760 00000100:00100000:3.0:1787739084.319821:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18412:x1874579652569344:12345-192.168.203.15@tcp:101:ssh.0 00010000:00010000:3.0:1787739084.319851:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739084.319889:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1af08f lrc: 2/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0956ee expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.319989:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1af096 lrc: 3/1,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.320005:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1af096 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.320020:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1af096 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.320038:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1af09d lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xa22460e7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.320044:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1af09d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.320061:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1af09d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.320125:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1af096 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.320133:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1af096 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.320140:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1af096 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.320148:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1af096 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.320155:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739084.320236:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1af096 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.320247:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1af096 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.320262:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739084.320266:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739084.320271:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1af096 lrc: 3/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.320278:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069abe00/0x7ee4453e4a1af096 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.320293:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1af09d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.320300:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1af09d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.320306:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1af09d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.320310:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739084.320313:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1af09d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.320319:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1af09d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.320325:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0xa22460e7 (ffffa0730608b300) 00010000:00010000:3.0:1787739084.320328:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739084.320331:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1af09d lrc: 2/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.320337:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab400/0x7ee4453e4a1af09d lrc: 0/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.320376:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1af08f lrc: 1/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956ee expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.320388:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739084.320390:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739084.320394:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aae00/0x7ee4453e4a1af08f lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956ee expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.320404:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aae00, rc 0) 00000100:00100000:3.0:1787739084.320748:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c3c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18412:x1874579652569344:12345-192.168.203.15@tcp:101:ssh.0 Request processed in 928us (1352us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739084.320758:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23760 00000100:00100000:0.0:1787739084.328886:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739084.329093:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652569600 00000100:00100000:1.0:1787739084.329106:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652569600 00000100:00100000:1.0:1787739084.329198:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23761 00000100:00100000:1.0:1787739084.329211:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304003840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18413:x1874579652569600:12345-192.168.203.15@tcp:101:ssh.0 00010000:00010000:1.0:1787739084.329239:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739084.329266:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841400/0x7ee4453e4a1af0a4 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0956f5 expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.329366:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1af0ab lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.329389:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1af0ab lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.329403:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1af0ab lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.329454:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1af0b2 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0xa22460e7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.329462:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1af0b2 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.329470:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1af0b2 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.329532:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1af0ab lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.329541:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1af0ab lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.329547:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1af0ab lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.329554:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1af0ab lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.329561:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739084.329566:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1af0ab lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.329574:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1af0ab lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.329588:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:1.0:1787739084.329591:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739084.329597:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1af0ab lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.329603:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840000/0x7ee4453e4a1af0ab lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.329618:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1af0b2 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.329625:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1af0b2 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.329631:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1af0b2 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.329636:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739084.329638:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1af0b2 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.329644:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1af0b2 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.329651:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0xa22460e7 (ffffa0730713a700) 00010000:00010000:1.0:1787739084.329653:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739084.329656:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1af0b2 lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.329661:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306840800/0x7ee4453e4a1af0b2 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.329706:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841400/0x7ee4453e4a1af0a4 lrc: 1/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956f5 expref: 31 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.329718:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:1.0:1787739084.329721:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739084.329725:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306841400/0x7ee4453e4a1af0a4 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956f5 expref: 31 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.329735:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306841400, rc 0) 00000100:00100000:1.0:1787739084.329809:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304003840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18413:x1874579652569600:12345-192.168.203.15@tcp:101:ssh.0 Request processed in 600us (947us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739084.329817:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23761 00000100:00100000:1.0:1787739084.337665:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739084.337768:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739084.337896:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652570112 00000100:00100000:2.0:1787739084.337908:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652570112 00000100:00100000:2.0:1787739084.338022:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652569984 00000100:00100000:2.0:1787739084.338027:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652569984 00000100:00100000:3.0:1787739084.338076:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23762 00000100:00100000:3.0:1787739084.338090:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18412:x1874579652570112:12345-192.168.203.15@tcp:101:ssh.0 00010000:00010000:3.0:1787739084.338127:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739084.338156:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1af0b9 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e0956fc expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338265:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1af0c0 lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338283:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1af0c0 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338301:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1af0c0 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338317:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1af0c7 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0x4d5ed3e0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338324:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1af0c7 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x4d5ed3e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338330:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1af0c7 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x4d5ed3e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338379:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1af0c0 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338387:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1af0c0 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338394:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1af0c0 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338402:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1af0c0 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338409:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739084.338448:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1af0c0 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338457:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1af0c0 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338471:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:3.0:1787739084.338476:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739084.338481:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1af0c0 lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338488:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab400/0x7ee4453e4a1af0c0 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338503:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1af0c7 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x4d5ed3e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338509:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1af0c7 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x4d5ed3e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338525:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1af0c7 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x4d5ed3e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338531:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069abe00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739084.338534:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1af0c7 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x4d5ed3e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338543:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1af0c7 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x4d5ed3e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338550:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x4d5ed3e0 (ffffa0730608b300) 00010000:00010000:3.0:1787739084.338552:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739084.338555:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1af0c7 lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x4d5ed3e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338560:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069abe00/0x7ee4453e4a1af0c7 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x4d5ed3e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338599:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1af0b9 lrc: 1/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956fc expref: 31 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338611:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:3.0:1787739084.338614:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739084.338618:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aae00/0x7ee4453e4a1af0b9 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e0956fc expref: 31 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338628:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aae00, rc 0) 00000100:00100000:3.0:1787739084.338711:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c2580 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18412:x1874579652570112:12345-192.168.203.15@tcp:101:ssh.0 Request processed in 624us (1061us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739084.338719:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23762 00000100:00100000:3.0:1787739084.338744:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23763 00000100:00100000:3.0:1787739084.338752:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18413:x1874579652569984:12345-192.168.203.15@tcp:101:ssh.0 00010000:00010000:3.0:1787739084.338762:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739084.338776:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1af0ce lrc: 2/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095703 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338815:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af0d5 lrc: 3/1,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338823:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af0d5 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338831:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af0d5 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338843:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af0dc lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xa22460e7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338854:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af0dc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338878:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af0dc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338901:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af0d5 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338908:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af0d5 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338914:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af0d5 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338920:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af0d5 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338925:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739084.338929:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af0d5 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338935:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af0d5 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338943:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739084.338946:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739084.338949:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af0d5 lrc: 3/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338955:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af0d5 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338965:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af0dc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338975:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af0dc lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338981:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af0dc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338985:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739084.338988:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af0dc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.338993:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af0dc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.339000:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0xa22460e7 (ffffa0730608bb00) 00010000:00010000:3.0:1787739084.339002:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739084.339005:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af0dc lrc: 2/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.339011:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af0dc lrc: 0/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0xa22460e7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.339034:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1af0ce lrc: 1/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095703 expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.339043:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739084.339045:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739084.339049:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1af0ce lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095703 expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.339057:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739084.339080:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c1a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18413:x1874579652569984:12345-192.168.203.15@tcp:101:ssh.0 Request processed in 330us (1312us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739084.339087:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23763 00000100:00100000:2.0:1787739084.345126:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739084.351833:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652570368 00000100:00100000:0.0:1787739084.351856:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652570368 00000100:00100000:0.0:1787739084.351997:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23764 00000100:00100000:0.0:1787739084.352009:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18412:x1874579652570368:12345-192.168.203.15@tcp:101:ssh.0 00010000:00010000:0.0:1787739084.352038:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739084.352066:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc400/0x7ee4453e4a1af0e3 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09570a expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.352238:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc200/0x7ee4453e4a1af0ea lrc: 3/1,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.352257:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc200/0x7ee4453e4a1af0ea lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.352269:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc200/0x7ee4453e4a1af0ea lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.352287:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fcc00/0x7ee4453e4a1af0f1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x4d5ed3e0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.352294:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fcc00/0x7ee4453e4a1af0f1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x4d5ed3e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.352300:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fcc00/0x7ee4453e4a1af0f1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x4d5ed3e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.352354:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc200/0x7ee4453e4a1af0ea lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.352362:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc200/0x7ee4453e4a1af0ea lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.352368:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc200/0x7ee4453e4a1af0ea lrc: 3/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.352375:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc200/0x7ee4453e4a1af0ea lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.352381:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fc200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739084.352386:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc200/0x7ee4453e4a1af0ea lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.352393:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc200/0x7ee4453e4a1af0ea lrc: 5/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.352407:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:0.0:1787739084.352410:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739084.352435:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc200/0x7ee4453e4a1af0ea lrc: 3/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.352442:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc200/0x7ee4453e4a1af0ea lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.352458:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fcc00/0x7ee4453e4a1af0f1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x4d5ed3e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.352464:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fcc00/0x7ee4453e4a1af0f1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x4d5ed3e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.352470:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fcc00/0x7ee4453e4a1af0f1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x4d5ed3e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.352475:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fcc00 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739084.352478:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fcc00/0x7ee4453e4a1af0f1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x4d5ed3e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.352484:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fcc00/0x7ee4453e4a1af0f1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x4d5ed3e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.352490:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x4d5ed3e0 (ffffa073068a9700) 00010000:00010000:0.0:1787739084.352493:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739084.352495:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fcc00/0x7ee4453e4a1af0f1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x4d5ed3e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.352501:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fcc00/0x7ee4453e4a1af0f1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x4d5ed3e0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.352690:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc400/0x7ee4453e4a1af0e3 lrc: 1/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09570a expref: 13 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.352703:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:0.0:1787739084.352705:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739084.352709:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073063fc400/0x7ee4453e4a1af0e3 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09570a expref: 13 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.352719:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fc400, rc 0) 00000100:00100000:0.0:1787739084.356217:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156ee1c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18412:x1874579652570368:12345-192.168.203.15@tcp:101:ssh.0 Request processed in 4191us (11086us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739084.356235:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23764 00000100:00100000:1.0:1787739084.364699:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739084.369331:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652570624 00000100:00100000:2.0:1787739084.369353:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652570624 00000100:00100000:2.0:1787739084.369581:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23765 00000100:00100000:2.0:1787739084.369596:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18412:x1874579652570624:12345-192.168.203.15@tcp:101:ssh.0 00010000:00010000:2.0:1787739084.369626:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739084.369654:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af0f8 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095711 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.369758:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af0ff lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.369775:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af0ff lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.369789:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af0ff lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.369807:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af106 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0x89b1ffd9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.369815:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af106 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.369823:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af106 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.369882:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af0ff lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.369889:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af0ff lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.369896:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af0ff lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.369907:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af0ff lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.369915:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739084.369919:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af0ff lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.369928:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af0ff lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.369941:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:2.0:1787739084.369944:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739084.369948:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af0ff lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.369955:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af0ff lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.369973:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af106 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.369979:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af106 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.369984:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af106 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.369990:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739084.369993:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af106 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.370003:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af106 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.370011:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x89b1ffd9 (ffffa07403eeb900) 00010000:00010000:2.0:1787739084.370013:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739084.370017:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af106 lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.370022:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af106 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.370064:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af0f8 lrc: 1/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095711 expref: 31 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.370075:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:2.0:1787739084.370077:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739084.370081:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af0f8 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095711 expref: 31 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.370091:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa1800, rc 0) 00000100:00100000:2.0:1787739084.372667:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c3840 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18412:x1874579652570624:12345-192.168.203.15@tcp:101:ssh.0 Request processed in 3057us (7966us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739084.372686:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23765 00000100:00100000:1.0:1787739084.386392:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739084.386568:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652570880 00000100:00100000:0.0:1787739084.386580:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652570880 00000100:00100000:3.0:1787739084.386931:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23766 00000100:00100000:3.0:1787739084.386947:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa073043c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18412:x1874579652570880:12345-192.168.203.15@tcp:101:ssh.0 00010000:00010000:3.0:1787739084.386977:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739084.387006:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1af10d lrc: 2/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095718 expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.387141:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af114 lrc: 3/1,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.387159:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af114 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.387172:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af114 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.387189:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af11b lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x89b1ffd9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.387195:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af11b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.387202:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af11b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.387255:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af114 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.387262:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af114 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.387269:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af114 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.387275:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af114 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.387281:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739084.387286:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af114 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.387296:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af114 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.387311:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739084.387314:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739084.387319:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af114 lrc: 3/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.387339:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af114 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.387356:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af11b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.387363:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af11b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.387369:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af11b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.387374:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739084.387377:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af11b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.387382:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af11b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.387390:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x89b1ffd9 (ffffa0730608bb00) 00010000:00010000:3.0:1787739084.387392:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739084.387395:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af11b lrc: 2/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.387400:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af11b lrc: 0/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.387488:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1af10d lrc: 1/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095718 expref: 13 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.387500:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739084.387503:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739084.387506:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1af10d lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095718 expref: 13 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.387706:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739084.387782:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa073043c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18412:x1874579652570880:12345-192.168.203.15@tcp:101:ssh.0 Request processed in 839us (1403us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739084.387795:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23766 00000100:00100000:1.0:1787739084.393526:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:0.0:1787739084.394839:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652571136 00000100:00100000:0.0:1787739084.394857:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652571136 00000100:00100000:0.0:1787739084.394956:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23767 00000100:00100000:0.0:1787739084.394967:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18413:x1874579652571136:12345-192.168.203.15@tcp:101:ssh.0 00010000:00010000:0.0:1787739084.394995:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1787739084.395022:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1af122 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09571f expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.395122:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc200/0x7ee4453e4a1af129 lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.395140:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc200/0x7ee4453e4a1af129 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.395154:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc200/0x7ee4453e4a1af129 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.395172:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1af130 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0x89b1ffd9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.395179:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1af130 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.395186:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1af130 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.395235:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc200/0x7ee4453e4a1af129 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.395242:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc200/0x7ee4453e4a1af129 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.395249:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc200/0x7ee4453e4a1af129 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.395261:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc200/0x7ee4453e4a1af129 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.395268:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fc200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739084.395273:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc200/0x7ee4453e4a1af129 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.395281:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc200/0x7ee4453e4a1af129 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.395295:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:0.0:1787739084.395298:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739084.395304:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc200/0x7ee4453e4a1af129 lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.395310:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc200/0x7ee4453e4a1af129 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.395329:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1af130 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.395336:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1af130 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.395342:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1af130 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.395346:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073063fc800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:0.0:1787739084.395349:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1af130 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.395359:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1af130 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.395367:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x89b1ffd9 (ffffa073068a9700) 00010000:00010000:0.0:1787739084.395369:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1787739084.395372:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1af130 lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.395378:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fc800/0x7ee4453e4a1af130 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.395475:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1af122 lrc: 1/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09571f expref: 31 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.395488:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:0.0:1787739084.395491:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1787739084.395495:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073063fcc00/0x7ee4453e4a1af122 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09571f expref: 31 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787739084.395507:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073063fcc00, rc 0) 00000100:00100000:0.0:1787739084.395883:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce03c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18413:x1874579652571136:12345-192.168.203.15@tcp:101:ssh.0 Request processed in 916us (2380us total) trans 0 rc 301/301 00000100:00100000:0.0:1787739084.395894:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23767 00000100:00100000:0.0:1787739084.407347:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739084.407454:0:13793:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652571392 00000100:00100000:2.0:1787739084.407468:0:13793:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652571392 00000100:00100000:2.0:1787739084.407552:0:13793:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23768 00000100:00100000:2.0:1787739084.407563:0:13793:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304003480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18413:x1874579652571392:12345-192.168.203.15@tcp:101:ssh.0 00010000:00010000:2.0:1787739084.407592:0:13793:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739084.407618:0:13793:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af137 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095726 expref: -99 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.407715:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af13e lrc: 3/1,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.407731:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af13e lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.407744:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af13e lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.407773:0:13793:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af145 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x89b1ffd9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 13793 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.407780:0:13793:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af145 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.407787:0:13793:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af145 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.407835:0:13793:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af13e lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.408040:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af13e lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.408050:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af13e lrc: 3/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.408056:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af13e lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.408064:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739084.408070:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af13e lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.408079:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af13e lrc: 5/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.408093:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:2.0:1787739084.408096:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739084.408101:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af13e lrc: 3/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.408108:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af13e lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.408127:0:13793:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af145 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.408133:0:13793:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af145 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.408138:0:13793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af145 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.408143:0:13793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739084.408145:0:13793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af145 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.408151:0:13793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af145 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.408162:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x89b1ffd9 (ffffa07403eeb900) 00010000:00010000:2.0:1787739084.408165:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739084.408167:0:13793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af145 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.408173:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af145 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x89b1ffd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 13793 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.408212:0:13793:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af137 lrc: 1/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095726 expref: 13 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.408222:0:13793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:2.0:1787739084.408224:0:13793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739084.408228:0:13793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af137 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095726 expref: 13 pid: 13793 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.408237:0:13793:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa1800, rc 0) 00000100:00100000:2.0:1787739084.408338:0:13793:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304003480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18413:x1874579652571392:12345-192.168.203.15@tcp:101:ssh.0 Request processed in 764us (1003us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739084.408348:0:13793:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23768 00000100:00100000:1.0:1787739084.417007:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:2.0:1787739084.417477:0:6523:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652571648 00000100:00100000:2.0:1787739084.417491:0:6523:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652571648 00000100:00100000:2.0:1787739084.417566:0:6523:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23769 00000100:00100000:2.0:1787739084.417578:0:6523:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304ce2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18412:x1874579652571648:12345-192.168.203.15@tcp:101:ssh.0 00010000:00010000:2.0:1787739084.417606:0:6523:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1787739084.417635:0:6523:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af14c lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09572d expref: -99 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.417743:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af153 lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.417762:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af153 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.417775:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af153 lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.417792:0:6523:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af15a lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0x6a049044 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.417799:0:6523:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af15a lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x6a049044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.417806:0:6523:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af15a lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x6a049044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.417887:0:6523:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af153 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.417897:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af153 lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.417904:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af153 lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.417912:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af153 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.417920:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa0400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739084.417925:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af153 lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.417935:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af153 lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.417950:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:2.0:1787739084.417954:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739084.417959:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af153 lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.417967:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa0400/0x7ee4453e4a1af153 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.417984:0:6523:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af15a lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x6a049044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.417992:0:6523:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af15a lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x6a049044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.417998:0:6523:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af15a lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x6a049044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.418005:0:6523:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306aa1200 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:2.0:1787739084.418008:0:6523:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af15a lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x6a049044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.418015:0:6523:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af15a lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x6a049044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.418023:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x6a049044 (ffffa07403eeb900) 00010000:00010000:2.0:1787739084.418026:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1787739084.418029:0:6523:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af15a lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x6a049044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.418036:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1200/0x7ee4453e4a1af15a lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x6a049044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6523 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.418100:0:6523:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af14c lrc: 1/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09572d expref: 31 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.418112:0:6523:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:2.0:1787739084.418115:0:6523:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1787739084.418119:0:6523:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa07306aa1800/0x7ee4453e4a1af14c lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09572d expref: 31 pid: 6523 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787739084.418130:0:6523:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306aa1800, rc 0) 00000100:00100000:2.0:1787739084.418205:0:6523:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304ce2940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18412:x1874579652571648:12345-192.168.203.15@tcp:101:ssh.0 Request processed in 628us (1213us total) trans 0 rc 301/301 00000100:00100000:2.0:1787739084.418214:0:6523:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23769 00000100:00100000:0.0:1787739084.422315:0:3645:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:1.0:1787739084.422594:0:7944:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652571904 00000100:00100000:1.0:1787739084.422608:0:7944:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652571904 00000100:00100000:1.0:1787739084.422759:0:7944:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23770 00000100:00100000:1.0:1787739084.422771:0:7944:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304002940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18412:x1874579652571904:12345-192.168.203.15@tcp:101:ssh.0 00010000:00010000:1.0:1787739084.422801:0:7944:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1787739084.422827:0:7944:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1af161 lrc: 2/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095734 expref: -99 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.422940:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1af168 lrc: 3/1,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.422959:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1af168 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.422973:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1af168 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.422992:0:7944:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1af16f lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x6a049044 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7944 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.422999:0:7944:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1af16f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x6a049044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.423006:0:7944:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1af16f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x6a049044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.423060:0:7944:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1af168 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.423069:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1af168 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.423076:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1af168 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.423082:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1af168 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.423089:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840000 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739084.423095:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1af168 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.423104:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1af168 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.423119:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:1.0:1787739084.423123:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739084.423128:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1af168 lrc: 3/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.423135:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840000/0x7ee4453e4a1af168 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.423149:0:7944:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1af16f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x6a049044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.423156:0:7944:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1af16f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x6a049044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.423162:0:7944:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1af16f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x6a049044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.423166:0:7944:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa07306840800 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:1.0:1787739084.423169:0:7944:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1af16f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x6a049044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.423175:0:7944:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1af16f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x6a049044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.423182:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x6a049044 (ffffa0730713a700) 00010000:00010000:1.0:1787739084.423185:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1787739084.423189:0:7944:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1af16f lrc: 2/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x6a049044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.423195:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840800/0x7ee4453e4a1af16f lrc: 0/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x6a049044 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7944 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.423232:0:7944:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1af161 lrc: 1/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095734 expref: 13 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.423244:0:7944:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:1.0:1787739084.423247:0:7944:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1787739084.423250:0:7944:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa07306840a00/0x7ee4453e4a1af161 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095734 expref: 13 pid: 7944 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787739084.424081:0:7944:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa07306840a00, rc 0) 00000100:00100000:1.0:1787739084.424190:0:7944:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304002940 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18412:x1874579652571904:12345-192.168.203.15@tcp:101:ssh.0 Request processed in 1417us (1885us total) trans 0 rc 301/301 00000100:00100000:1.0:1787739084.424201:0:7944:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23770 00000100:00100000:0.0:1787739084.618312:0:3643:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739084.618389:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652572160 00000100:00100000:3.0:1787739084.618403:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652572160 00000100:00100000:3.0:1787739084.618508:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23771 00000100:00100000:3.0:1787739084.618531:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304001a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18412:x1874579652572160:12345-192.168.203.15@tcp:101:ssh.0 00010000:00010000:3.0:1787739084.618560:0:6525:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739084.618589:0:6525:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1af176 lrc: 2/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e09573b expref: -99 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.618697:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1af17d lrc: 3/1,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.618714:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1af17d lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.618726:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1af17d lrc: 3/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.618742:0:6525:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1af184 lrc: 3/1,0 mode: --/PR res: [0x200000400:0xce:0x0].0x61cc3bd9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.618749:0:6525:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1af184 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x61cc3bd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.618755:0:6525:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1af184 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x61cc3bd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.618803:0:6525:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1af17d lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.618815:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1af17d lrc: 4/1,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.618821:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1af17d lrc: 3/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.618828:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1af17d lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.618834:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739084.618839:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1af17d lrc: 4/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.618848:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1af17d lrc: 5/0,0 mode: CR/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.618876:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:3.0:1787739084.618880:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739084.618885:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1af17d lrc: 3/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.618891:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aa400/0x7ee4453e4a1af17d lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.618906:0:6525:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1af184 lrc: 3/1,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x61cc3bd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.618913:0:6525:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1af184 lrc: 2/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x61cc3bd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.618919:0:6525:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1af184 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x61cc3bd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.618925:0:6525:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739084.618927:0:6525:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1af184 lrc: 3/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x61cc3bd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.618933:0:6525:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1af184 lrc: 4/0,0 mode: PR/PR res: [0x200000400:0xce:0x0].0x61cc3bd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.618941:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x61cc3bd9 (ffffa0730608bb00) 00010000:00010000:3.0:1787739084.618943:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739084.618946:0:6525:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1af184 lrc: 2/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x61cc3bd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.618952:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069ab600/0x7ee4453e4a1af184 lrc: 0/0,0 mode: --/PR res: [0x200000400:0xce:0x0].0x61cc3bd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6525 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.618992:0:6525:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1af176 lrc: 1/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09573b expref: 31 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.619004:0:6525:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000400:0xce:0x0].0x0 (ffffa07403a6f400) 00010000:00010000:3.0:1787739084.619006:0:6525:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739084.619010:0:6525:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffffa073069aba00/0x7ee4453e4a1af176 lrc: 0/0,0 mode: --/CR res: [0x200000400:0xce:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e09573b expref: 31 pid: 6525 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.619019:0:6525:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739084.619103:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304001a40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+30:18412:x1874579652572160:12345-192.168.203.15@tcp:101:ssh.0 Request processed in 576us (804us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739084.619111:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23771 00000100:00100000:0.0:1787739084.628036:0:3644:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.15@tcp (source: 12345-192.168.203.15@tcp) 00000100:00100000:3.0:1787739084.628203:0:6524:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579652572416 00000100:00100000:3.0:1787739084.628217:0:6524:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579652572416 00000100:00100000:3.0:1787739084.628290:0:6524:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.15@tcp, seq: 23772 00000100:00100000:3.0:1787739084.628301:0:6524:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa07304000b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18412:x1874579652572416:12345-192.168.203.15@tcp:101:ssh.0 00010000:00010000:3.0:1787739084.628337:0:6524:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1787739084.628367:0:6524:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1af18b lrc: 2/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd63b75c0e095742 expref: -99 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.628499:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af192 lrc: 3/1,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.628523:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af192 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.628537:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af192 lrc: 3/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.628554:0:6524:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af199 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x61cc3bd9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6524 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.628561:0:6524:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af199 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x61cc3bd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.628567:0:6524:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af199 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x61cc3bd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.628614:0:6524:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af192 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.628621:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af192 lrc: 4/1,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.628626:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af192 lrc: 3/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.628641:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af192 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.628647:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069ab600 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739084.628652:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af192 lrc: 4/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.628660:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af192 lrc: 5/0,0 mode: CR/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.628673:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739084.628677:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739084.628682:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af192 lrc: 3/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.628688:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069ab600/0x7ee4453e4a1af192 lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.628703:0:6524:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af199 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x61cc3bd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.628710:0:6524:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af199 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x61cc3bd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.628716:0:6524:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af199 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x61cc3bd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.628721:0:6524:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa073069aa400 already unused, calling callback (ffffffffc18da4f0) 00010000:00010000:3.0:1787739084.628723:0:6524:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af199 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x61cc3bd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.628734:0:6524:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af199 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xcc:0x0].0x61cc3bd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.628741:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x61cc3bd9 (ffffa0730608bb00) 00010000:00010000:3.0:1787739084.628743:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1787739084.628745:0:6524:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af199 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x61cc3bd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.628750:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aa400/0x7ee4453e4a1af199 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xcc:0x0].0x61cc3bd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6524 initiator: MDT1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.628782:0:6524:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1af18b lrc: 1/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095742 expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.628791:0:6524:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000400:0xcc:0x0].0x0 (ffffa07403a6f500) 00010000:00010000:3.0:1787739084.628793:0:6524:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1787739084.628797:0:6524:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: ffffa073069aba00/0x7ee4453e4a1af18b lrc: 0/0,0 mode: --/CR res: [0x240000400:0xcc:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.15@tcp remote: 0xd63b75c0e095742 expref: 13 pid: 6524 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787739084.628805:0:6524:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffffa073069aba00, rc 0) 00000100:00100000:3.0:1787739084.629346:0:6524:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa07304000b40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e0b637e5-5b36-4bfe-8bfd-67e9035ae5f2+12:18412:x1874579652572416:12345-192.168.203.15@tcp:101:ssh.0 Request processed in 1041us (1320us total) trans 0 rc 301/301 00000100:00100000:3.0:1787739084.629356:0:6524:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.15@tcp, seq: 23772 00000100:00100000:2.0:1787739084.741151:0:3650:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787739084.741181:0:3650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665806336:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1787739084.741275:0:3650:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1787739084.741923:0:6529:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665806336 00000100:00100000:0.0:1787739084.741935:0:6529:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665806336 00000100:00100000:0.0:1787739084.742010:0:6529:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3740 00000100:00100000:0.0:1787739084.742022:0:6529:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074156efc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+14:3650:x1874579665806336:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1787739084.742073:0:6529:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1445117952 avail 1298878464 00000100:00100000:0.0:1787739084.742160:0:6529:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074156efc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+14:3650:x1874579665806336:12345-0@lo:41:osp-pre-1-0.0 Request processed in 140us (885us total) trans 0 rc 0/0 00000100:00100000:0.0:1787739084.742168:0:6529:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3740 00000100:00100000:2.0:1787739084.742681:0:3650:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 4294968494 gen 2 00000004:00080000:2.0:1787739084.742689:0:3650:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968489 committed transno 4294968494 00000004:00080000:2.0:1787739084.742715:0:3650:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968490 committed transno 4294968494 00000004:00080000:2.0:1787739084.742728:0:3650:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968491 committed transno 4294968494 00000004:00080000:2.0:1787739084.742742:0:3650:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968492 committed transno 4294968494 00000004:00080000:2.0:1787739084.742755:0:3650:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968493 committed transno 4294968494 00000004:00080000:2.0:1787739084.742767:0:3650:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294968494 committed transno 4294968494 00000100:00100000:2.0:1787739084.742790:0:3650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3650:1874579665806336:0@lo:41:osp-pre-1-0.0 00010000:00010000:2.0:1787739085.122930:0:6511:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787739085.252181:0:3652:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa074156ef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665806464:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1787739085.252439:0:3652:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1787739085.252840:0:12410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874579665806464 00000100:00100000:2.0:1787739085.252851:0:12410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874579665806464 00000100:00100000:2.0:1787739085.252927:0:12410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3741 00000100:00100000:2.0:1787739085.252948:0:12410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa074356ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+17:3652:x1874579665806464:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1787739085.253003:0:12410:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1444077568 avail 1297838080 00000100:00100000:2.0:1787739085.253070:0:12410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa074356ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+17:3652:x1874579665806464:12345-0@lo:41:osp-pre-0-1.0 Request processed in 133us (660us total) trans 0 rc 0/0 00000100:00100000:2.0:1787739085.253078:0:12410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3741 00000100:00100000:3.0:1787739085.253184:0:3652:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa074156ef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3652:1874579665806464:0@lo:41:osp-pre-0-1.0 00000001:02000400:0.0:1787739085.291334:0:20978:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity test_60h: @@@@@@ FAIL: 0 data mismatch 00010000:00010000:1.0:1787739086.148402:0:10339:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 68099 lines, 68099 kept, 0 dropped, 0 bad.